https://kotlinlang.org logo
Join Slack
Powered by
Hello, does anyone know whether S3Bucket.copyObjec...
# http4k
m
Hello, does anyone know whether S3Bucket.copyObject should support UTF-8? AWS itself does. When I try to use it with a source/destination bucket containing a UTF-8 char, it throws an IllegalArgumentException: Unexpected char (e.g. char 0x301, U+0301).
🤔 1
a
By default, the s3 sdk uses dns bucket routing, and so bucket names would need to be DNS-compliant. I would guess DNS doesn't support UTF-8. SO my suggestion would be to try the path-based routing. The SDK does have auto-detection for this, but it's definitely not checking for UTF-8 characters, so you can force it with
Copy code
S3Bucket.Http(
  ...,
  forcePathStyle = true
)
m
Thank you for the quick reply, I will try that. What I've also found out, is that copyObject sets the destination path as header 
CopyObject.kt#31
which is validated and then that particular error is thrown in: 
OkHttp3 Headers.kt#checkValue
a
I see it setting the source path as a header. Not the destination though. 🤔
m
My bad, you are correct. Still, that is also an issue for copying files 😕
a
Right. Well, let's see if@dave has any input. But I suspect you've found a bug.
🙏 1
d
I've got no input on it I'm afraid. Of course, the beauty of connect is that you can just create your own version of the action to test it out really easily. If there's a fix to come back then please submit it 🙂
m
I understand this is tricky, as it is the underlying OkHttp3 client that is throwing the error and the AWS REST API cannot be harnessed to its full extent this way since it intends HTTP headers for this operation compared to the AWS SDK. I've raised a GitHub issue for now and I'll have a look at it at some point - for now, my workaround is to get and put the file in such cases. Thank you for checking it out.
a
It would be worth seeing if even the official java SDK can do it without error.
m
It does as this is where I came from and now I try to recreate the same functionality with http4k-s3 Another info that was found is, that when I URL-encode the source as the AWS documentation specifies, then it results in a 
SignatureDoesNotMatchError
d
Tbh, the easiest way to diagnose this is to get a gist to recreate the problem.
m
Thank you for your time. URL-encoding just the source bucket key did the trick. I would consider this resolved now, however, it would be helpful to either always include this encoding in the library as the AWS documentation suggests or at least add it to the http4k documentation/reference as well
a
It seems like there's two improvements we can make: 1. Automatically switch to path-style URIs if dest bucket name isn't DNS-compliant (beyond simply checking for 
.
) 2. Always url-encode 
source
for the 
copyObject
operation, and any others that use the bucket name in a header (I imagine if the bucket name was already DNS compliant, it wouldn't be affected by url-encoding)
Glad you found a solution
On that note, I'm surprised the 
Request.header
method doesn't automatically url-encode, which differs from the behaviour I notice in 
Request.query
👀 1
m
Interesting. Good points!
Open in Slack
PreviousNext
Powered by