Hi folks. I want to upload a huge multipart file t...

# ktor

[Mikhail]

Hi folks.
I want to upload a huge multipart file to my server via Ktor Client. Any ideas or tips?
The file is considered to be larger than 2GB.

[FunkyMuse]

Apache file upload

[Mikhail]

Well, right now I'm going to try to do that with Fuel
https://fuel.gitbook.io/documentation/
I'll write later

[Todd]

You could use an input stream

val request = client.prepareFormWithBinaryData(
    url = "<https://example.com>",
    formData = formData {
        append(key = "file", InputProvider(size) { inputStream.asInput() }, headers = Headers.build {
            append(HttpHeaders.ContentType, contentType)
            append(HttpHeaders.ContentDisposition, "filename=\"$fileName\"")
        })
    }
) { }

[ephemient]

I don't know how much control you have over the server, but for such large uploads, I would strongly recommend that it provide some mechanism that is able to create an upload session, upload individual chunks with retries, and the finish the session, as separate HTTP calls. unfortunately there is no universal standard, but there are some published conventions you can follow, such as S3, Google, Box, and ownCloud

[ephemient]

or simply have the server provide the client with a temporary S3 object to upload to, and report back when it's done. then you can use existing libraries to perform the bulk of the action

[Mikhail]

@ephemient
server is mine and written in Spring. Recently I successfully uploaded 21GB object on it with Curl.
Also, I can't use S3. AWS doesn't provide their services for russians anymore.

[ephemient]

there's other S3-compatible services, but even if you can't or don't use any of them, I would still recommend a resumable upload so the user does not have to re-upload the whole file if their wifi drops at 20GB out of 21

[Mikhail]

Yeah, I think it will be cool to have such a feature implemented. Thank you

[FunkyMuse]

@Mikhail maybe some CDN can help?

[Mikhail]

No. My project is open-source and is my pet-project: https://github.com/hole-project
I don't want to rely on any other service

[Mikhail]

And yeah, I implemented uploading on the client.
Everything is working perfectly fine.
https://github.com/hole-project/hole-kt/blob/7e669c7fd943ef20d179a51244d3d4e60d3fc[…]src/main/kotlin/dev/d1s/hole/client/core/impl/HoleClientImpl.kt

[Aleksei Tirman [JB]]

On JVM you can get a ByteReadChannel for a java.io.File and then use a ChannelProvider to send a binary part:

val client = HttpClient(OkHttp) {}

val file = File("garbage.zip")
val r = <http://client.post|client.post>("<https://httpbin.org/post>") {
    setBody(MultiPartFormDataContent(formData {
        append("data", ChannelProvider(file.length()) { file.readChannel() })
    }))
}

println(r.bodyAsText())