I'm currently facing a small challenge and don't q...

# ktor

[FloP]

I'm currently facing a small challenge and don't quite know what to do.
I want to stream the download of a file from a remote server directly to a client. So I don't want to download the complete file first and then pass the downloaded file on to the client.
This works quite well in most cases. However, the problem is that if the client's connection is not the best, the remote server interrupts the connection to my ktor server and the whole download fails.
Does anyone have any suggestions on what to do here? I may have considered temporarily downloading the remote file and streaming the first chunks to the client during the download.
This is my current code:

suspend fun streamDownload(
		httpClient: HttpClient,
		call: ApplicationCall,
		remoteDownloadUrl: String
	) {
		try {
			httpClient.prepareGet(remoteDownloadUrl).execute { response ->
				val channel = response.bodyAsChannel()

				call.respondOutputStream(
					contentType = ContentType.Application.Zip
				) {
                    try {
                        while (!channel.isClosedForRead) {
                            val packet = channel.readRemaining(DEFAULT_BUFFER_SIZE.toLong())
                            while (!packet.isEmpty) {
                                val bytes = packet.readBytes()
                                this.write(bytes)
                            }
                        }
                    } catch (e: Exception) {
                       println(e)
                    }
					
				}
			}
		} catch (e: Exception) {
			 println(e)
		}
	}

[Aleksei Tirman [JB]]

You can use the ApplicationCall.respondBytesWriter method to get a ByteWriteChannel and use it for copying the data from the ByteReadChannel . Here is a simplified code:

suspend fun streamDownload(httpClient: HttpClient, call: ApplicationCall, remoteDownloadUrl: String) {
    try {
        httpClient.prepareGet(remoteDownloadUrl).execute { response ->
            val channel = response.bodyAsChannel()

            call.respondBytesWriter(contentType = ContentType.Application.Zip) {
                channel.copyTo(this)
            }
        }
    } catch (e: Exception) {
        println(e)
    }
}

[FloP]

Thank you. This greatly simplifies my code, but does not solve my basic problem. It is that the remote server itself has a timeout when sending. This is actually not a problem in the communication between the ktor application and the remote server, as there is a stable and fast connection. The problem, however, is that the speed between the remote server and the ktor application is throttled by the client. If the client does not have a good Internet connection, the remote server times out, closing the ByteReadChannel and causing the download to fail on the client side.

[Aleksei Tirman [JB]]

Does the remote server time out because of the delay between the reads?

[FloP]

Seems like that

[FloP]

The exception I see is java.io.EOFException: Chunked stream has ended unexpectedly: no chunk size

[Aleksei Tirman [JB]]

Can you try to use the OkHttp engine instead of the CIO for the client?

[FloP]

I will try

[FloP]

Alright. That's my implementation using OkHttp

suspend fun streamDownload(
		call: ApplicationCall,
		remoteDownloadUrl: String
	) {
		val okHttpClient = OkHttpClient()

		val request = Request.Builder()
			.url(remoteDownloadUrl)
			.build()

		okHttpClient.newCall(request).execute().use { response ->
			if(!response.isSuccessful) {
				println("Request unsuccessful")
				return
			}

			val channel = response.body!!.byteStream().toByteReadChannel()

			call.respondBytesWriter(contentType = ContentType.Application.Zip) {
				channel.copyTo(this)
			}
		}

	}

Now I see the following exception if client connection is throttled: okhttp3.internal.http2.StreamResetException: stream was reset: CANCEL
Without a throttled connection everything is fine ...

[FloP]

So same behaviour

[Aleksei Tirman [JB]]

Can you also try configuring the ping interval on the OkHttp client instance? The third example in the documentation illustrates how to use a preconfigured OkHTTP client instance.