https://kotlinlang.org logo
#ktor
Title
# ktor
s

Satyam Agarwal

03/21/2023, 12:40 PM
Hei Ktor team. • We have seen this issue since 1.6.x, and have been bitten by it pretty much every other day in production. • This issue only occurs on apis where response header has transfer-enconding=chunked and without content length specified. • As soon as this happens, it poisons the whole ktor client and only way it comes back to normal is by restarting our servers. • We have seen this issue being raised in youtrack and marked as solved, but we still see it. we are using 2.2.2 as current ktor client version, with CIO as http client engine. • We have tried removing different http client plugins to eliminate points of failure, but it keeps happening. • We have also tried using Apache engine in the ktor HttpClient, it still happens. Problem is, we have never been successful in reproducing this issue locally. Can you look at the stack trace and give us some clue, eventually, identify if something needs to be done in the ktor client libraries ?
Copy code
java.io.EOFException: Invalid chunk: content block of size <8192| multiple of 8> ended unexpectedly
	at io.ktor.http.cio.ChunkedTransferEncodingKt.decodeChunked(ChunkedTransferEncoding.kt:93)
	at io.ktor.http.cio.ChunkedTransferEncodingKt$decodeChunked$3.invokeSuspend(ChunkedTransferEncoding.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
	Suppressed: java.io.EOFException: Invalid chunk: content block of size 8192 ended unexpectedly
		at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
		at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
		at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
		at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
		at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
		at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$3.invoke(ExceptionUtilsJvm.kt:103)
		at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$3.invoke(ExceptionUtilsJvm.kt:90)
		at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:66)
		at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:2400)
		at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
		at io.ktor.utils.io.ByteBufferChannel.setupStateForRead(ByteBufferChannel.kt:295)
		at io.ktor.utils.io.ByteBufferChannel.copyDirect$ktor_io(ByteBufferChannel.kt:3377)
		at io.ktor.utils.io.ByteBufferChannel$copyDirect$1.invokeSuspend(ByteBufferChannel.kt)
		... 8 more
	Caused by: [CIRCULAR REFERENCE: java.io.EOFException: Invalid chunk: content block of size 8192 ended unexpectedly]
Another stacktrace I see on the same issue
Copy code
java.io.EOFException: Invalid chunk: content block of size 8192 ended unexpectedly
	at io.ktor.http.cio.ChunkedTransferEncodingKt.decodeChunked(ChunkedTransferEncoding.kt:93)
	at io.ktor.http.cio.ChunkedTransferEncodingKt$decodeChunked$3.invokeSuspend(ChunkedTransferEncoding.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
@Aleksei Tirman [JB]
s

Sam

03/21/2023, 12:43 PM
👀 Watching this thread keenly as we have seen a similar issue and never been able to reproduce it or track it down to a root cause
@Satyam Agarwal do you have an example of the stacktrace when using Apache? So I can compare against the issue I’ve been seeing. In our application the symptom is normally
org.apache.http.TruncatedChunkException
.
s

Satyam Agarwal

03/21/2023, 12:47 PM
Our logs get purged, but as far as I remember, the stacktrace was very consistent with the one posted above (excluding suppressed exception) Exception always come from the ktor io utils library, so thats why I wasnt surprised that both Apache and CIO engine suffered from the same issue
🙏 1
🆗 1
s

Sam

03/21/2023, 12:48 PM
Perhaps it’s not the same as my issue 😞
s

Satyam Agarwal

03/23/2023, 8:58 AM
@Aleksei Tirman [JB] @e5l Can you please help ? 😊
e

e5l

03/23/2023, 9:13 AM
Could you log an issue in YT? We can take a look. There is a similar issue recently fixed in the 2.2.3 release. So it would be great if you check
2.2.4
as well
20 Views