sometimes my app throws one of these stacktraces without any

Question

sometimes my app throws one of these stacktraces without any references to anywhere in my code how would i even go about debugging this thanks `java lang NegativeArraySizeException 1537060379` `at core StringsKt readBytes Strings kt 165 ` `at core StringsKt readBytes$default Strings kt 162 ` `at io ktor client call SavedCallKt save SavedCall kt 73 ` `at io ktor client call SavedCallKt$save$1 invokeSuspend SavedCall kt ` `at kotlin coroutines jvm internal BaseContinuationImpl resumeWith ContinuationImpl kt 33 ` `at kotlinx coroutines DispatchedTask run DispatchedTask kt 106 ` `at kotlinx coroutines internal LimitedDispatcher run LimitedDispatcher kt 39 ` `at kotlinx coroutines scheduling TaskImpl run Tasks kt 95 ` `at kotlinx coroutines scheduling CoroutineScheduler runSafely CoroutineScheduler kt 571 ` `at kotlinx coroutines scheduling CoroutineScheduler$Worker executeTask CoroutineScheduler kt 750 ` `at kotlinx coroutines scheduling CoroutineScheduler$Worker runWorker CoroutineScheduler kt 678 ` `at kotlinx coroutines scheduling CoroutineScheduler$Worker run CoroutineScheduler kt 665 `

Sam · Answer

`NegativeArraySizeException` typically indicates an overflow i e something tried to allocate an array which is bigger than `Int MAX VALUE`

Sam · Answer

If this is happening when making an HTTP call it probably means that the response body is too big to fit in a byte array scream

Sam · Answer

I m not sure whether there s a way to get a better stacktrace and find out which specific call in your code is causing the problem

Dazai · Answer

in that case its probably definitely this function but i don t see how it can create a byte array thats too big ```suspend fun downloadFile saveLocation File fileUrl String retries Int = 2 delay Long = Random nextLong 2000 5000 overwrite Boolean = false Boolean if retries lt 0 return false if saveLocation exists amp amp overwrite return true if fileUrl == null return true saveLocation delete if saveLocation createNewFile return false try logger info Attempting to download $fileUrl $retries retries remaining val response HttpResponse = client get fileUrl val channel = response bodyAsChannel val byteBufferSize = 128000 while channel availableForRead gt 0 || channel isClosedForRead if channel availableForRead lt byteBufferSize saveLocation appendBytes channel readRemaining readBytes else saveLocation appendBytes channel readPacket byteBufferSize readBytes return true catch e Exception e printStackTrace delay delay return downloadFile saveLocation fileUrl retries 1 ```

Dazai · Answer

also any tips on breaking this apart is the biggest function in my project

Dazai · Answer

so i m not surprised that its probably the culprit

Sam · Answer

From the stack trace you shared it looks like the problem is happening inside ktor while downloading the response The stack trace mentions `SavedCall` which indicates to me that ktor is downloading the entire response into memory rather than streaming it

Sam · Answer

In your example you re using `client get`

Sam · Answer

Be aware that that will download the entire response before returning So even if it looks like you re streaming from a channel you re actually just reading data that s already been download into memory

Sam · Answer

Assuming this is ktor 2 you can use `client prepareGet execute ` to stream the response instead where the code to consume the response goes inside the `execute` block

Sam · Answer

In theory that will let you download things that are bigger than `Int MAX VALUE` bytes

Sam · Answer

Docs <https ktor io docs response html streaming>

christophsturm · Answer

you can also stream in ktor1 by doing `httpClient get lt HttpStatement gt `

Dazai · Answer

oh cool that makes sense thanks for the help I didn t realize it was loading it all into memory