https://kotlinlang.org logo
#ktor
Title
# ktor
m

Michael Burbidge

06/01/2020, 2:55 PM
Posting this question again, hoping someone will notice and answer: I’m working with push based HTTP API that uses chunked transfer encoding to push messages. I need access to the individual HTTP chunks while reading the response. Is there a way using the ktor HttpClient to get access to the HTTP Chunks?
e

e5l

06/01/2020, 2:57 PM
Hi @Michael Burbidge, what do you mean by chunks(chunks from chunked transfer encoding or stream liked pieces)?
m

Michael Burbidge

06/01/2020, 3:04 PM
I’m making calls against a in-house messaging system similar to Kafka. The client API for consuming messages uses chunked transfer encoding. Essentially the client makes an HTTP request, and the connection stays open. The service continues sending messages in HTTP chunks as response to the request. Messages never span chunks. I’m currently reading the data as a ByteReadChannel and calling readAvailable, but the bytes that I get back from readAvailable span the underlying http chunks and so I have to try and piece the messages back together from the previous call to readAvailable. It would be much easier if I could just read an chunk at a time as the chunks came back over the connection, because a single message is contained within a chunk.
e

e5l

06/02/2020, 7:43 AM
Unfortunately, we have no API to use chunks directly. Moreover, most of the underlying engines(except CIO) don't provide such API 😞 If you have the delimeter in your data, you can use
ByteReadChannel.readUntilDelimiter
to avoid splitting them by hand
m

Michael Burbidge

06/03/2020, 8:57 PM
Great thanks. I do have a delimiter. So I can try that. Thanks! I hadn’t noticed that call.
🙂 1
e

eenriquelopez

09/01/2022, 4:48 PM
Hi @e5l, is this still current? I am able to read chunk content, but just as a whole (not accessing each individual chunk). Ideally ktor should be able to access each individual chunk of data, but I am not sure if this is actually supported.
e

e5l

09/02/2022, 3:03 PM
Hey @eenriquelopez, could you tell me why do you need to read individual chunks?
e

eenriquelopez

09/02/2022, 3:07 PM
The use case could be an endpoint that delivers data as it generates it internally (for instance, JSON including different objects)
e

e5l

09/02/2022, 3:09 PM
Yep, you will receive all flushed data
Please note it could be splitted in chunks of different size than server flush it
3 Views