I'm getting an error when trying to stream a txt f...

# squarelibraries

[Colton Idle]

I'm getting an error when trying to stream a txt file from Okhttp (no retrofit) and putting it into sqlDelight. It works like 99% of the time, but sometimes I get an unexpected end of stream
error. This is roughly my code that takes the bufferedReader from okhttp. Anything look obviously bad here?

suspend fun deleteAllThenInsertAllLines(bufferedReader: BufferedReader) =
withContext(<http://Dispatchers.IO|Dispatchers.IO>) {
    myQueries.transaction {
      myQueries.deleteAll()
      var currentLine: String? = null
      try {
        while (bufferedReader.readLine().also { currentLine = it } != null) {
          myQueries.insert(currentLine)
        }
      } catch (e: Exception) {
        printlin("Exception! Failed on $currentLine with ${e.message}")
      }
    }
  }

[Daniel Perez]

I can't say I know for certain why this is happening because I/O is something I have little experience with, but you might try using `Okio`'s Source and Sink API's.

My understanding of Okio is that it's supposed to be a better, safer, and faster wrapper on top of <http://java.io|java.io> API's and it's something OkHttp depends on already

Definitely look at it first though, I don't want to just recommended using a tool that may just end up being overkill for a simple fix.

[ephemient]

from a purely code review standpoint, .useLines {} is preferred if possible. .readlnOrNull() is a better name for .readLine() anyhow

[ephemient]

but that's not related to the exception. that does come up with OkHttp from time to time, most often caused by connection reuse

[ephemient]

it's not okhttp's fault, it's just something that can happen asynchronously between client and server

[Colton Idle]

I think I had issues with some of the "better" apis for reading lines.
lemme find that convo... https://kotlinlang.slack.com/archives/C1CFAFJSK/p1694110868583289

[Colton Idle]

and thanks. yeah it seems like just something is dying between client and server... sometimes. the file has over 100k lines... so it takes a while to stream it i suppose. i wonder if it'd be better to not stream it with a buffered reader.

[ephemient]

that's not a problem with useLines,

bufferedReader.useLines { lines ->
    for (line in lines) {
        // can suspend

[ephemient]

or linesSequence() if you really don't want to close it for some reason…

[Colton Idle]

I wonder if I need Chunked transfer encoding

https://stackoverflow.com/a/58217819

[ephemient]

T-E:chunked is useful if the server doesn't know the size of the content before sending it, but otherwise not. also it's completely gone in HTTP/2

[ephemient]

https://github.com/square/okhttp/issues/2738 is what I meant earlier