https://kotlinlang.org logo
#squarelibraries
Title
# squarelibraries
c

Colton Idle

10/24/2023, 6:21 PM
I'm getting an error when trying to stream a txt file from Okhttp (no retrofit) and putting it into sqlDelight. It works like 99% of the time, but sometimes I get an
unexpected end of stream
error. This is roughly my code that takes the bufferedReader from okhttp. Anything look obviously bad here?
Copy code
suspend fun deleteAllThenInsertAllLines(bufferedReader: BufferedReader) =
withContext(<http://Dispatchers.IO|Dispatchers.IO>) {
    myQueries.transaction {
      myQueries.deleteAll()
      var currentLine: String? = null
      try {
        while (bufferedReader.readLine().also { currentLine = it } != null) {
          myQueries.insert(currentLine)
        }
      } catch (e: Exception) {
        printlin("Exception! Failed on $currentLine with ${e.message}")
      }
    }
  }
d

Daniel Perez

10/24/2023, 6:51 PM
I can't say I know for certain why this is happening because I/O is something I have little experience with, but you might try using `Okio`'s
Source
and
Sink
API's. My understanding of Okio is that it's supposed to be a better, safer, and faster wrapper on top of
<http://java.io|java.io>
API's and it's something
OkHttp
depends on already Definitely look at it first though, I don't want to just recommended using a tool that may just end up being overkill for a simple fix.
👍 1
e

ephemient

10/24/2023, 7:16 PM
from a purely code review standpoint,
.useLines {}
is preferred if possible.
.readlnOrNull()
is a better name for
.readLine()
anyhow
👍 1
but that's not related to the exception. that does come up with OkHttp from time to time, most often caused by connection reuse
it's not okhttp's fault, it's just something that can happen asynchronously between client and server
👍 1
c

Colton Idle

10/24/2023, 7:21 PM
I think I had issues with some of the "better" apis for reading lines. lemme find that convo... https://kotlinlang.slack.com/archives/C1CFAFJSK/p1694110868583289
and thanks. yeah it seems like just something is dying between client and server... sometimes. the file has over 100k lines... so it takes a while to stream it i suppose. i wonder if it'd be better to not stream it with a buffered reader.
e

ephemient

10/24/2023, 7:26 PM
that's not a problem with
useLines
,
Copy code
bufferedReader.useLines { lines ->
    for (line in lines) {
        // can suspend
👀 1
or
linesSequence()
if you really don't want to close it for some reason…
c

Colton Idle

10/25/2023, 3:05 AM
I wonder if I need
Chunked transfer encoding
https://stackoverflow.com/a/58217819
e

ephemient

10/25/2023, 4:47 AM
T-E:chunked is useful if the server doesn't know the size of the content before sending it, but otherwise not. also it's completely gone in HTTP/2
K 1
👍 1