https://kotlinlang.org logo
#squarelibraries
Title
# squarelibraries
c

Colton Idle

02/29/2024, 10:46 PM
Trying to pinpoint an issue where I'm searching for an id that I know exists in my database, but sometimes I see our internal logs saying that the id was not found. I think I mostly chalked it up to the fact that the database is updated once a day (from a network call), and maybe during that daily update it fails/is malformed/truncated for some reason, and that causes me not being able to find it. This is me doing the network request and putting it into the db
Copy code
try {
  client.newCall(request).execute().use { response ->
    if (!response.isSuccessful) return@withContext

    val bufferedReader = BufferedReader(InputStreamReader(response.body!!.byteStream()))
    persistence.deleteAllThenInsertAll(bufferedReader)
    appStateHolder.updateLastDbSync(System.currentTimeMillis())
  }
} catch (e: Exception) {
  // Network issue
}
Does that look sus to anyone?
Copy code
suspend fun deleteAllThenInsertAll(bufferedReader: BufferedReader) =
      withContext(<http://Dispatchers.IO|Dispatchers.IO>) {
        idQueries.transaction {
          idQueries.deleteAll()
          var currentLine: String? = null
          try {
            while (bufferedReader.readLine().also { currentLine = it } != null) {
              val (cardId, date) = currentLine!!.split(",")
              idQueries.insert(cardId, date)
            }
          } catch (e: Exception) {
            //log to crashlytics
          }
        }
      }
Maybe I'm completely misunderstanding the concept of a try/catch in a transaction?
Alright I think I was able to finally write a test that proces that my try catch is actually in the wrong spot and should actually be outside of the transaction. I think that makes sense. not sure what i was initially thinking.
t

Tolriq

03/01/2024, 9:41 AM
That's why you should always be explicit about marking a transaction successful or failed.
c

Colton Idle

03/04/2024, 3:33 PM
Hm. How do I do that? let me look up that docs in a bit. the way you're saying it makes me think that I can return true or false in a transaction? if so, TIL!
transaction docs are good. should have read them before. feel like an idiot putting the try catch in the transaction lol. tolriq, if you could though, could you clarify what you mean about explicitly marking it success or failed?
t

Tolriq

03/04/2024, 4:05 PM
I have not yet migrated to sqldelight but the concept in SQL are the same everywhere, you start a transaction do a bunch of stuff then before ending the transaction you tell the engine if the transaction is successful or not so that it apply or not everything you did during the transaction.
From your extract I guess sqldelight mark the transaction successful by default if there's no throw hence your problem. But it's something to remember when dealing with transactions.
c

Colton Idle

03/04/2024, 4:13 PM
got it. yeah this is also a perfect spot for a unit test, but i didn't know how to best create a bufferedReader that would blow up in the middle. lol. which seems to be happening from this very large file that i'm download -> streaming into my table.
t

Tolriq

03/04/2024, 4:14 PM
You should probably mock the response not the bufferedreader.
c

Colton Idle

03/04/2024, 4:23 PM
Yeah, but I'm not sure (in this case) whats up with the response that just causes it to blow up.
it's supposed to be a static file with just 100k lines of data
t

Tolriq

03/04/2024, 4:25 PM
That's a lot of data, you probably want to do streamparsing.
c

Colton Idle

03/04/2024, 4:25 PM
So I do have a network try catch surrounding this transaction and that does seem to work fine. but anyway. im going to add logs. fix this bug (because the rollback doesn't actually happen) and then see if the issue happens again in the wild.
streamparsing? hm. will look that up. thanks
t

Tolriq

03/04/2024, 4:27 PM
forget that I did not read again the second snippet that's already what you do.
c

Colton Idle

03/04/2024, 4:38 PM
oh cool. well thanks for the rubberducking. plenty of things i learned with this feature/bug.
2 Views