https://kotlinlang.org logo
Join Slack
Powered by
Im looking into a potential race condition with Sq...
# squarelibraries
z
Im looking into a potential race condition with SqlDelight:
Copy code
private val database by lazy {
    scope.async { init() }
}

suspend fun database() = database.await()
The 
suspend fun init()
function creates the actual database and makes sure some template data is inserted the first time around. The whole point is that this happens before other areas of my app can touch the database. Ive been doing this for years and it has worked brilliantly, but recently something seems to have changed: under some scenario, the database is accessed before the template data is inserted. Ill add some more details in 🧵 in order to keep this somewhat short.
The coroutineScope is tied to the application lifecycle + IO.
The specific scenario where Ive been able to somewhat recreate this is in response to firebase RTDB syncing some data directly on app launch. In this case, the template data already exists in the app. I can 100% verify that.
And heres what I can observe when the app is in this state: When I open up the app, normally data is loaded pretty much instantaneously. In this case however - perhaps 500 ms pass by w/o any data showing up in the UI, then the crash happens.
The query:
Copy code
find:
SELECT *
FROM muscleSql
WHERE type=?;
The call-site:
Copy code
val row =
    muscleQueries.find(type).executeAsOneOrNull()
        ?: error("No model found by type: $type")
I feel like this shouldnt be possible. Thats the reason why Im posting here - my only other suspicion is that somehow, the query fails, despite the underlying data being there?
e
What makes you think this is a SqlDelight issue? Did you update SqlDelight and then this started happening?
z
I feel like this shouldnt be possible. Thats the reason why Im posting here - my only other suspicion is that somehow, the query fails, despite the underlying data being there?
This, I have also upgraded sqldelight to its latest version since a few weeks back (ish). Ive also had the same logic running and working for several years before this.
e
If all of your invariants are correct, then my first guess would be that a new database is being created for some reason. However without more code it's just guessing. Can you share your driver setup code (assuming that's what the 
init
function is doing)?
z
Copy code
actual class SqlEnvironment(
    private val context: Context,
) {
    actual fun driver(
        name: String,
        schema: SqlSchema<Value<Unit>>,
    ): SqlDriver = AndroidSqliteDriver(schema, context, name)
}
The class w/ the database is injected using koin 
single { .. }
and the firebase logic is all run in the same process as the rest of the app.
e
Is 
name
constant or is it derived from something? Do you have a consistent repro for this?
z
Name is always the same, I only include it as a parameter so that I can pass it in from commonMain 🙂 I dont think I can create a viable repro for this, my project is large and it would be very hard to extract all the logic into a separate project. I can provide more details or code if that helps though.
e
By consistent repro I mean can you get the crash to happen reliably
z
Not really. Ive had it happen 26 across 4 different releases though (in a timespan of maybe 1 week)
e
OK and to clarify, 
init
creates the driver and database, and then inserts template data if it doesn't already exist, and then returns the database?
z
Yes, exactly
e
And when the crash occurs, the template data was already inserted in a previous run of the app and it is no longer present in the database, or the issue is that it hasn't been inserted yet, and somehow other database code is running before (or concurrently with) 
init
?
z
Im 100% certain that the data is inserted into the database at that point. In order to sign into firebase, I gotta head to the apps settings page; and there are at least a couple of queries happening throughout that (which would then call the database). So either the data is not accessible despite the previous insert (I never delete any of this data) or what you mention last: db code running before/conc with init.
And although I cant verify this, Im pretty sure that when the crash has happened, if I were to just stop trying to sync with firebase, then the template data would be available/shown throughout the app..
e
Do you have WAL enabled, and are you using explicit transactions?
z
No WAL. By explicit transactions, do you mean 
database.transaction { .. }
? If so - yes, all edits to the database happen in this way.
e
And all of your database access goes through 
suspend fun database()
?
z
The database is accessed either: 1. Inside the scope.async block (where its created) 2. Or through the suspend fun
e
So 
muscleQueries.find
is being called on an instance returned from 
database()
?
z
Yes
e
Are the templates inserted via schema creation / migration, or after the database instance is created you use SqlDelight to check for their existence and insert them if necessary?
z
I check for their existence w/ SqlDelight
e
The simplest explanation is that 
type
is not actually present (either it wasn't part of the data that was inserted, the insert failed, something removed it, etc...) when 
find
runs. I can't think of how SqlDelight would cause this, but it's hard to say without seeing the whole picture.
z
Could the entry be removed from the db even if I dont have any .delete (or similar) operations declared in my .SQ file? Since this is template data, I specifically never touch it outside of the init block; and the 
type
specifically points to one of the template ones.
Just thinking out loud here, but could the db somehow be corrupted if the app crashes while 1 or more SQL transactions are happening?
Although Im pretty sure the initial crash (after signing into firebase) is the same one Im seeing on app launch, whenever this scenario shows up
Maybe noteworthy as well: the missing data above, is never touched by firebase
But the crash has only ever happened in connection with the sync in firebase 😅
At least, I have no way of programatically deleting these entries. I insert them, and then I never touch them.
By touch I mean edit, of course I query them!
e
Are the templates hardcoded strings? Are you 100% sure that the 
type
you query for was present before calling 
find
? While corruption is technically possible, it would be very very unlikely.
z
Yes, 100% sure.
This is a bit outside of the scope of SqlDelight, but my only other idea for now is that in one way or another theres another process accessing the db (from firebase) and that in turn causes some rare race condition. But I dont think that should be possible either, I just dont know 100% what firebase does under the hood for all of their operations. If its a different process, a separate koin would be created, and a duplicate database, I suppose?
If anything else comes to mind, let me know.. Ill leave this be for now, and the next time I run into it Ill try to capture some more context!
e
I don't think the RTDB would start a new process, but I haven't used it in over 10 years so who knows 😅 I've had similar issues in the past where there were multiple Application instances related to app backup and restore.
z
Yeah, Id be surprised if that was the case too. But say if backup & restore were to launch the app in a separate process, would that potentially cause the db 
find
call to fail? Ive ran through the logic a billion times today without it ever crashing. But, I hate that I cant figure out what was causing it 😅
e
I don't remember the behavior if multiple processes access the same database, but I imagine it would throw an exception if there was an issue rather than cause the data to not be there. Let me know if you figure it out.
🤞🏽 1
2 Views
Open in Slack
PreviousNext
Powered by