https://kotlinlang.org logo
Join Slack
Powered by
I was assuming coroutine support in sqldelight wou...
# squarelibraries
c
I was assuming coroutine support in sqldelight would basically just give me suspend versions of my queries and inserts. Looks like it just converts to flowable instead? Am I missing something fundamental here? The docs are kinda sparse. The flowable support is of course awesome if I want to listen to a specific query, but mostly I just want to do one shot requests in a suspendable manner (sorta like retrofit)
k
You will need a database driver that supports asyncio. I believe the Postgres driver allows asyncio. If you set up your SQLDelight config to generate async stuff you can use 
awaitAs
for example
🤔 1
c
seems like i missed some async portion of docs. let me reread
k
SQLite does not offer asyncio for interaction. All of their operations are blocking. Therefore SQLdelight does not have an async driver for SQLite. The 
Flow
extensions are based on callbacks. The items which get emitted into the flow are still executed in a blocking manner with SQLite
If you want async single shot functions you should be exposing them as a suspend function from some repository, manager, etc and wrap it in a dispacther.
Copy code
class MyRepository(private val ioContext: CoroutineContext, private val db: Database) {

  suspend fun doTheQuery() = withContext(ioContext) {
    db.someQueries.myQuery(someValue).executeAsOne()
  }
}
👀 1
d
To elaborate on what Kevin said, SQLDelight 2.0 introduces support for drivers that are fundamentally asynchronous (like web workers and R2DBC), and while it will wrap all of your queries in suspending functions, it does not magically make those calls non-blocking for synchronous drivers!
👍 2
c
Cool. I have a DbManager that is a light wrapper around my sqldelight querires and inserts essentially. Looks like I'll just have to wrap those calls with a 
withContext
TIL
👍 1
for whatever reason. i just thought maybe i just had to declare suspend somewhere lol. thinking too much into retrofit (mostly because we use it exclusively in our project)
k
What you’re desiring with suspending (non-blocking) SQLite very quickly gets into OS-level concepts on how a device actually performs IO. SQLite uses blocking system calls whereas some of the other drivers use non-blocking variants.
🤯 1
c
To elaborate on what Kevin said, SQLDelight 2.0 introduces support for drivers that are fundamentally asynchronous (like web workers and R2DBC), and while it will wrap all of your queries in suspending functions, it does not magically make those calls non-blocking for synchronous drivers!
oh. do you know if this is in some part of the docs? want to be able to point my team to it.
i can jut screenshot this convo instead. lmao
so
Copy code
generateAsync = true
If set to true, SQLDelight will generate suspending query methods for us with asynchronous drivers.
basically pointeless to enable that if I'm just writing my app on android? (and writing my tests on jvm with in memory db?)
k
It is pointless to do that with SQLite, yes.
🤯 2
👍 1
c
suppose it cant hurt though. maybe a bit misleading i suppose if i enable it and my teammates start thinking that the suspend functions are main safe.
d
oh. do you know if this is in some part of the docs? want to be able to point my team to it.
It's sort of mentioned on the migrating to 2.0 page, but it's not really emphasized. Could definitely be improved on
K 1
c
went back to read the migration page. i still feel like without this slack chat, I would have definitely kept the generateAsync. Seems like there should be some general guidance/recipe on async. Should i just file a FR for that?
👍 1
yeah. like. i just searched the sqldelight site and there doesn't seem to be any indication that SQLite is not an async driver
d
Yeah feel free to open one
a
we're intentionally not broadcasting a ton of information about the async codegen for exactly the reason Kevin is explaining
SQLite is fundamentally not asyncronous in the way that web workers or remote connections are
c
So that last sentence. Is that in the docs anywhere? Or am I missing something obvious (like does everyone know that sqlite isn't async)? All I really want is something to point my team to to be able to say we setup SQL delight incorrectly lol
k
Everyone (aside from you, Colton) who has written in this thread has at one point contributed to SQLDelight. I would guess this is domain knowledge and not universally understood. Especially because the most prominent competitor, Room, does some magic to offload queries off the main thread for you, thus removing it from your list of concerns.
I suggest that you open a documentation PR with your newly learned knowledge about SQLDelight that addressed your confusion 🙂
a
I think there is a likely future where we supply an Android driver that has its own dispatcher and then you could use the asynchronous runtime with it
👍 1
its confusing enough that I haven't wanted to document it yet in case we change our minds, but we could definitely have some less opinionated documentation that just states what the current usage should be
👍 1
(which is synchronous, and you the caller switch threads before invoking SQLDelight via Rx or withContext or whatever)
👍 1
c
thanks! like i said. for whatever reason. i just kinda figured id add some annotation or something and get suspending functions out of the box (drawing parrallels from retrofit for whatever reason) and then when i saw the coroutines extension... i thought that would do what i was originally thinking. but then the coroutines extension was just having a flow you can listen to for updates.
k
A driver which handles the IO threading stuff would be pretty neat, actually. Would get rid of a bunch of 
withContext
calls everywhere. Might not be thinking about it enough from a testing standpoint but seems cool
c
thanks for teaching everyone!
👍 1
in my head. every api (even without out of the box coroutines support) can do the whole withContext{} thing and put it on a different dispatcher (maybe im wrong about that. still feel like a coroutine noob) and so i just thought when there was specific mention of coroutine support it was specifically for flowables on queries. and not like a out of the box suspend fun for an insert.
just trying to further clarify my perspective coming into this library anew. 😄 but i learned a lot. looking forward to fixing up our codebase
k
> I think there is a likely future where we supply an Android driver that has its own dispatcher and then you could use the asynchronous runtime with it I think this would be neat. One question -- since SQLite benefits from restricting the amount of unique writer threads to 1, but can concurrently read with an unlimited amount of threads, how would a 
SqlDriver
know if a query is a write operation or not? I would think we'd want to honor those limitations in whatever internal dispatcher is used for an "async" SQLite driver?
Also, would this perhaps also benefit more than just the Android SQLite driver?
d
how would a 
SqlDriver
know if a query is a write operation or not?
I don't think it would have to. The idea behind an "async" driver for Android is that there's a (bizarrely) small cap for the number of active connections a database can have. An "async" Android driver would still block a thread when reading/writing or waiting for the write lock to release, but the idea is that if the number of connections is exhausted, the driver could suspend while waiting for connections to free up, rather than just eating up threads from a thread pool
k
Wouldn’t that possibly run into writer thread lock contention?
d
also 
SqlDriver
kind of differentiates between writes and reads through 
execute
and 
executeQuery
k
Ah yes, I suppose that’s true.
d
Wouldn’t that possibly run into writer thread lock contention?
It's possible! It's all kind of untested. There are also already some fatal flaws that have been exposed with the async runtime in SQLDelight, particularly around transaction handling, so at this point I think it's unlikely an async driver for Android will happen in 2.0 (if ever)
👍 1
k
Interesting. Do the maintainers of SQLDelight consider the async runtime to be a failure, or something that just needs bugs addressed?
d
I think it's "good enough™" to support simple js web applications which was the main motivator for it, but for other async drivers like R2DBC there are definitely issues to be addressed. The TL;DR is that SQLDelight's transaction handling is heavily dependent on thread-locals, which don't really work with coroutines when you can be dispatched onto an arbitrary thread. This could probably be addressed with some kind of 
CoroutineContext
stuff, but it would require changes to the runtime and I haven't actually done the work to see whether or not it could be done in a non-breaking manner yet. (me, being the main person behind the async runtime)
👍 2
k
That’s interesting, thanks for sharing.
➕ 1
62 Views
Open in Slack
PreviousNext
Powered by