https://kotlinlang.org logo
Join Slack
Powered by
We are using SQLDelight with Coroutines in a Kotli...
# multiplatform
t
We are using SQLDelight with Coroutines in a Kotlin Multiplatform project. Hope this is the right place to ask. We couldn't really fully grasp the limitations on Kotlin Native. Our iOS team has pending questions regarding performance and whether if we will actually be doing database I/O operation on main thread. Can someone enlighten me and point me to right direction? Thanks.
👀 1
k
coroutines are executed on the main thread in the K/N model
that's about as much as I can say. i haven't had to solve this problem yet. curious what others are doing.
k
I wrote most of the driver, so I’ll jump in.
You cannot use Flow out of the box with sqldelight and native.
Currently, doing concurrency on native either means not coroutines, if multiple threads. You can suspend/resume to the same thread.
In general, the native sqlite driver in on par with Android, performance-wise, but can be 3x-5x faster if we ever actually merge some performance tweaks. This is presumably due to the lack of jni. However, I’d say main thread sqlite is in general to be avoided.
It would be easier to discuss a more concrete sample of what you’d like to do.
On Droidcon app, for example, we have a simple reactive pub/sub to listen to changes. When the db changes, part of the transaction commit in sqldelight pings registered listeners, in the thread that the update happened on. That eventually winds up push data to the main thread. That is not in coroutines, however.
For today, you’ll ultimately have to write something custom or use a different library.
k
i assume you cannot share a DB connection between threads?
k
You can. You must, really.
k
how does that work? the object doesn't have any mutable state?
k
Any mutable state is contained by atomic ref
k
gotcha
well that doesn't sound so bad then
t
So in this particular scenario we are actually dumping logger data. And do not actively listen to changes.
It's just that logging might happen frequently. We just want to store those in sqlite without blocking UI.
k
Copy code
expect fun sendLog(l:LogData)

//Actually save to db
internal fun dbLog(l:LogData){
  db.logQueries.insertLog(l)
}

//iOS
val worker = Worker.start()

actual fun sendLog(l:LogData){
  worker.execute(TransferMode.SAFE, {l.freeze()}){
    dbLog(it)
  }
}

//Android
//Use coroutines, Executor service, whatever you'd normally do
actual fun sendLog(l:LogData){
  //Android concurrency thing...
  {
    dbLog(it)
  }
}
LogData
is whatever your log data is, obv
If you want to get somewhat more complex, we have a suspending fun with background lambda here: https://github.com/touchlab/DroidconKotlin/blob/master/sessionize/lib/src/commonMain/kotlin/co/touchlab/sessionize/platform/Functions.kt#L13
m
Hm, cool, thanks guys. Thanks @taso for asking this question. Also thanks @kpgalligan for answering with this much details. What I actually did at the end is something like this:
Copy code
internal actual fun isPossibleToRunCoroutineInThisThread() = NSThread.isMainThread

private fun logAsync(
            loggers: AtomicCollection<LoggerContract>,
            log: (logger: LoggerContract) -> Unit
        ) {
            if (isPossibleToRunCoroutineInThisThread()) {
                launch {
                    loggers.forEach {
                        log(it)
                    }
                }
            } else {
                loggers.forEach {
                    log(it)
                }
            }
        }
I will definitely rethink it now. Especially, because I learned that 
NSThread.isMainThread
is not 100% reliable.
k
isMainThread isn't reliable? Under what circumstances?
m
Hm, not sure. I got that feedback from ios engineer. Also, I read few stackoverflow answers claiming the same 
NSThread.isMainThread() is not reliable because in rare cases the main queue blocks, and GCD reuses the main thread to execute other queues.
Also, this one http://blog.benjamin-encz.de/post/main-queue-vs-main-thread/ To be honest didn’t investigate deeper, I just took that for granted.
k
Well, in your case, it'll still return true if you're in the main thread. Kotlins runtime doesn't care if it's not the main queue. I would just create a worker for this stuff and forget coroutines. As presented, the DB of will still run on the main thread, even if delayed
m
In my case it works, it just feels a bit strange. Thanks for all the inputs. I will definitely look at it in the coming weeks.
k
What’s AtomicCollection, btw?
The worker version is super simple. It would look like this. Common 
Logger.kt
Copy code
typealias AtomicCollection<T> = Collection<T>
interface LoggerContract{}

internal expect object Logger{
    internal fun logAsync(
        loggers: AtomicCollection<LoggerContract>,
        log: (logger: LoggerContract) -> Unit
    )
}
On the native side…
Copy code
import kotlin.native.concurrent.TransferMode
import kotlin.native.concurrent.Worker
import kotlin.native.concurrent.freeze

internal actual object Logger {
    private val worker = Worker.start()

    internal actual fun logAsync(
        loggers: AtomicCollection<LoggerContract>,
        log: (logger: LoggerContract) -> Unit
    ) {
        worker.execute(TransferMode.SAFE, {Pair(loggers, log).freeze()}){p ->
            p.first.forEach {
                p.second(it)
            }
        }
    }
}
For jvm, an executor service (or whatever). That’s it. Don’t need to worry about threads.
m
Cool, thanks.
AtomicCollection, actually your library 
Stately
helped me to understand how freeze woks for native
so I just simplified it for my needs: JVM:
Copy code
actual class AtomicCollection<T> {

    private val atomCollection = CopyOnWriteArrayList<T>()

    actual fun clear() = atomCollection.clear()

    actual fun add(element: T) = atomCollection.add(element)

    actual fun forEach(action: (T) -> Unit) {
        for (element in atomCollection) action(element)
    }

}
Native:
Copy code
actual class AtomicCollection<T> {

    private val atomCollection = AtomicReference<List<T>>(ArrayList<T>().freeze())
    @UseExperimental(InternalAPI::class)
    private val lock = Lock()

    actual fun clear() = modifyList { it.clear() }

    actual fun add(element: T): Boolean = modifyList { it.add(element) }

    actual fun forEach(action: (T) -> Unit) {
        for (element in atomCollection.value) action(element)
    }

    @UseExperimental(InternalAPI::class)
    private inline fun <R> modifyList(proc: (MutableList<T>) -> R): R {

        lock.lock()
        try {
            val mutableList = ArrayList(atomCollection.value)
            val result = proc(mutableList)
            atomCollection.value = mutableList.freeze()

            return result
        } finally {
            lock.unlock()
        }
    }
}
I used 
Lock
from ktor library
k
Hmm. I need to look at Lock there. Several libraries recreate lock (Stately included). There’s a major rethink of Stately coming. I pulled collections out to a separate module as there needs to be an entirely different impl. Waiting on more clarity on concurrency, though.
👍 1
a
Just in case, a Lock needs to be destroyed at the end in Kotlin Native to avoid memory leaks.
t
This talk might be interesting as it explains some of the limitations of coroutines in Kotlin/Native and how the above mentioned 
coroutineworker
library works around it: https://www.droidcon.com/media-detail?video=362742333 tldr; on the JVM everything works as expected and on Kotlin/Native they launch a coroutine on a worker thread and poll from the caller thread until the work is done.
If you’re interested in the result, for your use case you could the just launch a coroutine on a worker thread to write the log to the DB
m
Hey, thanks. Actually I already changed the implementation based on Workers and ti looks good. The only problem is unit testing. Is there a way to force sync behavior?
k
Post code? I did this with an interface in the droidcon app. The test version will just run sync without a worker. The prod version actually moves around threads: https://github.com/touchlab/DroidconKotlin/blob/master/sessionize/lib/src/commonMain/kotlin/co/touchlab/sessionize/platform/Concurrent.kt
m
cool, thanks. 👍
422 Views
Open in Slack
PreviousNext
Powered by