https://kotlinlang.org logo
#multiplatform
Title
# multiplatform
t

taso

10/17/2019, 6:48 PM
We are using SQLDelight with Coroutines in a Kotlin Multiplatform project. Hope this is the right place to ask. We couldn't really fully grasp the limitations on Kotlin Native. Our iOS team has pending questions regarding performance and whether if we will actually be doing database I/O operation on main thread. Can someone enlighten me and point me to right direction? Thanks.
👀 1
k

Kris Wong

10/17/2019, 6:51 PM
coroutines are executed on the main thread in the K/N model
that's about as much as I can say. i haven't had to solve this problem yet. curious what others are doing.
k

kpgalligan

10/17/2019, 6:53 PM
I wrote most of the driver, so I’ll jump in.
You cannot use Flow out of the box with sqldelight and native.
Currently, doing concurrency on native either means not coroutines, if multiple threads. You can suspend/resume to the same thread.
In general, the native sqlite driver in on par with Android, performance-wise, but can be 3x-5x faster if we ever actually merge some performance tweaks. This is presumably due to the lack of jni. However, I’d say main thread sqlite is in general to be avoided.
It would be easier to discuss a more concrete sample of what you’d like to do.
On Droidcon app, for example, we have a simple reactive pub/sub to listen to changes. When the db changes, part of the transaction commit in sqldelight pings registered listeners, in the thread that the update happened on. That eventually winds up push data to the main thread. That is not in coroutines, however.
For today, you’ll ultimately have to write something custom or use a different library.
k

Kris Wong

10/17/2019, 6:59 PM
i assume you cannot share a DB connection between threads?
k

kpgalligan

10/17/2019, 7:11 PM
You can. You must, really.
k

Kris Wong

10/17/2019, 7:55 PM
how does that work? the object doesn't have any mutable state?
k

kpgalligan

10/17/2019, 7:57 PM
Any mutable state is contained by atomic ref
k

Kris Wong

10/17/2019, 7:57 PM
gotcha
well that doesn't sound so bad then
t

taso

10/17/2019, 9:42 PM
So in this particular scenario we are actually dumping logger data. And do not actively listen to changes.
It's just that logging might happen frequently. We just want to store those in sqlite without blocking UI.
k

kpgalligan

10/17/2019, 10:00 PM
Copy code
expect fun sendLog(l:LogData)

//Actually save to db
internal fun dbLog(l:LogData){
  db.logQueries.insertLog(l)
}

//iOS
val worker = Worker.start()

actual fun sendLog(l:LogData){
  worker.execute(TransferMode.SAFE, {l.freeze()}){
    dbLog(it)
  }
}

//Android
//Use coroutines, Executor service, whatever you'd normally do
actual fun sendLog(l:LogData){
  //Android concurrency thing...
  {
    dbLog(it)
  }
}
LogData
is whatever your log data is, obv
If you want to get somewhat more complex, we have a suspending fun with background lambda here: https://github.com/touchlab/DroidconKotlin/blob/master/sessionize/lib/src/commonMain/kotlin/co/touchlab/sessionize/platform/Functions.kt#L13
m

mkojadinovic

10/18/2019, 6:51 AM
Hm, cool, thanks guys. Thanks @taso for asking this question. Also thanks @kpgalligan for answering with this much details. What I actually did at the end is something like this:
Copy code
internal actual fun isPossibleToRunCoroutineInThisThread() = NSThread.isMainThread

private fun logAsync(
            loggers: AtomicCollection<LoggerContract>,
            log: (logger: LoggerContract) -> Unit
        ) {
            if (isPossibleToRunCoroutineInThisThread()) {
                launch {
                    loggers.forEach {
                        log(it)
                    }
                }
            } else {
                loggers.forEach {
                    log(it)
                }
            }
        }
I will definitely rethink it now. Especially, because I learned that
NSThread.isMainThread
is not 100% reliable.
k

kpgalligan

10/18/2019, 1:11 PM
isMainThread isn't reliable? Under what circumstances?
m

mkojadinovic

10/18/2019, 1:17 PM
Hm, not sure. I got that feedback from ios engineer. Also, I read few stackoverflow answers claiming the same
NSThread.isMainThread() is not reliable because in rare cases the main queue blocks, and GCD reuses the main thread to execute other queues.
Also, this one http://blog.benjamin-encz.de/post/main-queue-vs-main-thread/ To be honest didn’t investigate deeper, I just took that for granted.
k

kpgalligan

10/18/2019, 1:52 PM
Well, in your case, it'll still return true if you're in the main thread. Kotlins runtime doesn't care if it's not the main queue. I would just create a worker for this stuff and forget coroutines. As presented, the DB of will still run on the main thread, even if delayed
m

mkojadinovic

10/18/2019, 2:14 PM
In my case it works, it just feels a bit strange. Thanks for all the inputs. I will definitely look at it in the coming weeks.
k

kpgalligan

10/18/2019, 2:16 PM
What’s AtomicCollection, btw?
The worker version is super simple. It would look like this. Common
Logger.kt
Copy code
typealias AtomicCollection<T> = Collection<T>
interface LoggerContract{}

internal expect object Logger{
    internal fun logAsync(
        loggers: AtomicCollection<LoggerContract>,
        log: (logger: LoggerContract) -> Unit
    )
}
On the native side…
Copy code
import kotlin.native.concurrent.TransferMode
import kotlin.native.concurrent.Worker
import kotlin.native.concurrent.freeze

internal actual object Logger {
    private val worker = Worker.start()

    internal actual fun logAsync(
        loggers: AtomicCollection<LoggerContract>,
        log: (logger: LoggerContract) -> Unit
    ) {
        worker.execute(TransferMode.SAFE, {Pair(loggers, log).freeze()}){p ->
            p.first.forEach {
                p.second(it)
            }
        }
    }
}
For jvm, an executor service (or whatever). That’s it. Don’t need to worry about threads.
m

mkojadinovic

10/18/2019, 2:23 PM
Cool, thanks.
AtomicCollection, actually your library
Stately
helped me to understand how freeze woks for native
so I just simplified it for my needs: JVM:
Copy code
actual class AtomicCollection<T> {

    private val atomCollection = CopyOnWriteArrayList<T>()

    actual fun clear() = atomCollection.clear()

    actual fun add(element: T) = atomCollection.add(element)

    actual fun forEach(action: (T) -> Unit) {
        for (element in atomCollection) action(element)
    }

}
Native:
Copy code
actual class AtomicCollection<T> {

    private val atomCollection = AtomicReference<List<T>>(ArrayList<T>().freeze())
    @UseExperimental(InternalAPI::class)
    private val lock = Lock()

    actual fun clear() = modifyList { it.clear() }

    actual fun add(element: T): Boolean = modifyList { it.add(element) }

    actual fun forEach(action: (T) -> Unit) {
        for (element in atomCollection.value) action(element)
    }

    @UseExperimental(InternalAPI::class)
    private inline fun <R> modifyList(proc: (MutableList<T>) -> R): R {

        lock.lock()
        try {
            val mutableList = ArrayList(atomCollection.value)
            val result = proc(mutableList)
            atomCollection.value = mutableList.freeze()

            return result
        } finally {
            lock.unlock()
        }
    }
}
I used
Lock
from ktor library
k

kpgalligan

10/18/2019, 2:28 PM
Hmm. I need to look at Lock there. Several libraries recreate lock (Stately included). There’s a major rethink of Stately coming. I pulled collections out to a separate module as there needs to be an entirely different impl. Waiting on more clarity on concurrency, though.
👍 1
a

Arkadii Ivanov

10/18/2019, 9:03 PM
Just in case, a Lock needs to be destroyed at the end in Kotlin Native to avoid memory leaks.
t

Tobi

10/27/2019, 2:26 PM
This talk might be interesting as it explains some of the limitations of coroutines in Kotlin/Native and how the above mentioned
coroutineworker
library works around it: https://www.droidcon.com/media-detail?video=362742333 tldr; on the JVM everything works as expected and on Kotlin/Native they launch a coroutine on a worker thread and poll from the caller thread until the work is done.
If you’re interested in the result, for your use case you could the just launch a coroutine on a worker thread to write the log to the DB
m

mkojadinovic

10/28/2019, 1:43 PM
Hey, thanks. Actually I already changed the implementation based on Workers and ti looks good. The only problem is unit testing. Is there a way to force sync behavior?
k

kpgalligan

10/28/2019, 3:07 PM
Post code? I did this with an interface in the droidcon app. The test version will just run sync without a worker. The prod version actually moves around threads: https://github.com/touchlab/DroidconKotlin/blob/master/sessionize/lib/src/commonMain/kotlin/co/touchlab/sessionize/platform/Concurrent.kt
m

mkojadinovic

10/28/2019, 3:17 PM
cool, thanks. 👍
162 Views