Since coroutines on iOS can't be started on any ot...

# kotlin-native

[Prateek Grover]

Since coroutines on iOS can't be started on any other thread other than the main one, is it a good idea to still launch coroutines on the main thread, perform async operation and then again call back to main thread?

Or should iOS developers completely switch to workers for the async needs.

Why is this made in such a way? Why can't coroutines be launched from any other thread? Is the reason related to that other dispatch_queues do not have thread affinity (main queue is always connected to main thread) and that conflicts with the concurrency model of KN?

[kpgalligan]

Some history. Short answer is it's coming: https://github.com/Kotlin/kotlinx.coroutines/issues/462

[kpgalligan]

We use main thread coroutines backed by workers, and just simple workers with callbacks. Depends how you want to structure it

[Prateek Grover]

Thanks @kpgalligan I read that issue thread. Just wanted to know why was this done this way starting out. Also somewhere down the thread, this was mentioned https://github.com/Kotlin/kotlinx.coroutines/issues/462#issuecomment-429117616

[Prateek Grover]

We use main thread coroutines backed by workers, and just simple workers with callbacks. Depends how you want to structure it

Do you mean that the coroutine is launched on main thread and then the work is switched to workers? Doesn't this requires a lot of freezing of objects? How does mutable objects work in such an environment?

[alex009]

use https://github.com/Autodesk/coroutineworker - it will be done job on background thread and resume coroutine

[kpgalligan]

@Prateek Grover That comment. There was a branch a while ago, but they stopped that. The JVM coroutines work was happening at that time and I assume they wanted to stabilize coroutines in the JVM. Now that that’s “done”, there is effort to sort out how it’ll work with native. The native memory and concurrency model is quite different, so resolving that and coroutines is a challenge and taking time (I’d guess. I don’t work at JB)

[kpgalligan]

As for freezing, trying to avoid freezing data objects will be difficult generally. We have done some work on reactive data models coming from sqldelight where the result of a query gets moved to the main thread unfrozen, but in general we treat data flowing between functions as immutable. If you have code samples of what you’re concerned about it’s easier to discuss. Speaking in abstracts won’t mean much once we’re beyond the super generic.

[Prateek Grover]

thanks @kpgalligan for your responses.

[olonho]

note that situation with coroutines and multiple workers may become dramatically better with 1.3.60, stay tuned

[kpgalligan]

Assume it’s related to this: https://github.com/JetBrains/kotlin-native/pull/3493

[kpgalligan]

Or maybe this. Not sure how I didn’t see that https://github.com/Kotlin/kotlinx.coroutines/commits/native-mt

[kpgalligan]

Whatever happens, please publish like a week out from Kotlinconf. Have to update so many slides…

[Prateek Grover]

I am just trying to learn the concepts. Would you be kind enough to give me an overview of how they are trying to make coroutines multithreaded on native platform? I feel with the concurrency model they have for KN, it would be a difficult thing to pass around work to different coroutines unless they give a similar construct as workers freeze and DetachedObjectGraphs.

[elizarov]

First of all, coroutines can be started in any worker on K/N. What is not working right now is that coroutines in different workers cannot communicate, since coroutine primitives (like Job, etc) are not shareable between workers. This is going to change with Kotlin 1.3.60 release. You can already play with it in native-mt branch. For example, all of the following tests now pass: https://github.com/Kotlin/kotlinx.coroutines/blob/native-mt/kotlinx-coroutines-core/native/test/WorkerDispatcherTest.kt

[elizarov]

Don’t use in prod, though, yet. It will randomly fail under load.

[Prateek Grover]

Thanks Roman. With 1.3.60, will it be possible for us to start coroutines from a thread other than the main thread in case of iOS? Or is it a separate problem to solve?

[elizarov]

When you say “start coroutines from a thread other than the main thread”, what exactly are you trying to achieve? I’m trying to understand, because you can already do it.

[Prateek Grover]

In the comment https://github.com/Kotlin/kotlinx.coroutines/issues/470#issuecomment-440080970, if the MainLoopDispatcher instead of dispatch_get_main_queue() uses dispatch_get_global_queue(), it does not work. Will that be available in 1.3.60? I hope i am explain ing it right

[elizarov]

Can you explain your use-case in a bit more detail. What kind of application you are writing and what are you trying to do?

[Prateek Grover]

Sure. I am an iOS developer and currently trying my hand on kotlin/native to create a common framework for android and iOS. The first problem is that I do not want to invoke KN framework from the main thread always. Lets say there is an analytics framework. Some events like clicking button will surely come from main thread but some events like error in parsing will come from a background thread. Since KN coroutines can't be invoked from a thread other than the main thread, I will have to wrap the "error parsing" events too inside the main thread and invoke the coroutine in KN framework so that it can send the events to server.

So my question was, in the MainLoopDispatcher, will it be possible for me to use global queue (background queue) instead of main queue when 1.3.60 is available?

While writing this explanation, I think I got what you were trying to explain. So, we can still start coroutines from a thread other than the main one by using a worker. We can start a worker which is attached with a thread and then work with coroutines on top of that. The problem that 1.3.60 solves is when two workers have different coroutines running on them and they want to communicate. Am I right in my understanding? I have kept the above explanation intact just in case I am wrong in understanding you.

Thanks, for your patient responses.

[elizarov]

Yes. Communication between coroutines is currently missing. It will be tentatively available in 1.3.60. You will not be able to use a background queue, though. However, you will be to create a separate worker for your background coroutines with newSingleThreadedContext function (that will be available) and launch coroutines in this background context, while being able to communicate with the core running in the main queue by transferring immutable objects back and forth.

[Prateek Grover]

Thanks Roman for the explanation

[louiscad]

Honestly, I'm very excited to see that coming out and try using it for sample apps (for now), and production-ready libraries.

Also, the granular Kotlin metadata, which I believe is part of HMPP (hierarchical #multiplatform projects) seems very promising for the experience of library developers and developers of consumer projects.

[kpgalligan]

Building Droidcon app now with 1.3.60 eap. Had to deploy several libraries locally, but getting to the end hopefully

[louiscad]

I'm now using GitHub Actions to build and publish on a dev repo (hosted on bintray), and on jcenter (synced with another bintray repo) without making my machine busy for 5 minutes or more. I publishes only when I edit the file that holds the library version, and each build is checked on PRs and pushes to develop and master.

Should I link the workflow file + relevant gradle setup?

[kpgalligan]

I’d be interested in see how that’s set up. All my actual publishing is manual

[kpgalligan]

Feels like a dumb question, but I have kotlinx.coroutines from the ‘native-mt’ branch building and publishing locally. My test app seems to find the right dependencies, but newSingleThreadContext is not found when compiling. Before I go into a deep dive with the config, is there a different dependency here? Currently using the following in the ios dependencies “org.jetbrains.kotlinxkotlinx coroutines core native1.3.2-native-mt-SNAPSHOT”

[elizarov]

You might need kotlin.mpp.enableGranularSourceSetsMetadata=true in your gradle.properties

[kpgalligan]

Thanks

[elizarov]

I’ve published the first development preview version of multi-threaded kotlinx.coroutines version 1.3.2-native-mt-1 for Kotlin/Native (version 1.3.60) . Make sure to study the docs before using it:
https://github.com/Kotlin/kotlinx.coroutines/blob/native-mt/kotlin-native-sharing.md