https://kotlinlang.org logo
#coroutines
Title
# coroutines
c

Carrascado

08/23/2020, 4:06 PM
Ok I'm still learning about coroutines, and I'm a bit confused with the coroutines builders zoo... Is this all correct? • launch: Creates a new coroutine. Code below
launch {}
continues being executed instantly (although because structured concurrency, coroutine will finally wait for its children to finish). • async: Creates a new coroutine. Code below
async {}
continues being executed instantly (although because structured concurrency, coroutine will finally wait for its children to finish). • runBlocking: Creates a new coroutine. Execution is blocked. Code below
runBlocking {}
won't execute until runBlocking has ended. • coroutineScope: Creates a new coroutine. Execution is suspended (it's the suspending version of runBlocking). Code below
coroutineScope {}
won't execute until coroutineScope has ended. Now, if this is correct, the only coroutine builder whose name makes sense is runBlocking 🤷‍♂️ The other ones seem to have absolutely random names. The 4 coroutine builders "launch" the coroutine, two of them are "asynchronous" in some sense (both launch and async), and 4 of 4 again have a "coroutine scope", what am I not understanding here? Why do they have these names?
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:18 PM
I think
async
is the most surprising of these names, but I think it’s called that because it’s supposed to be one-half of the `async`/`await` keywords in other languages (javascript, python, c#). It’s probably named this way for discoverability and to hint at how it’s meant to be used for people familiar with async/await. But if you don’t have that context, the name does seem super random.
launch
is the most basic way to “launch” a coroutine. It just launches it, fire-and-forget, it doesn’t do anything else.
async
could probably be called
launchForValue
if it wasn’t for the async/await thing.
c

Carrascado

08/23/2020, 4:23 PM
Absolutely, and the one which bugs me the most is coroutineScope, why is it not called runSuspending? Man the documentation should be reviewed by the devs... I'm having a hard time to understand anything. If I understand something now it's not because the docs but because I'm doing a lot of CSI stuff on the Internet
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:24 PM
This is how I think about it, although I definitely see your point:
coroutineScope
doesn’t really create a new coroutine, really, anymore than
runBlocking
creates a new thread. It simply delineates a scope within which new coroutines can be created. Whereas
runBlocking
is usually used in a way where it will run a bunch of coroutines for some time and block the thread for potentially quite a long time (the “blocking” nature of the call is very important),
coroutineScope
is primarily meant to define a scope. The scoping is the more important thing, not the suspending.
Framed another way,
coroutineScope
is used when you need to get a scope in which to start some coroutines from inside a suspend function.
runBlocking
is used in the more rare case when you’re not already in the coroutine world and need to block the current thread in order to run some coroutines.
c

Carrascado

08/23/2020, 4:27 PM
So technically speaking coroutineScope is not a coroutine builder? And making a new scope... doesn't that destroy the idea of structured concurency? Because it won't be inheriting the parent scope anymore
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:28 PM
The new scope does inherit from the parent scope, that’s the whole point.
c

Carrascado

08/23/2020, 4:28 PM
And what does launch/async/runBlocking do with the scope, don't they inherit too? What's the difference then? 🤔
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:29 PM
Everything (except
runBlocking
) inherits scope.
coroutineScope {}
is basically the same thing as writing
with(CoroutineScope(coroutineContext + Job(parent = coroutineContext[Job]))) {}
c

Carrascado

08/23/2020, 4:31 PM
And launch is doing the same thing isn't it? It's creating a new Job and adding it to the scope
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:32 PM
launch
and
async
create new jobs, yes. I don’t really think of them as “adding the job to the scope” though, since they aren’t mutating the current scope. They’re creating a new scopes which shares everything from the current scope, except the job is a child of the current job.
c

Carrascado

08/23/2020, 4:34 PM
What's the difference then with using coroutineScope? Because if I have an explicit "scope builder" like coroutineScope, why do launch and async create new scopes too?
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:35 PM
It’s important not to conflate “job” and “scope”. A job defines the lifetime of a coroutine. A scope contains a job, but also contains other context. So when you say a scope inherits from another scope, you’re really kind of saying two things: the child scope’s job is a child of the parent scope’s job, and the child scope might share some/all of the rest of the context from the parent (e.g. dispatcher, error handler, name, etc).
The use case for
coroutineScope
is to create a “group” of child coroutines so that the parent coroutine can wait for them all to be completed, and that will all be cancelled if the parent is.
c

Carrascado

08/23/2020, 4:39 PM
To me, that sounds like a heavy ton of implicit information from something called
coroutineScope
, specially when launch and async don't make you wait but they create new "coroutineScopes" too 😕
I still don't see the difference exactly between coroutineScope and the other ones, except that coroutineScope makes you "wait" via suspending
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:40 PM
E.g. you might try to write something like
Copy code
suspend fun doSomeThings() {
  launch { doFirstThing() }
  launch { doSecondThing() }
}
But that won’t work, because
launch
is defined as an extension function on
CoroutineScope
. And if it did work, e.g. you used the
GlobalScope
, then these coroutines would be leaked – if they never finished, then they would take up memory/resources forever. And there’s no way to cancel them. Instead you’d write:
Copy code
suspend fun doSomeThings() {
  coroutineScope {
    launch { doFirstThing() }
    launch { doSecondThing() }
  }
}
Another way of thinking about it is that
coroutineScope
just gives you a block which lets you start coroutines. You can’t even call launch/async without a scope, and
coroutineScope
is a good way to get one if you’re already in a suspend function.
c

Carrascado

08/23/2020, 4:41 PM
And in your example, how does that
coroutineScope
know who is the parent scope? Is it the parent scope of the suspending function caller?
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:43 PM
The job of the coroutine that calls
doSomeThings()
is the parent of the job inside
coroutineScope
, which is the parent of the jobs that
doFirstThing()
and
doSecondThing()
see.
c

Carrascado

08/23/2020, 4:44 PM
Umm ok... and do you personally think this is well explained in the docs? Because I have spent more than a week trying to understand coroutines and I still don't get it well
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:45 PM
Well the docs make sense to me now for the most part, but that doesn’t mean they would make sense to me if I was coming at this for the first time 😅
c

Carrascado

08/23/2020, 4:46 PM
Soooo the only way to call launch and async from a suspend fun is using coroutineScope? Or is there some other strange edge use case for it?
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:47 PM
Basically yes!
Technically you can call `launch`/`async` on any
CoroutineScope
instance, but the most common and simplest way is to use
coroutineScope{}
. But you can, for example, write
Copy code
val scope = CoroutineScope(<http://Dispatchers.IO|Dispatchers.IO>)
scope.launch {}
However in that case, the scope’s job is not a child of the current job, so you’ve just broken structured concurrency. Don’t do that unless you really mean to.
I’m not sure if this will make any more sense or just add to your confusion, but I tried to explain all this stuff a while back from a different angle:

https://www.youtube.com/watch?v=OqOGze29xPQ

c

Carrascado

08/23/2020, 4:52 PM
Thank you very much! I'm gonna check it out. By the way your example breaks structured concurrency, but doesn't
withContext(<http://Dispatchers.IO|Dispatchers.IO>)
break it too? Why is it recommended practice?
Well the question is more like "Why does
withContext
is called like that and not
withDispatcher
?"
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:53 PM
However in that case, the scope’s job is not a child of the current job, so you’ve just broken structured concurrency.
Yes it does 🙂
withContext
can change any part of the context, not just the dispatcher. E.g. you can do
withContext(CoroutineName("debug me"))
c

Carrascado

08/23/2020, 4:54 PM
I know you said it, I was affirming it, my english is sloppy sorry 😂
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 4:54 PM
withContext
doesn’t break structured concurrency unless you pass in a job without a parent.
so
withContext(<http://Dispatchers.IO|Dispatchers.IO>)
is fine, but
withContext(Job())
would break it (actually not sure if that would even run, it might throw an exception if you try to pass a different job in).
and for completeness, if you changed my earlier example to
CoroutineScope(Job(parent = coroutineContext[Job]))
, then it would not break structured concurrency.
The coroutine context is like an immutable
Map
.
withContext(foo) {}
is basically just
with(coroutineContext + foo) {}
, where
coroutineContext
is the context of the caller.
c

Carrascado

08/23/2020, 4:59 PM
So the job hierarchy is what really "defines" the structured concurrency? Sorry I'm not really fluent in Kotlin (yet), do you mean that if Context has 5 properties, when you call
withContext(<http://Dispatchers.IO|Dispatchers.IO>)
, it creates a new context with the new dispatcher but it defaults all the other stuff with whatever it was before?
z

Zach Klippenstein (he/him) [MOD]

08/23/2020, 5:01 PM
So the job hierarchy is what really “defines” the structured concurrency?
YES! Exactly!
if Context has 5 properties, when you call withContext(Dispatchers.IO), it creates a new context with the new dispatcher but it defaults all the other stuff with whatever it was before?
Also yes!
c

Carrascado

08/23/2020, 5:02 PM
I'm glad I'm understanding something at least 😂 Thank you for your time! I'm gonna watch your video
j

julian

08/24/2020, 3:28 PM
I nominate this thread to be permanently pinned on the channel. Brilliant.
❤️ 3
t

taer

08/26/2020, 3:21 AM
old thread, but quick question.. this code(screenshotted to get the implicit type hints), the suspend function calls coroutineScope, who now can call a launch, who now is his own coroutine parented under the new scope. My simple question: how does the caller of
test()
which HAS to be a couroutine(since it is suspendable) get to become the parent of the
coroutineScope
defined in
test
? from the doc:
Copy code
The provided scope inherits its [coroutineContext][CoroutineScope.coroutineContext] from the outer scope, but overrides
 * the context's [Job].
Is it just part of the
suspendCoroutineUninterceptedOrReturn
implementation that links the two?
I sent that, and semi-figured it out... another screenshot:
coroutineContext
seems to be a local variable inside of suspend functions.
Copy code
@InlineOnly
public suspend inline val coroutineContext: CoroutineContext
    get() {
        throw NotImplementedError("Implemented as intrinsic")
    }
What is that?
I know I'm getting into the guts, and they're not really for human consumption.. But just digging more into this thread(which was awesome!)
z

Zach Klippenstein (he/him) [MOD]

08/26/2020, 4:25 PM
old thread
Only two days 😛
Is it just part of the suspendCoroutineUninterceptedOrReturn implementation that links the two?
Not quite.
Copy code
public suspend fun <R> coroutineScope(block: suspend CoroutineScope.() -> R): R {
    return suspendCoroutineUninterceptedOrReturn { uCont ->
        val coroutine = ScopeCoroutine(uCont.context, uCont)
        coroutine.startUndispatchedOrReturn(coroutine, block)
    }
}
The current context (
uCont.context
) is passed to the
ScopeCoroutine
constructor, which passes it through to the
AbstractCoroutine
constructor.
AbstractCoroutine
wires up the job inside the context to be the parent of itself (see
initParentJobInternal()
).
AbstractCoroutine
also implements
CoroutineScope
, and so the
coroutine
is passed as the receiver for the coroutine block (i.e. the first arg to
startUndispatchedOrReturn()
).
🤯 1
c

Carrascado

08/26/2020, 4:29 PM
I'm a bit curious about the question @taer made... but, inside every coroutine, what does really exist, a "current scope" or a "current context" at top level?
I guess a scope because it has the context inside as a property?
And that is passed somewhat magically to
coroutineScope
as a parent?
z

Zach Klippenstein (he/him) [MOD]

08/26/2020, 4:30 PM
coroutineContext seems to be a local variable inside of suspend functions.
It’s not a local variable, it’s a top-level/global property. It’s basically shorthand for writing:
Copy code
val coroutineContext = suspendCoroutine { continuation ->
  continuation.resume(continuation.context)
}
That is, the continuations that are passed around as the core currency of coroutines always contains the current context, which you can see by calling one of the
suspendCoroutine
functions to actually get access to a continuation.
coroutineContext
is just a convenient way to get it without the boilerplate.
CoroutineScope
is nothing special. It’s just a thing that points to a
CoroutineContext
.
c

Carrascado

08/26/2020, 4:32 PM
But don't you mean that the core currency of coroutines always contains the current scope (not exactly the context)?
z

Zach Klippenstein (he/him) [MOD]

08/26/2020, 4:33 PM
The only reason
CoroutineScope
is a thing at all is to serve as a place to hang extension methods off of. It’s a common pattern for functions that take lambdas-with-receivers to call the receiver type
SomethingScope
. But as far as the actual semantics are concerned for coroutines, “scope” and “context” are the same thing (a scope is just a different way of referring to a context).
CoroutineScope
is literally just
Copy code
interface CoroutineScope {
  val coroutineContext: CoroutineContext
}
Conceptually, you can think of it like this:
Copy code
interface CoroutineScope {
  val coroutineContext: Map<ContextKey, Any?>
}
c

Carrascado

08/26/2020, 4:36 PM
That makes sense, so normally when people is talking about the context in coroutines, they might be referring to a CoroutineScope?
z

Zach Klippenstein (he/him) [MOD]

08/26/2020, 4:40 PM
It’s more the other way around.
CoroutineContext
is the core data type that actually matters, it came before
CoroutineScope
, it actually holds the context, etc. And the two are effectively interchangeable –
CoroutineScope(CoroutineScope(CoroutineScope.coroutineContext).coroutineContext).coroutineContext
The only caveat is that the
CoroutineScope()
function will always return a scope with a context that has a
Job
– if the context passed to it does not have a
Job
, an orphan one will be created.
c

Carrascado

08/26/2020, 4:44 PM
But... okay this question is kinda noob 😂 I'm seeing CoroutineScope as an Interface with Intellij IDEA, so, how can you call that
CoroutineScope
, isn't it a constructor?
c

Carrascado

08/26/2020, 4:49 PM
Oh okay thank you!
What puzzles me a bit is... why can you make
withContext(<http://Dispatchers.IO|Dispatchers.IO>)
? That makes you guess that built in dispatchers (IO, Default, etc.) are actually some context instances... but in that case, how can that work? Those instances will probably have their own Job instance, their own everything. If the code inside
withContext
runs within a different context, then it will run within a different coroutine because the Job is different isn't it? 🤔
That seems like breaking the structured concurrency but there must be something wrong in my reasoning
z

Zach Klippenstein (he/him) [MOD]

08/26/2020, 4:59 PM
You're right that dispatchers are actually context instances themselves. CoroutineContext acts like a map, but it's implemented as a linked list. Every context "element" is both the thing that holds the value for its map entry, and a pointer to the rest of the map.
Job
is a context element. A dispatcher is a context element too. A context element is a context that only contains one element.
Not every context contains a Job.
c

Carrascado

08/26/2020, 5:03 PM
My head exploded, so it's like a composite pattern? I'm looking at the function, so there in the blue section it's getting the Job for the coroutine etc. and combining it with the dispatcher? (obviously in the case that you wanted to call it with Dispatchers.IO or something like that). Because I guess it needs a job in order to work (no pun intended)
🥁 1
z

Zach Klippenstein (he/him) [MOD]

08/26/2020, 5:14 PM
Yep, pretty much.
c

Carrascado

08/26/2020, 5:15 PM
Thank you very much, I'm understanding a lot more now
z

Zach Klippenstein (he/him) [MOD]

08/26/2020, 5:15 PM
oldContext
is a map,
context
is a map,
oldContext + context
is like
+
-ing any maps – the keys in the left one are “overwritten” with the same keys in the right one.
c

Carrascado

08/26/2020, 5:16 PM
Yeah, so it's like a "merge"
7 Views