https://kotlinlang.org logo
Join Slack
Powered by
The kdoc for `runBlocking` claims it blocks the th...
# coroutines
s
The kdoc for 
runBlocking
claims it blocks the thread from which it is called (interuptibly). But I have observed behaviour where other coroutines waiting for dispatch on that thread (e.g. on the main thread) will be dispatched by the event loop because 
runBlocking
falls back to using the calling thread's 
eventLoop
even when a specific other dispatcher (which may be busy) is specified for that which to run the coroutine launched by 
runBlocking
on. Why is this and is this a bug? If not what is the explanation for this choice of behaviour? Code reproduction example in thread:
I realize 
runBlocking
has a note about not using it from within a coroutine - but I'm not sure it's possible to know that as you could be running on any coroutine but not within a suspend function and you wouldn't know your execution context.
Untitled.cpp
Also, the reason this is using an 
Executor
wrapped coroutine dispatcher is just that this is where I first observed the behaviour as the 
testThreadDispatcher
in this case can't get the work dispatched immediately. There may be a simpler way to reproduce.
Anyway, long story short, this prints: 
A
, 
D
, 
B
,
C
then hangs. I would have thought the coroutine for 
B
and 
C
would never have gotten dispatched but 
runBlocking
picks up the calling threads 
eventLoop
to 
processNextEvent
while waiting?? It should be blocked though?
e
internally, runBlocking uses a ThreadLocal<EventLoop>, so an inner runBlocking on the same thread as an outer runBlocking will very possibly dispatch coroutines outside of its scope
s
why though? Is that even desirable?
s
Nothing about 
runBlocking
is desirable 😄. Solution: don't use it. Ever.
e
or use it in one place, ever. nothing about nesting runBlocking is ever ok
s
seems if that is the case it should be much harder to do that, and it should be clear why that is not OK in kdoc etc.
👍 1
that's not very satisfying as it is a useful bridge and has use cases
s
Anecdotally I've heard that the Kotlin team wanted to remove it, but couldn't because it had already been (ab)used in so many places. Hence the thread local event loop compromise. Maybe somebody here knows if that story's true.
s
wrt using it within another 
runBlocking
- you have no way of knowing this, e.g. as a library developer
so it needs to be enforced in some way
c
There are many perfectly legitimate reasons to use 
runBlocking
to create top-level coroutines. I can't think of any valid reason why one would ever use 
runBlocking
inside of an existing coroutine. I'd think 
withContext(newFixedThreadPoolContext(1, name))
would be a better way to go about making an existing coroutine single-threaded
s
that existing coroutine can call any other non suspending function though
the example code here removes that detail just for simplicity
e
IMO as a library developer you should not ever use runBlocking. only the top-level application should use runBlocking to start suspend funs
now I get there are reasons that is not always possible, but they lead to difficult-to-detect problems
s
sure, again that is a valid opinion, but that is neither made clear nor enforced by the library
s
I'm not sure it's possible to know that as you could be running on any coroutine but not within a suspend function and you wouldn't know your execution context.
☝️ I think this point is crucial, and bit me quite a few times when converting an existing application to use coroutines
s
In real world programs coroutines are rarely all or nothing
c
It's not necessarily strictly enforced, but it's very much in-line with the understanding of "structured concurrency" that a library shouldn't call 
runBlocking
, because it has no way to tie into a parent coroutine's context/job
e
I think kotlinx-coroutines-debug may be able to detect misuses of runBlocking at runtime, but I have actually never run into it…
I gave it a shot now, but kotlinx-coroutines-debug's BlockHound setup didn't seem to catch runBlocking. that's probably worth filing as a feature request
s
https://github.com/Kotlin/kotlinx.coroutines/issues/3204 setup as an issue if you want to add your support please
z
Would one solution to this be to be explicit about the dispatcher inside your runBlocking?
s
sticking just to the example, you mean the inner 
runBlocking
? I am being explicit about the dispatcher there
e
doesn't help because runBlocking still needs some way to… well, block, which is handled by EventLoop
z
does 
runBlocking
start its coroutine undispatched? if it does, then if your runBlocking lambda never suspends it would never have a chance to hop threads. Curious what would happen if you did
Copy code
runBlocking {
  withContext(dispatcher) { … }
}
instead, or even
Copy code
someScopeWithDispatcher.launch { … }
  .let {
    runBlocking {
      it.join()
    }
  }
s
Tested both of these @Zach Klippenstein (he/him) [MOD] and they both have the unexpected behaviour.
Namely B. and C. still get dispatched
instead of having the whole thread blocked. I believe the reason for this is because the problem isn't that we can't get the 
EventLoop
of the calling context, its that we can't not get it. In my understanding we shouldn't be dispatching on the 
EventLoop
while blocked.
Another way of asking this question with the underlying code would be to say why does 
runBlocking
 pull out the 
EventLoop
 of the 
ThreadLocal
?
Copy code
if (contextInterceptor == null) {
        // create or use private event loop if no dispatcher is specified
        eventLoop = ThreadLocalEventLoop.eventLoop
        newContext = GlobalScope.newCoroutineContext(context + eventLoop)
    } else {
        // See if context's interceptor is an event loop that we shall use (to support TestContext)
        // or take an existing thread-local event loop if present to avoid blocking it (but don't create one)
        eventLoop = (contextInterceptor as? EventLoop)?.takeIf { it.shouldBeProcessedFromContext() }
            ?: ThreadLocalEventLoop.currentOrNull()
        newContext = GlobalScope.newCoroutineContext(context)
    }
Since it processes the next event on that 
EventLoop
 while blocking if it is available we by definition won't be blocking the 
ThreadLocal
 thread, which it claims it is.
Copy code
while (true) {
                    @Suppress("DEPRECATION")
                    if (Thread.interrupted()) throw InterruptedException().also { cancelCoroutine(it) }
                    val parkNanos = eventLoop?.processNextEvent() ?: Long.MAX_VALUE
                    // note: process next even may loose unpark flag, so check if completed before parking
                    if (isCompleted) break
                    parkNanos(this, parkNanos)
                }
My understanding would have been that we would only 
processNextEvent
 on the 
EventLoop
 if was passed in as the 
contextInterceptor
 and 
shouldBeProcessedFromContext()
. That way we don't block that Dispatcher/EventLoop but we do block the 
ThreadLocal
 thread, which is what the context specifies we should do.
Thread bump. Any other thoughts on this?
z
I'm not clear on what you're trying to accomplish. Are you trying to figure out how to get B not to execute?
s
I'm trying to understand why exactly B is executing; or more deeply why we take the EventLoop of the ThreadLocal in 
runBlocking
? Given that the documentation states that 
runBlocking
will block the calling thread, this seems contradictory.
z
So you're just curious? Have you dug through the coroutines source code? It's usually pretty well documented and sometimes explains intent for stuff like this.
e
https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html doesn't promise that no other coroutines are run, just that the current one will be completed
s
doesn't promise that no other coroutines are run
Well, it promises that the Thread is blocked, which I guess is vague in this case - but I would have interpreted it as not running other coroutines dispatched on that thread until those dispatched within 
runBlocking
are complete.
So you're just curious? Have you dug through the coroutines source code? It's usually pretty well documented and sometimes explains intent for stuff like this.
As far as the 
runBlocking
builder and the 
BlockingCoroutine
and the 
EventLoop
. I understand how the 
EventLoop
is needed to prevent deadlock and keep things moving. What I want though is a mechanism to provide my own 
EventLoop
(can't because internal) to 
runBlocking
such that it will dispatch only coroutines queued for dispatch within the 
runBlocking
lambda. That's what my interpretation of it thinks that 
runBlocking
should be doing anyway.
z
Ok but why? It sounds like you're trying to do synchronization by tuning the coroutine runtime
e
as I think we brought up earlier,
This function should not be used from a coroutine
you're outside of `runBlocking`'s use case to begin with
s
@ephemient I don't think that specification of `runBlocking`'s use case is valid because any coroutine can call any non-suspending function (do we have a word for this yet?) and so you won't know if you are in a coroutine.
@Zach Klippenstein (he/him) [MOD] 2 possible reasons I can think of: 1. Real world code that mixes java Threads, Rx Executors, and coroutines and have some APIs 'thread-confined' by contract (and thus can be blocking) but which want to call coroutines APIs. 2. blocking to call a suspend function in a time-critical way - e.g. while crashing
e
you can't dynamically know, but you should structure your code so that you statically know
s
you can't dynamically know, but you should structure your code so that you statically know
That's valid I guess if we decide never to use this in any library or shared ("distant") code. seems like a strong restriction.
e
it is. but doing otherwise breaks structured concurrency
s
how?
e
the inner scope is not parented to the outer scope
kotlinx.coroutines does try to make this at least somewhat work since there are libraries doing that, but it should not be recommended
s
ah IC - this gets to the heart of it - https://github.com/Kotlin/kotlinx.coroutines/issues/860.
z
Re thread-confined apis, from what you've said it doesn't sound like things are running on the wrong thread?
s
by 'thread-confined' i just meant not thread safe, contracted to be run from a particular thread so you don't need to switch Dispatchers to run the coroutine API, but I realize this is really orthogonal anyway to the blocking nature of it.
z
Right, from what you said it doesn’t sound like “tasks” that are destined for other threads are running on the wrong threads, just tasks for the current thread are running at an unexpected time?
s
Yes exactly.
@Vsevolod Tolstopyatov [JB] cleared this up noting that this is needed (pulling in outer EventLoop) is needed to prevent deadlock. I'm still wondering than how we could call a suspending API (not from within a coroutine but bridging to a coroutine) in a truly blocking way where other coroutines on that thread would not run either (yes we would risk deadlock but it would have its use case - e.g. while the process is crashing).
40 Views
Open in Slack
PreviousNext
Powered by