Are there any known plans to improve the debugger ...

# coroutines

[CLOVIS]

Are there any known plans to improve the debugger experience? I see many people stopping to use coroutines in their project because debugging is so hard.

[Daniel Lukic]

Imho cx are really easy to test. So instead of debugging, add test coverage in case of issues. If not testable, start over by writing tests and let this guide the implementation. Personally, I haven't used a debugger since the 80s and 90s in the 8 and 16 bit years.. ^^ Debugging is overrated.. :-D

[CLOVIS]

…I completely agree with this statement, and that's the reason I don't have a problem in my personal projects.

However, that's not how most projects are developed 😕

[louiscad]

I personally use println/loggers and try/catch/finally to debug. Simple and works well.

[Dmitry Khalanskiy [JB]]

Are there any known plans to improve the debugger experience?

Yes, certainly, there's active work in this direction: https://youtrack.jetbrains.com/issue/IDEA-338878/Coroutine-debugger-improvements

[CLOVIS]

Thanks 🙏

[CLOVIS]

I love coroutines, but the problem is always that stacktraces don't teach you much, because they are always rooted in the coroutine machinery and not in what we developpers believe to be the calling code 😕

[Dmitry Khalanskiy [JB]]

Isn't the problem inherent to any multithreading? You probably wouldn't get much useful information from the stacktraces when this non-coroutine code crashes:

val result = AtomicInteger()
thread {
  if (!inputIsValid(input)) throw IllegalStateException()
  result.set(10)
}
while (result.get() != 10) { Thread.yield() }

[CLOVIS]

To any async work queue, yes.

Project Loom doesn't have this problem.

[Dmitry Khalanskiy [JB]]

There's no async work queue in my example, just a new thread is created to do some work.

[CLOVIS]

But since Kotlin has a great deal of syntax around making async-style code feel like sync code, it's even more jarring when the stacktraces are obviously not at all sync 😕

[CLOVIS]

Keep in mind, I have no idea if it's even possible to display normal stacktraces in this situation! But I also couldn't have imagined a way to write async-style code imperatively as nice as coroutines are, so…

[Dmitry Khalanskiy [JB]]

Would stacktrace recovery help? https://github.com/Kotlin/kotlinx.coroutines/blob/master/docs/topics/debugging.md#stacktrace-recovery I'd assume that in all cases where Project Loom virtual threads allow meaningful stacktraces, our stacktrace recovery does the same. If you have some counterexamples, please share them.

[CLOVIS]

I'm not completely sure I understand how stacktrace recovery works or if I even have it enabled 😕 I'll look into it more.

If you have some counterexamples, please share them.

I don't have any on hand, but just in case—do I send them on YouTrack or in the KotlinX.Coroutines github?

[Dmitry Khalanskiy [JB]]

To github, or here in this thread, or just to me personally. Stacktraces are purely a problem on the side of the coroutines, and the YouTrack issue is for the IDE support.
cc @Vsevolod Tolstopyatov [JB]: some feedback about our stacktraces. Maybe one more data point to allowing enabling it in production.

[louiscad]

I never understood why this is not enabled in production. Stacktraces are expensive anyway…

[Vsevolod Tolstopyatov [JB]]

Mostly because it changes the semantics of the program and highjacks exceptions' identities sad panda

[louiscad]

It would be a godsend to have it in production on all platforms, and Android in particular, since a lot of exceptions can happen throughout the framework and first party libraries

[louiscad]

If it's only a semantic problem, and not a performance concern, I think it's fine advertising those with a warning

[louiscad]

Right now, it's buried somewhere, nothing nudges devs to even know it exists, and convince them to use it

[louiscad]

CancellationException is already changing the semantics of catch (e: Exception), and for the worst anyway (zombie coroutines, infinite loops, ANRs…)…

[Dmitry Khalanskiy [JB]]

If it's only a semantic problem, and not a performance concern

If you mean enabling stacktrace recovery by default, then I'd think that subtly breaking production code in thousands of projects is quite a bit more problematic than making it slightly slower, actually.

[louiscad]

I think enabling stacktrace recovery by default on Android, and pushing it for JVM projects would be less disruptive than CancellationException

[louiscad]

I'd be curious to see the kinds of production code that relies on exception identity

[Dmitry Khalanskiy [JB]]

For one, CancellationException doesn't break any codebases that just upgrade to the new version of the coroutines library.

[louiscad]

It breaks code that is adopting coroutines, slightly different

[louiscad]

I have been using and avertising coroutines since Kotlin 1.1 in 2017, and CancellationException being an Exception is really my top one complaint, and stacktrace recovery being more known and available on Android in production is my top 2 wish.

[louiscad]

I don't know what's next for coroutines, but these 2 things being worked on would be my favorite announcement.

[Dmitry Khalanskiy [JB]]

What do you think CancellationException should be instead?

[CLOVIS]

To avoid clutting this, I gave an exemple here: https://kotlinlang.slack.com/archives/C1CFAFJSK/p1713176546595309

[louiscad]

We already talked about that in another thread, and an issue has been opened on that topic, but short story:
CancellationSignal which would be a direct Throwable subclass after a migration phase before kotlinx.coroutines 2.0 to make it no longer an Exception subclass.

[louiscad]

Maybe if union types come to Kotlin 2.x soon, it would also be possible to leverage those for a non Throwable, stacktrace-free CancellationSignal.

[louiscad]

But I think that union-type optimization could be added later on in a binary compatible way.

Migrating to such a thing from CancellationException or from CancellationSignal : Throwable would be about the same anyway, so I think going with CancellationSignal : Throwable first is a good step, one that can fix many bugs by just upgrading kotlinx.coroutines to 2.x

[louiscad]

The issue I was mentioning: https://github.com/Kotlin/kotlinx.coroutines/issues/4073

[louiscad]

About stacktrace recovery on Android in release mode:
https://github.com/Kotlin/kotlinx.coroutines/issues/1998

[CLOVIS]

Waiiiit, CancellationException inherits from IllegalStateException? So even catching IllegalStateException creates zombies? 😭

[louiscad]

Yeah, learned that this year thanks to @kevin.cianfarini. It's not affecting me too much since all my codebases are now kinda polluted with a rethrowing block specifically for CancellationException first every time I need to catch for other possible things, but it's bad…

[kevin.cianfarini]

Generally you probably shouldn't catch ISE since it's supposed to represent illegal program state and should tear down everything, but in large production codebases with many engineers of varying levels that's a pretty unrealistic expectation to maintain over time.

[kevin.cianfarini]

Also, I did not realize that cancelation exception was declared in the stdlib, not the library, when I originally filed this issue. Considering Kotlin 2.0 is already in the RC phase I doubt we'll see a change on flattening the inheritance hierarchy of CE

[kevin.cianfarini]

I agree with Louis, though. The current implementation of CancellationException has lots of sharp edges that tend to cut engineers of all levels at least a few times. It's one of my top gripes as well, though I still love the ergonomics of coroutines regardless.

[kevin.cianfarini]

Just our of curiosity, are there other places in the Kotlin ecosystem that make heavy use of CancellationException that isn't the coroutines library? I don't entirely understand the benefits of having that declared in the stdlib and not the library

[louiscad]

Yeah, CancellationException was silently added to the stdlib at some point, and I don't understand why it was.

[kevin.cianfarini]

You might also argue that CancellationException inheriting from ISE doesn't make a lot of sense because ISE is generally used to model irrecoverable program state, though I haven't thought entirely through that argument so take it with a grain of salt

[louiscad]

That is because it's reusing the one from java.concurrent, which is where it extends from IllegalStateException.

[kevin.cianfarini]

Right. But perhaps that's a mistake on Java's side?

[kevin.cianfarini]

And then it was a mistake on Kotlin's side to follow suit

[louiscad]

There are mistakes on Java's side? How dare you say! 😳

[louiscad]

A Java mistake that Kotlin didn't fix? 😮

[kevin.cianfarini]

Well I mean let's be pragmatic here. Too much deviation from Java would have seriously dampened Kotlin's adoption.

[kevin.cianfarini]

Mostly I think it's taken our community several years to understand why these might be mistakes for our use cases

[kevin.cianfarini]

For me personally, I didn't really start to wrap my head around the semantics of CE until I started using Compose, which cancels things tons when keys change. Before that I was blissfully unaware

[CLOVIS]

Generally you probably shouldn't catch ISE since it's supposed to represent illegal program state and should tear down everything, but in large production codebases with many engineers of varying levels that's a pretty unrealistic expectation to maintain over time.

Well, yeah, but it's also one of the most common exceptions we throw (require, check, error) so it's one of the most likely you'd want to target in event-loop-level exception handlers (as in, UI threads, web server endpoints...).

[CLOVIS]

Just our of curiosity, are there other places in the Kotlin ecosystem that make heavy use of CancellationException that isn't the coroutines library?

I believe Arrow does, but it's mostly in order to interop with KotlinX.Coroutines.

[CLOVIS]

So far I haven't really been bitten by CancellationException, but then again I religiously avoid catching exceptions. I've had more issues with `withContext(Dispatcher.IO)`which is peppered everywhere in the code, and that breaks coroutines-test's time control—but oh well.

[Daniel Lukic]

for a long time, i used a "global" something like this:

object Cx {
    var Default: CoroutineDispatcher = Dispatchers.Default
    var IO: CoroutineDispatcher = <http://Dispatchers.IO|Dispatchers.IO>
    var Main: CoroutineDispatcher = Dispatchers.Main
    var MainImmediate: CoroutineDispatcher = Dispatchers.Main.immediate
    var Unconfined: CoroutineDispatcher = Dispatchers.Unconfined
}

(with all the globals in android it never really bothered me much to a few more.. 😄 )

and then simply set what i need in tests. i always felt it is clearer than passing around dispatchers. but i guess it comes down to opinions.. as it often does.. 🙃

and i didn't actually know (or care) about what CancellationException extends.. i created - a long time ago, too - tryCatching to mimic runCatching, but for coroutines.. and it handles cancellation in one place.. and provides a CxResult (instead of Result because iirc Result wasn't initially available to pass around or something?).. anyway, main point: maybe it's possible to manage cancellation in one place and consistently use something like tryCatching? and forget about CancellationException mostly? but i'm pretty sure i'm missing something here.. skimmed over all the comments here only.. ☯️

bottom line: can the issues you mention be avoided mostly with some helper code? 🤔 again, i'm probably missing something...

[CLOVIS]

maybe it's possible to manage cancellation in one place and consistently use something like tryCatching? and forget about CancellationException mostly?

I recommend Either.catch from Arrow which does this perfectly well (and also does it for other types of expressions that you shouldn't catch).

Personnally, all my classes that have a lifecycle of their own accept a CoroutineContext constructor argument. But in the end, it's fairly rare that I need to switch dispatchers, so there are very few of those.