Do you have recommendations on how to test that my...

# coroutines

[CLOVIS]

Do you have recommendations on how to test that my program handles cancellation correctly? I can start a task and cancel it immediately, but there are many different points at which cancellation could occur and I'm not sure how to target a specific one.

[Dmitry Khalanskiy [JB]]

My perspective is skewed by mostly using kotlinx.coroutines from the inside, not as part of an application, so I'm not sure how applicable this advice is.

If the program is a black box (i.e., you can't inject your callbacks to arbitrary places), you'll have to write concurrent stress tests. Start two threads in parallel, one cancelling the program at a random point in time, the other running the program. Repeat 10000 times.

If you can inject actions to specific points, then what you want is probably a "latch" (docs.oracle.com/javase/…/CountDownLatch.html) or a "barrier" (docs.oracle.com/javase/…/CyclicBarrier.html). The suspend version of a latch is fun Job(): CompletableJob with its CompletableJob.complete() and Job.join() methods. Barriers are trickier, but also more rarely needed. With a latch, you can wait until right before the suspension, then trigger the cancellation and observe the result.

Please feel free to share specific code snippets, so that we could brainstorm ways to test them.

[CLOVIS]

I get the theory, but I don't know how to inject the latch in the production code without making it unreadable

[CLOVIS]

Here's an example of the production code. I'm interested by its behavior when request.output (which is a CompletableDeferred) is cancelled, because that means the consumer isn't listening for the results anymore

The red lines are the positions where I think it's interesting to know how the code behaves on cancellation

[simon.vergauwen]

I've caught a couple of cancellation bugs with github.com/JetBrains/lincheck I think it might help you out here with some cases but maybe not all.

[CLOVIS]

I think I'm having the opposite problem than Lincheck solves 🤔 I'd like to reliably reproduce one scenario, whereas Lincheck is for finding scenarii you don't know, right?

[CLOVIS]

I currently have this, but the first coroutine is able to terminate before the second launch cancels it 😅

[simon.vergauwen]

Yes, exactly but that's much trickier and I've also had a couple false positives in the past because they're hard to write (and review)... 😅
In Arrow-kt we have a bunch of tests that use latches from the user API to veriy (cancellation) behavior but it's not always easily possible. I used mostly CompletableDeferred<Unit>() as latch or CompletableDeferred<Throwable?>() to inspect exit state of an operation where null is success here.

[CLOVIS]

I don't really want to inject latches into the production code though… it's already complex to read

[simon.vergauwen]

Yes, you'd want to avoid that but I'd give Lincheck a try. It's definitely worth checking out.

[CLOVIS]

Oh, I will use it later when I'm hunting for bugs

[CLOVIS]

It is very interesting to see how coroutines now that I have a bunch of logs in the algo

[CLOVIS]

When a user makes a request, it is basically immediately sent because it's just a pass-over through a rendezvous channel

[CLOVIS]

compared to all the work such a hand-off would be with threads, it's impressive

[Oliver.O]

From my 2024 KotlinConf talk about stress testing, there is still that old repo around which uses a latch-like Gate to reliably provoke potentially conflicting scenarios: github.com/OliverO2/kmp-e2e-stress-testing-example/blob/…/Gate.kt

I accept that you're wary of polluting production code, but maybe you can find a less intrusive solution which might be acceptable.

[Wout Werkman]

There's also Fray, I just attended a talk from Adam Warski on the topic 2 days ago. And I recommend checking it out.
2026.javazone.no/program/…

They both have different tradeoffs. And because Fray introduces less interleavings, it will have better coverage per unit of time, because it will cover your use case (because isActive uses atomics under the hood IIUC).

One thing I've done personally, is go even higher level. I have a lil utility that changes the ordering of coroutine dispatches. Meaning that I have a very lazy (implementation wise) method of exploring suspension and resumption interleavings.
In your case you'd have to insert a yield() before both your isActive
checks.
github.com/brokenhappy/KotlinMultiplatformWorkshopWarsaw2024/blob/…/RandomizedDispatchOrder.kt#…

While there are many concurrency bugs it won't catch. It could get much better coverage across suspension point interleavings.
And it has found many bugs in my project.
I've been talking about it with some people in the field to discuss whether it's worth exploring this idea. The cool thing is that this doesn't need instrumentation, and it can be done per coroutine tree.

You can build very domain specific interleaving tests in coroutines because you have access to dispatchers, which is pretty cool.