Working on concurrency in Kotest, I am investigati...

# kotest-contributors

[Oliver.O]

Working on concurrency in Kotest, I am investigating current limitations of io/kotest/mpp/replay.kt, which changes a test invocation in (as I see it) unexpected ways if configured with threads > 1.

First, by using a plain runBlocking call, it is losing the CoroutineContext and thereby loses Kotest's entire internal configuration represented by context elements.

What originally was this (in a BlockHound test):

ProjectContextElement(projectContext=ProjectContext(suite=TestSuite(specs=[Reference(kclass=class io.kotest.extensions.blockhound.BlockHoundCaseTest), Reference(kclass=class io.kotest.extensions.blockhound.BlockHoundSpecTest)]), tags=TagExpression(expression=), configuration=io.kotest.core.config.ProjectConfiguration@1b442bb5)), kotlinx.coroutines.UndispatchedMarker@4e8c790a, ConfigurationContextElement(configuration=io.kotest.core.config.ProjectConfiguration@1b442bb5), CoroutineId(12), io.kotest.common.TestPathContextElement@6dc92973, io.kotest.common.TestNameContextElement@4ac54186, io.kotest.assertions.ErrorCollectorContextElement@7066d2e8, io.kotest.extensions.blockhound.BlockHound$ContextElement@5d8fa1c7, "coroutine#12":DispatchedCoroutine{Active}@156e340c, java.util.concurrent.ScheduledThreadPoolExecutor@4a470feb[Running, pool size = 2, active threads = 2, queued tasks = 1, completed tasks = 0]

shrinks inside replay multithreading to this:

[CoroutineId(13), "coroutine#13":BlockingCoroutine{Active}@5a7deccb, BlockingEventLoop@1a030c80]

Second, it introduces Java-style multithreading, initially confining each test invocation to a single thread (where it can then break out if it uses a multithreading dispatcher inside the test). This is not representative for Kotlin multithreading and (as I see it) it is not necessary.

I could successfully replace the Java-style multithreading with Kotlin-style coroutines multithreading by replacing this

val executor = Executors.newFixedThreadPool(threads, NamedThreadFactory("replay-%d"))
      for (k in 0 until times) {
         executor.submit {
            runBlocking {
               try {
                  action(k)
               } catch (t: Throwable) {
                  error.compareAndSet(null, t)
               }
            }
         }
      }
      executor.shutdown()
      executor.awaitTermination(1, TimeUnit.DAYS)

with

@OptIn(DelicateCoroutinesApi::class)
      newFixedThreadPoolContext(threads, "replay").use { dispatcher ->
         for (k in 0 until times) {
            withContext(dispatcher) {
               try {
                  action(k)
               } catch (t: Throwable) {
                  error.compareAndSet(null, t)
               }
            }
         }
      }

The changed implementation retains the configuration and passes almost all (1081) engine tests, but fails on those (8) tests using Java's ReentrantLock, which is thread-bound. So my questions are:
1. What are the intentions guiding the current implementation?
2. Could we change this from Java-style multithreading to Kotlin-like (coroutine hopping) multithreading?

[Oliver.O]

Oh, and I‘d better correct myself: That withContext call should be replaced with launch.

[Oliver.O]

Using launch and wrapping the invocation loop in coroutineScope brings back true parallel execution. As a result, some more tests fail which currently rely on ThreadLocal. These can all be fixed and made to test "natural" coroutine behavior without relying on thread-confinement.

Searching through all of those ThreadLocal places, assertionCounter seems to suffer from the same thread hopping problem as errorCollector did (https://github.com/kotest/kotest/issues/2447).

I'd try to fix all of that in one multithreading PR if there are no objections.