Is there a good way to fail a test case from a task given to an executor service? A normal `shouldB...

# kotest

[bbaldino]

Is there a good way to fail a test case from a task given to an executor service? A normal shouldBe failure exception in the task just gets swallowed by the executor service. (I know this--using a real executor service/threads--is not a great unit test practice, but it's what I'm stuck with for verifying something for now.)

[bbaldino]

I ended up submitting a Callable and returned the result and verified outside the task. Would be great to be able to fail the test within the task somehow, though. I think this would involve a method on TestContext to force-fail it (as an alternative to a thrown exception), but this may not be a common use-case.

[sam]

you mean, Executors.newFixedThreadPool(8).submit { error("foom") } the execution is lost

[bbaldino]

Right

[sam]

You can set a thread exception error handler on the threads when you create the executor

[bbaldino]

Oh...good idea

[bbaldino]

So that'd be a custom thread factory which does that?

[sam]

yea

[bbaldino]

Ok, more code than my workaround but I could take a look at a PR if you think it'd be interesting

[sam]

I just wrote this, it's scala, but basically the same

Executors.newFixedThreadPool(5, new ThreadFactory {
      override def newThread(r: Runnable): Thread = {
         val thread = new Thread(r)
         thread.setUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler {
            override def uncaughtException(t: Thread, e: Throwable): Unit = ???
         })
         thread
      }
   })

[sam]

obviously anything you do in the uncaughtException method would need to propagate out to the thread that the test is using. So a bit of setup work but pretty generic.

[sam]

A PR could provide a TestExecutors instance which does this and reports the errors back to the thread that invoked TestExecutors.fixed(8) or whatever.

[bbaldino]

Is there a good way to do that? Transfer an exception across thread contexts, basically? That's why I was wondering if the TestContext should have a fail method on it or something which matched the behavior of an exception being thrown, but actually I guess we've still got mixed threads there.

[sam]

You could have an AtomicRef[Exception] in the caller thread, and set it from the executors handler

[sam]

Then once the executor has shutdown, check that ref

[bbaldino]

Yeah, I wanted to have it fail immediately though

[sam]

The uncaughtException could kill the thread, which would kill off that task

[bbaldino]

In my use case I spin up lots of tasks--it's a load test of sorts--so I wanted the test to fail on the first task failing

[sam]

Or you could sleep the main thread and notify it when an exception is set on the atomic

[bbaldino]

Yeah, I think it'd have to be something like that...the main test thread is in a coroutine, right?

[bbaldino]

It'd have to wait on an event channel or something I guess

[sam]

Yes, there might be some coroutine equivilent, but in regular java you'd just do .wait() and then the uncaughtException method would set the exception in an atomic ref, and then call .notify

[bbaldino]

Yeah

[sam]

A handoff channel might work

[sam]

rendezvous channel

[bbaldino]

That sounds like a deeper change..I assume now the test coro just waits on the test block to finish...instead it'd have to wait on some event channel which would have success or failure events maybe?

[sam]

well if you have a channel and call receive in the test coroutine, the test coroutine will sleep until it receives something

[bbaldino]

Right

[sam]

and that could be an exception or "all tasks passed"

[bbaldino]

Yeah, exactly