I'm running into more troubles after migrating to ...

# kotest

[Sebastian Schuberth]

I'm running into more troubles after migrating to Kotest 5.0.0. Can it be that the order in which beforeTest from a TestListener vs. a function from within the test spec are being called? I'm now seeing beforeTest from a `TestListener`being caller after the beforeTest overridden in a test spec, whereas I believe the order was different before. Is that expected?

[sam]

It could be the order got changed. No guarantees are given on the order in which the same type of callbacks are invoked. Although we could introduce a guarantee.

[Sebastian Schuberth]

Yes, I definitely think order should be guaranteed and documented. Also, having special registerFirst() or registerLast() function could help to register multiple listeners in order.

[sam]

You could also just wrap all your listeners in a CompositeListener class which executes in a given order.

[sam]

We could provide such a wrapper.

[sam]

It's going to get very complicated if we have registerFirst in addition to all the other callbacks

[Sebastian Schuberth]

A composite listener would not help me right away, as I'm using a test class hierarchy with a mix of listeners and overridden callback functions. Let's see how I can clean this up...

[Sebastian Schuberth]

No guarantees are given on the order in which the same type of callbacks are invoked.

Is that also the case when I register listeners A and B by passing an ordered list to register()? Is there no guarantee that A.beforeTest() is called before B.beforeTest()?

[sam]

Technically no guarantees, practically, in order. All extensions/listeners end up in this list: https://github.com/kotest/kotest/blob/5be5cbef245bcfc1a1665c5196edb7f21a198ed8/kot[…]rc/commonMain/kotlin/io/kotest/core/config/ExtensionRegistry.kt

So they will definitely be executed in order. We could extend that guarantee to 5.0.0, since it's the current behavior, and add a test / docs for it.

[Sebastian Schuberth]

And practically, how about any listeners from extensions vs. overridden functions? Are overridden function always called before any listeners from the extension registry?

[sam]

This is the current order

[sam]

return spec.extensions() + // overriding the extensions function in the spec
   spec.listeners() + // overriding the listeners function in the spec
   spec.functionOverrideCallbacks() + // dsl
   spec.registeredExtensions() + // added to the spec via register
   registry.all() // globals

[sam]

https://github.com/kotest/kotest/blob/master/kotest-framework/kotest-framework-eng[…]st/engine/extensions/guarantees/BeforeTestExecutionOrderTest.kt

[sam]

There's your beforeTest guarantees

[Sebastian Schuberth]

Thanks a lot! Would it make sense to also add a override fun beforeTest() directly at the FunSpec level?

[sam]

yep can do

[sam]

I'll sprinkle a few about.

[sam]

Done, if you want to check

[Sebastian Schuberth]

Awesome, thanks!

[sam]

No problem