i was thinking why tests are defined in classes in...

# kotest

[christophsturm]

i was thinking why tests are defined in classes instead of just in top level, and i guess the main reason is discovery. but why are they classes and not objects?

[Javier]

object works?

[sam]

Yeah objects should work

[christophsturm]

cool, I never tried it because the examples all use classes. aren’t objects more natural if they are just used for discovery?

[sam]

Yep.

[sam]

You can't reuse vals in objects tho

[sam]

Like the framework can't instantiate new instances

[christophsturm]

object IsolationTest : FunSpec(
    {
        isolationMode = IsolationMode.InstancePerTest
        context("ctx") {
            val uuid = UUID.randomUUID().toString()
            test("1") {
                println("uuid$uuid")
            }
            test("2") {
                println("uuid$uuid")
            }
        }
    }
)

I created a new project with just this test and neither the gradle plugin nor the idea plugin finds it. if i change it to class it works

[christophsturm]

also i was thinking about InstancePerTest vs single instance. the class instance does not really matter to me, because the variables are defined in the body of the context method, and i think most people don’t declare their dependencies on the class.

[sam]

I put all my vals in the instance but that could be me being odd

[christophsturm]

btw, how does that work, i have no idea how this can be implemented., how does it run test 2 in a fresh created context?

[sam]

With classes it can make a new instance

[christophsturm]

my kotest classes don’t even have a body because I use funspec and everything is in the constructor

[sam]

Same thing. Everything in the class is created fresh with a new instance

[christophsturm]

hmm but the uuid field is a local variable in the context method

[sam]

For local variables it's no different to invoking a method twice

[christophsturm]

ok and in the second invocation of context it skips the first test?

[sam]

Yes

[christophsturm]

so it runs the context method twice, and each time it runs only one test.

[sam]

Yep

[sam]

It skips any nested contexts or tests as required

[christophsturm]

now that i understand it its really simple. i guess in never thought it through. and i also think that no other test framework does it like that.

[christophsturm]

or maybe scalatest. surely not rspec

[sam]

Once I wrote it it ended up being surprisingly straightforward

[sam]

It's also why fixtures aren't really needed. Just use vals.

[christophsturm]

yeah

[sam]

I could introduce fixtures too though if people just prefer the syntax

[sam]

In 4.4 there will also be parallelism at the test level (default to off)

[christophsturm]

i was wondering if a syntax without classes is possible. just a context call at the file level. but probably no easy way to discover the tests then

[christophsturm]

maybe i could help out with the paralellism. Is there some work done already?

[sam]

The problem isn't discovering top level tests, because they get compiled as statics in classes anyway

[sam]

The problem is in the syntax, I'm not sure it ends up looking any nicer.

[sam]

Kotlin supports top level classes, objects, functions and vals

[sam]

So you can't just write test("foo") { }

[sam]

because that's a function call, not a function declaration

[sam]

you could do val mytest = test("foo") { } though, but then you need to name the val and the test.

[sam]

I guess we could introduce something like this,
val mytest = test { }

[sam]

which is similar to the factory builders we currently have, which are

val factory = funSpec { 
   test("here) { }
}

[sam]

The downside being you have the junit style problem of having to put your test name into a variable, so it ends up looking like

val myTestWithLongNameTestingThings = ... or you have to back tick the names.

[christophsturm]

hmm you could just put the test method calls into a main method 🙂

[sam]

yeah you could, but then fun main() { } isn't much time saving over class Foo : FunSpec() { }

[sam]

I wonder if .kts kotlin scripts could work

[christophsturm]

classes just somehow feel wrong. maybe thats just a relict of junit type thinking. in the end we just need some group of method calls that can be discovered. like a kts script for example.

[christophsturm]

btw do you have an idea why my test object from above is not found by the kotest runner?

[sam]

I agree. Classes don't get you anything. I mean even moving from function declarations to function invocations is a mind shift for testing.

[sam]

So many tools, like gradle, assume that all tests are method defined in a class.

[sam]

Maybe objects aren't working properly. I thought they were supported, and there's no reason they can't be. We don't have any tests for it though. Tht could be considered a bug and should be easily fixable for 4.3.2

[christophsturm]

ok I’ll file a ticket

[sam]

thanks

[sam]

Kotlin script

[sam]

[sam]

[sam]

That's really nice

[christophsturm]

one reason why I’m thinking of this is for faster startup. when i put this into my kotest config, i see that it takes 2 to 3 seconds until my tests start running:

override fun beforeAll() {
        val uptime = ManagementFactory.getRuntimeMXBean().uptime;
        println("beforeAll started after: $uptime")

[sam]

It could be the scanning phase

[sam]

If you enable KOTEST_DEBUG=true

[sam]

you should see info

[sam]

You would need to set that as an env var and make sure its exported to the forked process.

[christophsturm]

> Task :kotest
OpenJDK 64-Bit Server VM warning: Option AllowRedefinitionToAddDeleteMethods was deprecated in version 13.0 and will likely be removed in a future release.
Starting test discovery scan...
Test discovery completed in 1135ms
After filters there are 13 spec classes
After discovery extensions there are 13 spec classes
Discovery is returning 13 specs
~~~ Kotest Configuration ~~~
-> Parallelization factor: 12
-> Default test timeout: 600000ms
-> Default test order: Sequential
-> Default isolation mode: InstancePerTest
-> Global soft assertations: False
-> Write spec failure file: False
-> Fail on ignored tests: False
-> Spec execution order: SpecExecutionOrder
-> Extensions
  - io.kotest.engine.extensions.SystemPropertyTagExtension
  - io.kotest.core.extensions.RuntimeTagExtension
  - io.kotest.engine.extensions.RuntimeTagExpressionExtension
  - io.kotest.engine.extensions.SpecifiedTagsTagExtension
-> Listeners
  - class io.kotest.engine.config.LoadConfigFromClasspathKt$toDetectedConfig$beforeAfterAllListener$1

>> Kotest
- Mama always said testing was like a box of chocolates. You don't know which ones are gonna fail
- Test plan has 13 specs

invokeBeforeProject
beforeAll startet after: 2525

[christophsturm]

so test discovery is 1.1s and total time to beforeAll is 2.5s

[sam]

which version of kotest

[christophsturm]

thats total jvm uptime.

[christophsturm]

4.3.1

[sam]

to be fair it might be taking 1 second to load the jvm

[sam]

then 1 second to find the specs

[christophsturm]

test discovery could output the uptime too, then its easier to see

[sam]

agreed

[sam]

start time and end time

[christophsturm]

can i just use kotest from a composite build and run it via the gradle kotest plugin?

[christophsturm]

then i can try with kotest master

[sam]

I'm not sure what you mean

[christophsturm]

use kotest in my project via a composite gradle build

[sam]

I don't know what you mean by a composite build.

By the way, using a .kts file, with this syntax, the tests are output to a class per test. So it would be possible to find the tests easily using the regular discovery mechanism.

[sam]

The vals for configuration are placed into another class though, so that might require some trickery. Or different syntax.

[christophsturm]

composite build = includeBuild("../kotest") in settings.gradle.kts

[sam]

I didn't know that existed.

[christophsturm]

thats super useful to try local changes

[christophsturm]

https://docs.gradle.org/current/userguide/composite_builds.html#defining_composite_builds

[sam]

Yes it looks super useful. Gonna read about that.

[sam]

The gradle plugin only requires you to have

io.kotest.engine.launcher.MainKt

and

io.kotest.engine.reporter.TaycanConsoleReporter

on your classpath, and then the usual kotest stuff like framework. So it should work.