https://kotlinlang.org logo
Title
c

christophsturm

12/12/2020, 12:22 PM
i was thinking why tests are defined in classes instead of just in top level, and i guess the main reason is discovery. but why are they classes and not objects?
j

Javier

12/12/2020, 12:29 PM
object works?
s

sam

12/12/2020, 2:52 PM
Yeah objects should work
c

christophsturm

12/12/2020, 3:23 PM
cool, I never tried it because the examples all use classes. aren’t objects more natural if they are just used for discovery?
s

sam

12/12/2020, 3:25 PM
Yep.
You can't reuse vals in objects tho
Like the framework can't instantiate new instances
c

christophsturm

12/12/2020, 3:49 PM
object IsolationTest : FunSpec(
    {
        isolationMode = IsolationMode.InstancePerTest
        context("ctx") {
            val uuid = UUID.randomUUID().toString()
            test("1") {
                println("uuid$uuid")
            }
            test("2") {
                println("uuid$uuid")
            }
        }
    }
)
I created a new project with just this test and neither the gradle plugin nor the idea plugin finds it. if i change it to class it works
also i was thinking about InstancePerTest vs single instance. the class instance does not really matter to me, because the variables are defined in the body of the context method, and i think most people don’t declare their dependencies on the class.
s

sam

12/12/2020, 3:53 PM
I put all my vals in the instance but that could be me being odd
c

christophsturm

12/12/2020, 3:53 PM
btw, how does that work, i have no idea how this can be implemented., how does it run test 2 in a fresh created context?
s

sam

12/12/2020, 3:53 PM
With classes it can make a new instance
c

christophsturm

12/12/2020, 3:54 PM
my kotest classes don’t even have a body because I use funspec and everything is in the constructor
s

sam

12/12/2020, 3:55 PM
Same thing. Everything in the class is created fresh with a new instance
c

christophsturm

12/12/2020, 3:55 PM
hmm but the uuid field is a local variable in the context method
s

sam

12/12/2020, 3:56 PM
For local variables it's no different to invoking a method twice
c

christophsturm

12/12/2020, 3:57 PM
ok and in the second invocation of context it skips the first test?
s

sam

12/12/2020, 3:58 PM
Yes
c

christophsturm

12/12/2020, 3:58 PM
so it runs the context method twice, and each time it runs only one test.
s

sam

12/12/2020, 3:58 PM
Yep
It skips any nested contexts or tests as required
c

christophsturm

12/12/2020, 4:00 PM
now that i understand it its really simple. i guess in never thought it through. and i also think that no other test framework does it like that.
or maybe scalatest. surely not rspec
s

sam

12/12/2020, 4:02 PM
Once I wrote it it ended up being surprisingly straightforward
It's also why fixtures aren't really needed. Just use vals.
c

christophsturm

12/12/2020, 4:03 PM
yeah
s

sam

12/12/2020, 4:03 PM
I could introduce fixtures too though if people just prefer the syntax
In 4.4 there will also be parallelism at the test level (default to off)
c

christophsturm

12/12/2020, 4:04 PM
i was wondering if a syntax without classes is possible. just a context call at the file level. but probably no easy way to discover the tests then
maybe i could help out with the paralellism. Is there some work done already?
s

sam

12/12/2020, 4:08 PM
The problem isn't discovering top level tests, because they get compiled as statics in classes anyway
The problem is in the syntax, I'm not sure it ends up looking any nicer.
Kotlin supports top level classes, objects, functions and vals
So you can't just write
test("foo") { }
because that's a function call, not a function declaration
you could do
val mytest = test("foo") { }
though, but then you need to name the val and the test.
I guess we could introduce something like this,
val mytest = test { }
which is similar to the factory builders we currently have, which are
val factory = funSpec { 
   test("here) { }
}
The downside being you have the junit style problem of having to put your test name into a variable, so it ends up looking like
val myTestWithLongNameTestingThings = ...
or you have to back tick the names.
c

christophsturm

12/12/2020, 4:14 PM
hmm you could just put the test method calls into a main method 🙂
s

sam

12/12/2020, 4:14 PM
yeah you could, but then
fun main() { }
isn't much time saving over
class Foo : FunSpec() { }
I wonder if .kts kotlin scripts could work
c

christophsturm

12/12/2020, 4:16 PM
classes just somehow feel wrong. maybe thats just a relict of junit type thinking. in the end we just need some group of method calls that can be discovered. like a kts script for example.
btw do you have an idea why my test object from above is not found by the kotest runner?
s

sam

12/12/2020, 4:17 PM
I agree. Classes don't get you anything. I mean even moving from function declarations to function invocations is a mind shift for testing.
So many tools, like gradle, assume that all tests are method defined in a class.
Maybe objects aren't working properly. I thought they were supported, and there's no reason they can't be. We don't have any tests for it though. Tht could be considered a bug and should be easily fixable for 4.3.2
c

christophsturm

12/12/2020, 4:18 PM
ok I’ll file a ticket
s

sam

12/12/2020, 4:18 PM
thanks
Kotlin script
That's really nice
c

christophsturm

12/12/2020, 4:21 PM
one reason why I’m thinking of this is for faster startup. when i put this into my kotest config, i see that it takes 2 to 3 seconds until my tests start running:
override fun beforeAll() {
        val uptime = ManagementFactory.getRuntimeMXBean().uptime;
        println("beforeAll started after: $uptime")
s

sam

12/12/2020, 4:22 PM
It could be the scanning phase
If you enable KOTEST_DEBUG=true
you should see info
You would need to set that as an env var and make sure its exported to the forked process.
c

christophsturm

12/12/2020, 4:25 PM
> Task :kotest
OpenJDK 64-Bit Server VM warning: Option AllowRedefinitionToAddDeleteMethods was deprecated in version 13.0 and will likely be removed in a future release.
Starting test discovery scan...
Test discovery completed in 1135ms
After filters there are 13 spec classes
After discovery extensions there are 13 spec classes
Discovery is returning 13 specs
~~~ Kotest Configuration ~~~
-> Parallelization factor: 12
-> Default test timeout: 600000ms
-> Default test order: Sequential
-> Default isolation mode: InstancePerTest
-> Global soft assertations: False
-> Write spec failure file: False
-> Fail on ignored tests: False
-> Spec execution order: SpecExecutionOrder
-> Extensions
  - io.kotest.engine.extensions.SystemPropertyTagExtension
  - io.kotest.core.extensions.RuntimeTagExtension
  - io.kotest.engine.extensions.RuntimeTagExpressionExtension
  - io.kotest.engine.extensions.SpecifiedTagsTagExtension
-> Listeners
  - class io.kotest.engine.config.LoadConfigFromClasspathKt$toDetectedConfig$beforeAfterAllListener$1

>> Kotest
- Mama always said testing was like a box of chocolates. You don't know which ones are gonna fail
- Test plan has 13 specs

invokeBeforeProject
beforeAll startet after: 2525
so test discovery is 1.1s and total time to beforeAll is 2.5s
s

sam

12/12/2020, 4:27 PM
which version of kotest
c

christophsturm

12/12/2020, 4:28 PM
thats total jvm uptime.
4.3.1
s

sam

12/12/2020, 4:28 PM
to be fair it might be taking 1 second to load the jvm
then 1 second to find the specs
c

christophsturm

12/12/2020, 4:29 PM
test discovery could output the uptime too, then its easier to see
s

sam

12/12/2020, 4:29 PM
agreed
start time and end time
c

christophsturm

12/12/2020, 4:30 PM
can i just use kotest from a composite build and run it via the gradle kotest plugin?
then i can try with kotest master
s

sam

12/12/2020, 4:31 PM
I'm not sure what you mean
c

christophsturm

12/12/2020, 4:32 PM
use kotest in my project via a composite gradle build
s

sam

12/12/2020, 4:33 PM
I don't know what you mean by a composite build. By the way, using a .kts file, with this syntax, the tests are output to a class per test. So it would be possible to find the tests easily using the regular discovery mechanism.
The vals for configuration are placed into another class though, so that might require some trickery. Or different syntax.
c

christophsturm

12/12/2020, 4:34 PM
composite build =
includeBuild("../kotest")
in settings.gradle.kts
s

sam

12/12/2020, 4:34 PM
I didn't know that existed.
c

christophsturm

12/12/2020, 4:35 PM
thats super useful to try local changes
s

sam

12/12/2020, 4:35 PM
Yes it looks super useful. Gonna read about that.
The gradle plugin only requires you to have
io.kotest.engine.launcher.MainKt
and
io.kotest.engine.reporter.TaycanConsoleReporter
on your classpath, and then the usual kotest stuff like framework. So it should work.