wasyl
05/13/2021, 7:22 PMDescribeSpec({
describe("a") {
println("a")
describe("b") {
println("b")
it("check 1") { println("1") }
it("check 2") { println("2") }
}
describe("c") {
println("c")
it("check 1") { println("1") }
it("check 2") { println("2") }
}
}
})
to print
a // first instance
b
1
2
a // second instance
c
1
2
ab12c12
InstancePerTest:
a ab ab1 ab2 ac1 ac2
InstancePerLeaf
ab1 ab2 ac1 ac2
WhatIWant:
ab12 ac12
sam
05/13/2021, 7:26 PMwasyl
05/13/2021, 7:27 PMroot
as container which has at least one test
then exactlysam
05/13/2021, 7:27 PMwasyl
05/13/2021, 7:29 PMit() {}
in describe spec, and describe/context
are containerssam
05/13/2021, 7:30 PMwasyl
05/13/2021, 7:33 PMsam
05/13/2021, 7:34 PMInstancePerTestSpecRunner:142
change
if (isTarget) {
to
if (isTarget || nested.type == TestType.Test) {
I think that's all that would be requiredwasyl
05/13/2021, 7:38 PMInstancePerTest
runner I see
a
b
1
2
c
1
2
so it seems like it created just one spec instancesam
05/13/2021, 8:19 PMwasyl
05/13/2021, 8:22 PMpublishToMavenLocal
fails at Task :kotest-tests:kotest-tests-native:generateMetadataFileForIosArm32Publication FAILED
and it’s difficult to use different version of just kotest-framework-engine
sam
05/13/2021, 8:34 PMwasyl
05/13/2021, 8:43 PMInstancePerTest
runner I get bunch of
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at ./src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 873
😬if (nested.type == TestType.Test) {
run(t, target)
} else if (isTarget) {
executeInCleanSpec(t).getOrThrow()
} else if (t.description.isOnPath(target.description)) {
run(t, target)
}
and I see aab12ac12
which is close enough
e: but also fails some tests with unfinished coroutines during teardown
so not quitechristophsturm
05/14/2021, 9:49 AMDescribeSpec({
describe("a") {
...
describe("b", isolate:false) {
println("b")
it("check 1") { println("1") }
it("check 2") { println("2") }
}
...
})
wasyl
05/14/2021, 9:56 AMisoloate = false
makes sense, especially if there are technical difficulties without it, ideally I’d like the isolation behavior default for all tests. That said, I still haven’t determined if it’s really worth it 🙂christophsturm
05/14/2021, 10:00 AMwasyl
05/14/2021, 10:05 AMchristophsturm
05/14/2021, 10:06 AMwasyl
05/14/2021, 10:12 AMone possible problem with such a lifecycle is that its not really obviousthat’s for sure, that’s why I wanted to measure first. Although arguably
InstancePerTest
is not obvious either, at least with specs that have dedicated assertions block
why can’t you do singleInstance?Isolating tests is important to me, honestly I don’t even know how to write tests with single instance config, it’d null the biggest benefit of tree-like test structure Kotest offers
sam
05/14/2021, 10:14 AMwasyl
05/14/2021, 10:18 AMsam
05/14/2021, 10:21 AMwasyl
05/14/2021, 10:22 AMoverride val parallelism = 4
in project config class?sam
05/14/2021, 10:23 AMwasyl
05/14/2021, 10:24 AMconcurrentTests/Specs
? Although I already know some of our tests aren’t ready for parallel execution yet due to shared static state in some dependencies. Seems like a noticeable improvement thoughsam
05/14/2021, 10:26 AMchristophsturm
05/14/2021, 10:26 AMOr you can become a kotest contributor and help improve the best test framework on the jvm ;) @christophsturmI’m really hoping that failfast will become one of the (two?) best test frameworks on the jvm 🙂 it may look similar to kotest but i think its absolutely not. there are no features to enable there are no config options there are no different dsls, its for people who mostly care about test speed and think that multi threading is the way to go.
sam
05/14/2021, 10:26 AMchristophsturm
05/14/2021, 10:27 AMsam
05/14/2021, 10:27 AMwasyl
05/14/2021, 10:28 AMInstancePerLeaf
? Then a spec is a test anyway, so no difference?sam
05/14/2021, 10:29 AMchristophsturm
05/14/2021, 10:35 AMsam
05/14/2021, 10:39 AMwasyl
05/14/2021, 10:39 AMchristophsturm
05/14/2021, 10:39 AMcontext("does stuff", isolation: false) {…}
may be a good solutionsam
05/14/2021, 10:40 AMchristophsturm
05/14/2021, 10:41 AMwasyl
05/14/2021, 10:44 AMchristophsturm
05/14/2021, 10:45 AMsam
05/14/2021, 10:46 AMchristophsturm
05/14/2021, 10:48 AMwasyl
05/14/2021, 10:49 AMchristophsturm
05/14/2021, 10:51 AMwasyl
05/14/2021, 11:05 AMchristophsturm
05/14/2021, 11:07 AMwasyl
05/14/2021, 11:07 AMchristophsturm
05/14/2021, 11:09 AMsam
05/14/2021, 11:09 AMwasyl
05/14/2021, 11:09 AMsam
05/14/2021, 11:10 AMchristophsturm
05/14/2021, 11:10 AMwasyl
05/14/2021, 11:10 AMSo you use kotest for the non android bitswe use it for everything that doesn’t require robolectric, so some Android components still qualify, albeit with custom listener to set this global dispatcher state
sam
05/14/2021, 11:11 AMwasyl
05/14/2021, 11:12 AMchristophsturm
05/14/2021, 11:17 AMsam
05/14/2021, 11:18 AMchristophsturm
05/14/2021, 11:19 AMwasyl
05/14/2021, 11:22 AM