https://kotlinlang.org
Join Slack
Hi, I just updated my Maven configuration to `kotest-runner-junit5-jvm` 5.7.1 and I realized I start...
# kotest
g
Hi, I just updated my Maven configuration to 
kotest-runner-junit5-jvm
5.7.1 and I realized I started experiencing A LOT of warnings on 
Warning: Kotest autoscan is enabled
. But it also looks like it raises for every test - I can see a lot of them - and that it delays the build a lot. It's like the build is waiting seconds for every warning. 🧵
I tried to solve it by following the instructions at https://kotest.io/docs/framework/project-config.html#runtime-detection but with no result.
Is there anything I can setup on the build to understand what's is hanging on?
c
those delays are due to the autoscanning; perhaps its worsened by your specific configuration. Is it perhaps forking the tests, resulting in a new JVM (and hence redoing the autoscan)? You can disable this (it will be disabled in future kotest versions) by adding the below to a 
kotest.properties
file:
Copy code
kotest.framework.classpath.scanning.config.disable=true
kotest.framework.classpath.scanning.autoscan.disable=true
If you have extensions that need to be registered those can be done in a configuration class, referenced via:
Copy code
kotest.framework.config.fqn=***.testing.KotestConfig
g
It's a thing I already tried, putting the 
kotest.properties
in the 
src/test/resources
folder. But it seemed this stopped the discovery process at all; I mean, it found 0 tests.
I'd also like to understand whether it's something added "recently"; I've checked my project previous builds and it started showing it recently.
When discovering 0 tests the error showed is:
Copy code
[ERROR] There was an error in the forked process
[ERROR] TestEngine with ID 'kotest' failed to discover tests
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR] TestEngine with ID 'kotest' failed to discover tests
Is there anything I'm doing wrong? Maybe in the Surefire configuration, I don't know... 🙄
I bumped down to 5.6.2 and the warning disappeared. They started appearing - and slowing a lot of my build - with 5.7.0.
Just as a reference, since 5.62 my 
maven clean test
lasted less than 2:30m. Starting from 5.7.0 - where the warning first appeared - it takes something like more than 10m. Still I don't get why it's waiting so much time now that the warning appears. 😐
c
Check that test runner isn’t configured to fork on each test. Don’t know the maven specifics for that.
don’t often see Kotlin projects using maven, generally Gradle is used.
s
The warning has nothing to do with bumped time. Can you try 5.7.1, there were two regressions fixed that were slowing things down.
g
Hi @sam thanks for your support. The issue started with 5.7.0 and continues with 5.7.1. 😕
Setting up properties is not helping...
s
We can remove the warning, but it's literally just a println
If the test suite is slow, it's something else
The idea behind the println is that auto scan is slow, and we want to remove it ultimately.
g
Yeah, I saw the source code of the change. It's more than reasonable what you say! 🙂
👍🏻 1
c
the issue appears to be that this (Maven SureFire) test suite is running each test individually (the warning is printed repeatedly), with odd behaviour when scanning is disabled (tests are no longer located).
👌 1
g
Unfortunately it's the only symptom I have at hand...
c
that change was for a use case I had 😉 . The default mirrors the existing behaviour, which is now configurable.
g
So it cannot be that it IS scanning Jars in my build 😋
c
kotest does scan jars to find test cases, etc - that setting is whether nested jars (Spring Boot app in my case) are scanned as well. Not a common case, and not scanned by default.
g
What about using the properties raising the issue about 0 tests? Could it be a hint of something?
c
it’s indeed odd. What does your SureFire configuration look like?
g
Copy code
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <argLine>${surefire.jacoco.args}</argLine>
    </configuration>
</plugin>
c
nothing anywhere in Maven for 
forkCount
or similar?
g
From the effective POM
Copy code
<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.22.2</version>
  <executions>
    <execution>
      <id>default-test</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <argLine>${surefire.jacoco.args}</argLine>
      </configuration>
    </execution>
  </executions>
  <configuration>
    <argLine>${surefire.jacoco.args}</argLine>
  </configuration>
</plugin>
c
what is the value of 
surefire.jacoco.args
property?
g
Copy code
[INFO] surefire.jacoco.args set to -javaagent:/Users/gvespucci/.m2/repository/org/jacoco/org.jacoco.agent/0.8.10/org.jacoco.agent-0.8.10-runtime.jar=destfile=/Users/gvespucci/projects/<OBFUSCATED>/target/jacoco-output/jacoco-unit-tests.exec,excludes=<OBFUSCATED>/Application*.*:<OBFUSCATED>/infrastructure/**:<OBFUSCATED>/**/model/*
Obfuscating proprietary paths 🙂
👍 1
c
generally looks OK.
😭 1
g
Kotest Config is like
Copy code
object KotestProjectConfig : AbstractProjectConfig() {

    override val parallelism = 10

    override fun extensions() = listOf(SpringExtension)

OBFUSCATED
Is there any DEBUG switch I can use?
c
Let’s try this, it helped me align the auto-discovered extensions with doing them manually. Add `kotest.properties`:
Copy code
kotest.framework.dump.config=true
#kotest.framework.classpath.scanning.config.disable=true
#kotest.framework.classpath.scanning.autoscan.disable=true
#kotest.framework.config.fqn=***.testing.KotestConfig
…grab the kotest config dump (this will be with autoscan). Note all the extensions added. btw, that extension should likely be this:
Copy code
override fun extensions(): List<Extension> {
    return listOf(SpringAutowireConstructorExtension)
}
…unless you intend to apply 
SpringExtension
to all test specs (above matches what kotest auto-configures).
g
Let me try with that properties, first 🙂
Where the Kotest dump should be?
c
in the build output
g
Copy code
~~~ Kotest Configuration ~~~
-> Parallelization factor: 10
-> Concurrent specs: null
-> Global concurrent tests: 1
-> Dispatcher affinity: true
-> Coroutine debug probe: false
-> Spec execution order: Lexicographic
-> Default test execution order: Sequential
-> Default test timeout: nullms
-> Default test invocation timeout: nullms
-> Default isolation mode: SingleInstance
-> Global soft assertions: false
-> Write spec failure file: false
-> Fail on ignored tests: false
-> Fail on empty test suite: false
-> Duplicate test name mode: Warn
-> Remove test name whitespace: false
-> Append tags to test names: false
-> Extensions
  - io.kotest.engine.extensions.SystemPropertyTagExtension
  - io.kotest.extensions.spring.SpringAutowireConstructorExtension
  - class io.kotest.engine.config.ApplyConfigFromAbstractProjectConfigKt$applyConfigFromProjectConfig$projectListener$1
  - io.kotest.extensions.spring.SpringTestExtension
  - io.kotest.runner.junit.platform.gradle.GradleClassMethodRegexTestFilter
-> Tags:
Apart from the 
parallelism
I think it's all default...
Since I'm with Maven, is this normal?
Copy code
- io.kotest.runner.junit.platform.gradle.GradleClassMethodRegexTestFilter
s
Yeah that filter just checks for --test flag in gradle
👍 1
o
For reference, I'm seeing the autoscan warnings once per module (Gradle subproject), I'm not seeing any delays, and I have configured Kotest on JVM with parallelism and as much concurrency as possible:
Copy code
object JvmTestConfiguration : CommonTestConfiguration() {
    override val parallelism = Runtime.getRuntime().availableProcessors()
    override var dispatcherAffinity: Boolean? = false
    override val concurrentSpecs = Int.MAX_VALUE
    override val concurrentTests = Int.MAX_VALUE
    // ...
}
👍 1
💡 1
g
Would it be an issue with 
kotest-runner-junit5-jvm
and Maven, then? 🤔
Done a try with 
maven-profiler
, just to give you an idea...
c
could be something in 
kotest-runner-junit5-jvm
. Start with a simple Maven project w/ kotest and a single test, see what happens.
503 Views
Open in Slack
PreviousNext
Powered by

kotlinlang

A modern programming language that makes developers happier.

Powered by