When I run a single test of a spec that contains multiple tests, Kotest generates output for every i...

# kotest

[julian]

When I run a single test of a spec that contains multiple tests, Kotest generates output for every ignored test.

It's challenging to wade through all that logging when I'm looking for the output of print statements that I've added to the code under test in order to debug.

Is there a way to turn off the logging?

Otherwise, how could I have the output from print statements be associated with the test that was run instead of the spec containing all the tests?

[Jim]

three things:
first, I need to update the docs
second, there's a bug forcing you to set a envvar or sysprop to do what I'm about to suggest I just discovered
third, you can use built in logging to achieve this:

object ProjectConfig : AbstractProjectConfig() {
    override val logLevel: LogLevel = LogLevel.Warn
    override fun extensions(): List<Extension> = listOf(
        object : LogExtension {
            override suspend fun handleLogs(testCase: TestCase, logs: List<LogEntry>) {
                logs.forEach { println(it.level.name + " - " + it.message) }
            }
        }
    )

class TestsWithLogging : FunSpec({
    test("foo") {
        warn { "bar" }
    }

    test("foo2") {
        warn { "barbaz" }
        warn { "bloop" }
        error { "boom" }
    }

[Jim]

[julian]

@Jim Thanks! What's the envvar/sysprop?

[Jim]

Damn it 😂 I'll try to dig that up for you on my phone, super sick rn

[Jim]

https://kotest.io/docs/framework/framework-config-props.html

See logLevel

[Jim]

kotest.framework.loglevel=warn
kotest.framework.loglevel=info
Etc

[julian]

@Jim Took me a minute to realize I needed to add the envvar to the IntelliJ run configuration.
All good now, though.
Thank you!
Hope you feel better!

[Jim]

👍👍 thanks, glad it's working, I have a pr on my laptop to make it so you can use abstract project config

[julian]

Btw, in case anyone else tries the code snippet above, change
error { "boom" } to error("boom").

[Jim]

Hmmm I think the former logs while the later throws, not sure

[julian]

I could find only this definition

public inline fun error(message: Any): Nothing = throw IllegalStateException(message.toString())

error { "boom" } produced this

() -> kotlin.String
java.lang.IllegalStateException: () -> kotlin.String

whereas
error("boom") produces this

boom
java.lang.IllegalStateException: boom

[Jim]

Hmmm

[julian]

Maybe there's an error implementation missing?

[Jim]

https://github.com/kotest/kotest/blob/master/kotest-framework/kotest-framework-engine/src/commonMain/kotlin/io/kotest/engine/test/logging/logs.kt#L59

[Jim]

Maybe an import conflict

[julian]

Oh, shoot! You're right. the error I was using didn't even need to be imported - it's from kotlin.Preconditions. It got picked up by default.

[Big Chungus]

Hmm, setting log level via sys props doesn't seem to do anything. Here's how I've set it up in gradle

tasks {
  withType<Test>().configureEach {
    systemProperty("kotest.framework.loglevel", "info")
  }
}

I've also verified that sys prop is indeed visible from test scope, however no info logs show up in gradle test output in intellij

@Jim any clues? Has anyone actually managed to get this working?

[Jim]

did you add a logger to the AbstractProjectConfig @Big Chungus?

[Big Chungus]

Oh, didn't realise there's no default logger. Let me try.

[Big Chungus]

Wait how do I do that? There's nothing mentioning logger in the override targets other than logLevel

[Big Chungus]

Found it, had to implement and add LogExtension

[Big Chungus]

Sadly kotest.framework.loglevel sys property doesn't seem to do anything as I'm still expected to filter the logs in the extension

[Big Chungus]

Here's what I ended up with to get sys prop working

object KotestConfig : AbstractProjectConfig() {
  override val logLevel: LogLevel
    get() = System.getProperty("kotest.framework.loglevel")?.let(LogLevel::from) ?: LogLevel.Warn

  override fun extensions(): List<Extension> = listOf(
    object : LogExtension {
      override suspend fun handleLogs(testCase: TestCase, logs: List<LogEntry>) {
        logs
          .filter { it.level >= logLevel }
          .forEach { println(it.message) }
      }
    }
  )
}

[Jim]

https://github.com/kotest/kotest/pull/2923

[Big Chungus]

Unfortunately the sys prop doesn't seem to do anything for me. Had to set it in project config afterall

[Jim]

🤔 I wrote a test checking it here: https://github.com/kotest/kotest/pull/2922 - and some changes to wire up project config so I'm surprised that project config is working 😂

[julian]

@Big Chungus I don't think you need .filter { it.level >= logLevel }. I think the logs are pre-filtered by logLevel before they're passed to handleLogs.

[Jim]

They are 👍

[Big Chungus]

Hmm, I could swear they were not for me. I always try to trim my setups as much as possible.

[Jim]

If not then that's a bug and we should open up an issue cause we def have that

[julian]

Hmm... I don't know if something changed since I first applied @Jim’s solution or if I just didn't notice this when I did - But I still continue to see foo is excluded by test filter(s): Excluded by test path filter: 'foo2' when running TestsWithLogging.
Does Jim's solution get rid of this noise for others? Is it just me?

[Jim]

Hmmm I don't think the internal log statements are controlled by the testcase logger interface

[julian]

Ah, ok. So I probably just didn't notice they were still there when I added the LogExtension
Is there another way to control it?

[Jim]

Hmmm I'm not sure, I'll have to check in the fp log function which is what I think it uses

[julian]

What's "fp" stand for?

[julian]

I just noticed that the noise is only present if I run a single test function. If I prefix the test function with f: and run the spec itself, I don't get the noise.
That's fine for my purposes.

[Jim]

It's like an internal functional programming lib in kotest