I'd like some help getting Benchmark to work. The ...

# getting-started

[Vinicius]

I'd like some help getting Benchmark to work. The instructions I found aren't working.

[Vinicius]

build.gradle.kts

plugins {
    alias(libs.plugins.kotlinMultiplatform)
    alias(libs.plugins.vanniktech.mavenPublish)
    id("org.jetbrains.kotlinx.benchmark") version "0.4.13"
}

group = "mrvin.kql"
version = "0.1.0"

kotlin {
    jvm()

    sourceSets {
        val jvmMain by getting

        val jvmTest by getting {
            dependencies {
                implementation("org.xerial:sqlite-jdbc:3.49.0.0")
                implementation("org.postgresql:postgresql:42.7.5")
                implementation(kotlin("test"))

                val kotlinBenchmarkVersion = "0.4.13"
                implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:$kotlinBenchmarkVersion")
                implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime-jvm:$kotlinBenchmarkVersion")
            }
        }
    }
}

benchmark {
    targets {
        register("jvm")
    }

    configurations {
        named("main") {
            warmups = 20
            iterations = 10
            iterationTime = 3
            iterationTimeUnit = "s"
        }
    }
}

benchmark test (copied from instruction)

@State(Scope.Benchmark)
open class MyBenchmark {
    private val size = 10
    private val list = ArrayList<Int>()

    @Setup
    fun prepare() {
        for (i in 0..<size) {
            list.add(i)
        }
    }

    @TearDown
    fun cleanup() {
        list.clear()
    }

    @Benchmark
    fun benchmarkMethod(): Int {
        return list.sum()
    }
}

the error im getting:
mrvin@fedora:~/kql$ ./gradlew kql jdbcjvmBenchmark
Reusing configuration cache.

> Task kql jdbcjvmBenchmark FAILED
Running 'main' benchmarks for 'jvm'
Error: Could not find or load main class kotlinx.benchmark.jvm.JvmBenchmarkRunnerKt
Caused by: java.lang.ClassNotFoundException: kotlinx.benchmark.jvm.JvmBenchmarkRunnerKt

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task 'kql jdbcjvmBenchmark'.
> Process 'command '/usr/lib/jvm/jdk-21-oracle-x64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 457ms
3 actionable tasks: 1 executed, 2 up-to-date
Configuration cache entry reused.

[Vinicius]

project hierarchy, if for some reason it makes a difference (it shouldn't).

[Vinicius]

Ok, I made it work. Apparently, it needs to be within its own main module, it cannot be a test module.

[Filipp Zhinkin]

Benchmarks could be placed in a test module, you just need to register appropriate "test" targets instead:

benchmarks {
   targets {
       register("jvmTest") // instead of "jvm"
   }
}

However, it's more common to either put them into a separate source set (https://github.com/Kotlin/kotlinx-benchmark/blob/master/docs/separate-benchmark-source-set.md), or create a separate Gradle sub-project that'll host benchmarks in its main source set.

[Filipp Zhinkin]

And we have the #C057XEG4QRZ channel to discuss various topics related to kotlinx-benchmark