I m trying to get Gradle JVM toolchain working for a multi m kotlinlang #gradle

I'm trying to get Gradle JVM toolchain working for...

Alex Spence

11/22/2021, 6:13 PM

I'm trying to get Gradle JVM toolchain working for a multi module project. I was never able to get it working without specifying custom launcher for every task.. now I'm having issues getting KSP to use a custom launcher so I'm trying the global approach again. I'm setting this in my root build.gradle.kts:

Copy code

kotlin {
    jvmToolchain {
        (this as JavaToolchainSpec).apply {
            languageVersion.set(JavaLanguageVersion.of(17))
        }
    }
}

java {
    toolchain {
        languageVersion.set(JavaLanguageVersion.of(17))
    }
}

but I'm getting errors like this When I run gradle using Java 8

Copy code

No matching variant of csdisco.athena.common:data:1.5.1 was found. The consumer was configured to find an API of a library compatible with Java 8, preferably in the form of class files, preferably optimized for standard JVMs, and its dependencies declared externally, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm' but:
             - Variant 'apiElements' capability csdisco.athena.common:data:1.5.1 declares an API of a library, packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm':
                 - Incompatible because this component declares a component compatible with Java 17 and the consumer needed a component compatible with Java 8
             - Variant 'runtimeElements' capability csdisco.athena.common:data:1.5.1 declares a runtime of a library, packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm':
                 - Incompatible because this component declares a component compatible with Java 17 and the consumer needed a component compatible with Java 8

I've tried to specify the kotlin / java toolchain blocks above per project, in allProjects or subProjects but those result in compile errors What is the proper way to configure the toolchain for multi module projects?

Hanno

11/22/2021, 6:48 PM

The error sais you have issues with incompatible java target versions. For example when you use a platform that requires java 8, you cant just use java17. I can't see exactly what your mismatches are, but configuring the target java version has to be done for every subproject that produces jvm stuff, just like you already tried. Lets try again and let us take a look at the compiler errors, they are probably solvable. Is you project public somewhere?

Alex Spence

11/22/2021, 6:57 PM

No the project is private. My issue is that I build in Jenkins which is running on JDK 1.8 but I have a dependency that is targeting JVM 17

Alex Spence

11/22/2021, 6:57 PM

I can solve this with Docker, but when toolchains came out I decided to try it out and speed up my jenkins builds, which it did and worked great.. though my configuration was a bit rough

Alex Spence

11/22/2021, 6:59 PM

Copy code

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

plugins {
    base
    kotlin("jvm")
    id("org.springframework.boot") apply false
    id("io.spring.dependency-management") apply false
    kotlin("plugin.spring") apply false
    kotlin("plugin.serialization") apply false
    id("org.jmailen.kotlinter")
    `version-catalog`
}

tasks.compileJava {
    options.release.set(16)
}

kotlin {
    jvmToolchain {
        (this as JavaToolchainSpec).apply {
            languageVersion.set(JavaLanguageVersion.of("16"))
            vendor.set(JvmVendorSpec.ADOPTOPENJDK)
        }
    }
}

allprojects {

    group = "revenue"


    repositories 
        maven("<https://jitpack.io>")
        maven {
            url = uri("<https://dl.bintray.com/kotlin/ktor>")
        }
        maven {
            url = uri("<https://dl.bintray.com/kotlin/kotlinx>")
        }
    }

}

val service = project.extensions.getByType<JavaToolchainService>()
val customLauncher = service.launcherFor {
    languageVersion.set(JavaLanguageVersion.of("16"))
}


subprojects {
    apply(plugin = "org.jmailen.kotlinter")

    buildDir = File(rootProject.projectDir, "build/gradle/" + project.name)

    tasks.withType<KotlinCompile>().configureEach {
        kotlinOptions {
            jvmTarget = "16"
            freeCompilerArgs = listOf("-Xjsr305=strict", "-Xopt-in=kotlin.RequiresOptIn")
        }
        kotlinJavaToolchain.toolchain.use(customLauncher)
    }

    tasks.withType<JavaCompile>().configureEach {
        options.release.set(16)
    }

    tasks.withType<Test> {
        useJUnitPlatform()
        maxParallelForks = 4
        javaLauncher.set(customLauncher)
    }

    kotlinter {
        ignoreFailures = false
        disabledRules = arrayOf(
            "max_line_length",
            "no-wildcard-imports",
            "parameter-list-wrapping",
            "experimental:argument-list-wrapping",
            "filename"
        )
    }
}

tasks.register("generate") {
    dependsOn(":generator:run")
    finalizedBy(":generated:formatKotlin")
}

tasks.register("generate-tests") {
    dependsOn(":test-generator:run")
    finalizedBy(":core-tests:formatKotlin")
}

Alex Spence

11/22/2021, 7:00 PM

This was the configuration that I finally got working in jenkins, running gradle via 1.8.

Alex Spence

11/22/2021, 7:01 PM

But now I'm using KSP, and this doesn't work anymore. I can't figure out make KSP use the custom launcher

Alex Spence

11/22/2021, 7:01 PM

I posted a similar question in the #ksp channel about that

Alex Spence

11/22/2021, 7:02 PM

Also - if I try to specify a target java version, I get a different error

Alex Spence

11/22/2021, 7:04 PM

Copy code

Could not determine the dependencies of task ':compileKotlin'.
> Could not resolve all dependencies for configuration ':kotlinCompilerPluginClasspathMain'.
   > The new Java toolchain feature cannot be used at the project level in combination with source and/or target compatibility

Alex Spence

11/22/2021, 7:08 PM

This is my attempt at the old project to make KSP use the custom launcher

Alex Spence

11/22/2021, 7:08 PM

Copy code

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

plugins {
    base
    kotlin("jvm")
    id("org.springframework.boot") apply false
    id("io.spring.dependency-management") apply false
    kotlin("plugin.spring") apply false
    kotlin("plugin.serialization") apply false
    id("org.jmailen.kotlinter")
    id("com.google.devtools.ksp")
    `version-catalog`
    idea
}

kotlin {
    jvmToolchain {
        (this as JavaToolchainSpec).apply {
            languageVersion.set(JavaLanguageVersion.of(17))
            vendor.set(JvmVendorSpec.ADOPTOPENJDK)
        }
    }
}

java {
    toolchain {
        languageVersion.set(JavaLanguageVersion.of(17))
        vendor.set(JvmVendorSpec.ADOPTOPENJDK)
    }
}

allprojects {

    group = "revenue"


    repositories {
        maven("<https://jitpack.io>")
        maven {
            url = uri("<https://dl.bintray.com/kotlin/ktor>")
        }
        maven {
            url = uri("<https://dl.bintray.com/kotlin/kotlinx>")
        }
        google()
    }

}

val service = project.extensions.getByType<JavaToolchainService>()
val customLauncher = service.launcherFor {
    languageVersion.set(JavaLanguageVersion.of(17))
}


subprojects {
    apply(plugin = "org.jmailen.kotlinter")

    buildDir = File(rootProject.projectDir, "build/gradle/" + project.name)

    tasks.withType<KotlinCompile>().configureEach {
        kotlinOptions {
            jvmTarget = "17"
            freeCompilerArgs = listOf("-Xjsr305=strict", "-Xopt-in=kotlin.RequiresOptIn")
        }
       kotlinJavaToolchain.toolchain.use(customLauncher)
    }

    tasks.withType<JavaCompile>().configureEach {
        options.release.set(17)
    }

    tasks.withType<com.google.devtools.ksp.gradle.KspTaskJvm> {
        kotlinJavaToolchain.toolchain.use(customLauncher)
    }

    tasks.withType<Test> {
        useJUnitPlatform()
        maxParallelForks = 4
        javaLauncher.set(customLauncher)
    }

    kotlinter {
        ignoreFailures = false
        disabledRules = arrayOf(
            "max_line_length",
            "no-wildcard-imports",
            "parameter-list-wrapping",
            "experimental:argument-list-wrapping",
            "filename"
        )
    }
}

tasks.register("generate") {
    dependsOn(":generator:run")
    finalizedBy(":generated:formatKotlin")
}

tasks.register("generate-tests") {
    dependsOn(":test-generator:run")
    finalizedBy(":core-tests:formatKotlin")
}

tapchicoma

11/22/2021, 7:08 PM

FYI, toolchain when it is not explicetly specified, also sets

jvmTarget

to be equal to the toolchain target. So, in your case, you just need to set

kotlinOptions.jvmTarget

to all

KotlinCompile

tasks

Alex Spence

11/22/2021, 7:13 PM

If I remove the launcher I go back to the errors about the api requested 1.8

Alex Spence

11/22/2021, 7:13 PM

Copy code

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

plugins {
    base
    kotlin("jvm")
    id("org.springframework.boot") apply false
    id("io.spring.dependency-management") apply false
    kotlin("plugin.spring") apply false
    kotlin("plugin.serialization") apply false
    id("org.jmailen.kotlinter")
    id("com.google.devtools.ksp")
    `version-catalog`
    idea
}

kotlin {
    jvmToolchain {
        (this as JavaToolchainSpec).apply {
            languageVersion.set(JavaLanguageVersion.of(17))
            vendor.set(JvmVendorSpec.ADOPTOPENJDK)
        }
    }
}

java {
    toolchain {
        languageVersion.set(JavaLanguageVersion.of(17))
        vendor.set(JvmVendorSpec.ADOPTOPENJDK)
    }
}

allprojects {

    group = "revenue"


    repositories {
        maven("<https://jitpack.io>")
        maven {
            url = uri("<https://dl.bintray.com/kotlin/ktor>")
        }
        maven {
            url = uri("<https://dl.bintray.com/kotlin/kotlinx>")
        }
        google()
    }

}

//val service = project.extensions.getByType<JavaToolchainService>()
//val customLauncher = service.launcherFor {
//    languageVersion.set(JavaLanguageVersion.of(17))
//}


subprojects {
    apply(plugin = "org.jmailen.kotlinter")

    buildDir = File(rootProject.projectDir, "build/gradle/" + project.name)

    tasks.withType<KotlinCompile>().configureEach {
        kotlinOptions {
            jvmTarget = "17"
            freeCompilerArgs = listOf("-Xjsr305=strict", "-Xopt-in=kotlin.RequiresOptIn")
        }
       //kotlinJavaToolchain.toolchain.use(customLauncher)
    }

//    tasks.withType<JavaCompile>().configureEach {
//        options.release.set(17)
//    }

//    tasks.withType<com.google.devtools.ksp.gradle.KspTaskJvm> {
//        kotlinJavaToolchain.toolchain.use(customLauncher)
//    }

    tasks.withType<Test> {
        useJUnitPlatform()
        maxParallelForks = 4
       // javaLauncher.set(customLauncher)
    }

    kotlinter {
        ignoreFailures = false
        disabledRules = arrayOf(
            "max_line_length",
            "no-wildcard-imports",
            "parameter-list-wrapping",
            "experimental:argument-list-wrapping",
            "filename"
        )
    }
}

tasks.register("generate") {
    dependsOn(":generator:run")
    finalizedBy(":generated:formatKotlin")
}

tasks.register("generate-tests") {
    dependsOn(":test-generator:run")
    finalizedBy(":core-tests:formatKotlin")
}

Alex Spence

11/22/2021, 7:13 PM

Results in :

Copy code

No matching variant of csdisco.athena.common:config:1.5.1 was found. The consumer was configured to find an API of a library compatible with Java 8, preferably in the form of class files, preferably optimized for standard JVMs, and its dependencies declared externally, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm' but:
             - Variant 'apiElements' capability csdisco.athena.common:config:1.5.1 declares an API of a library, packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm':
                 - Incompatible because this component declares a component compatible with Java 17 and the consumer needed a component compatible with Java 8
             - Variant 'runtimeElements' capability csdisco.athena.common:config:1.5.1 declares a runtime of a library, packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm':
                 - Incompatible because this component declares a component compatible with Java 17 and the consumer needed a component compatible with Java 8
      > Could not resolve csdisco.athena.common:data:1.5.1.

Alex Spence

11/22/2021, 7:14 PM

Copy code

The consumer was configured to find an API of a library compatible with Java 8

Alex Spence

11/22/2021, 7:14 PM

Toolchain is set to 17, but this is still referencing the JDK version I called gradle with

tapchicoma

11/22/2021, 7:15 PM

could you post consumer

build.gradle.kts

here?

Alex Spence

11/22/2021, 7:16 PM

Copy code

plugins {
    kotlin("jvm")
}

dependencies {

    implementation(libs.bundles.kotlin)
    implementation(libs.logging)
    implementation(libs.bundles.jackson)
    implementation(athenaCommon.config)
    implementation(athenaCommon.data)
    implementation(libs.datadog.statsd.client)

}

Alex Spence

11/22/2021, 7:16 PM

I'm using version catalog

tapchicoma

11/22/2021, 7:16 PM

wait, if posted above is the root

build.gradle.kts

- you are applying toolchain only to this module. You need to apply toolchain for every JVM module

Alex Spence

11/22/2021, 7:16 PM

Alex Spence

11/22/2021, 7:16 PM

So I suspected that was an issue

tapchicoma

11/22/2021, 7:16 PM

also you don't need to configure both Kotlin and Java toolchains - they are equal and configure each other

tapchicoma

11/22/2021, 7:17 PM

just one of them would be enough

Alex Spence

11/22/2021, 7:17 PM

Alex Spence

11/22/2021, 7:17 PM

Can I set the JVM Toolchain in allProjects or subProjects? All my attempts result in compiler failures

Alex Spence

11/22/2021, 7:17 PM

complaining kotlin extension doesn't exist

Alex Spence

11/22/2021, 7:18 PM

Copy code

Extension with name 'kotlin' does not exist. Currently registered extension names: [ext, kotlinter]

Alex Spence

11/22/2021, 7:19 PM

adding this to the top of subProjects gives me this

Alex Spence

11/22/2021, 7:19 PM

Copy code

apply(plugin = "kotlin")

Alex Spence

11/22/2021, 7:19 PM

Copy code

* What went wrong:
An exception occurred applying plugin request [id: 'org.gradle.java-platform']
> Failed to apply plugin 'org.gradle.java-platform'.
   > The "java-platform" plugin cannot be applied together with the "java" (or "java-library") plugin. A project is either a platform or a library but cannot be both at the same time.

Alex Spence

11/22/2021, 7:19 PM

I'm using java-platform to import jackson and aws bom's

tapchicoma

11/22/2021, 7:19 PM

something like this may work:

Copy code

subprojects {
   it.extensions.configure<KotlinTopLevelExtension> {
       jvmToolchain { ... }
   }
}

Alex Spence

11/22/2021, 7:20 PM

Ah - I saw something similar to that in the kotlin codebase while trying to figure out how it configures it

Alex Spence

11/22/2021, 7:21 PM

Copy code

Extension of type 'KotlinTopLevelExtension' does not exist. Currently registered extension types: [ExtraPropertiesExtension, KotlinterExtension]

tapchicoma

11/22/2021, 7:22 PM

Then try:

Copy code

subprojects {
   it.plugins.withId("org.jetbrains.kotlin.jvm") {
       it.extensions.configure<KotlinTopLevelExtension> {
          jvmToolchain { ... }
       }
   }
}

Alex Spence

11/22/2021, 7:26 PM

Ok back to the kspFailure now

Alex Spence

11/22/2021, 7:26 PM

Copy code

e: java.lang.UnsupportedClassVersionError: csdisco/athena/revenue/core/metrics/processor/MetricsAnnotationProcessorProvider has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 52.0

tapchicoma

11/22/2021, 7:27 PM

this means

csdisco/athena/revenue/core/metrics/processor/MetricsAnnotationProcessorProvider

was compiled with target higher then KSP is running on

Alex Spence

11/22/2021, 7:28 PM

Yeah KSP isn't picking up the toolchain I suspect

tapchicoma

11/22/2021, 7:30 PM

yep, it requires plugin authors to add such support

tapchicoma

11/22/2021, 7:32 PM

mm, looking at a sources I see KSP extends Kotlin plugin tasks. Could you share the code or run

./gradlew help --task <ksp_task_name>

Alex Spence

11/22/2021, 7:33 PM

Copy code

./gradlew help --task kspKotlin
Type-safe dependency accessors is an incubating feature.
Type-safe project accessors is an incubating feature.

> Task :help
Detailed task information for kspKotlin

Paths
     :annotation-processor:kspKotlin
     :models:metrics:kspKotlin

Type
     KspTaskJvm (com.google.devtools.ksp.gradle.KspTaskJvm)

Description
     -

Group
     -

tapchicoma

11/22/2021, 7:33 PM

which KSP version are you using?

Alex Spence

11/22/2021, 7:34 PM

Copy code

kspVersion=1.6.0-1.0.1

tapchicoma

11/22/2021, 7:34 PM

then it should pickup toolchain 🤔

tapchicoma

11/22/2021, 7:35 PM

could you share full stacktrace?

Alex Spence

11/22/2021, 7:35 PM

Yeah - I thought so too.. with KAPT I was able to get it working by setting the custom launcher

tapchicoma

11/22/2021, 7:35 PM

kapt tasks should pickup toolchain via extension configuration as well

Alex Spence

11/22/2021, 7:36 PM

I tried this:

Copy code

tasks.withType<com.google.devtools.ksp.gradle.KspTaskJvm> {
        kotlinJavaToolchain.toolchain.use(customLauncher)
    }

Alex Spence

11/22/2021, 7:36 PM

Yeah much better without the custom launcher now

Alex Spence

11/22/2021, 7:36 PM

But that still should have worked, I thought.

Alex Spence

11/22/2021, 7:37 PM

Copy code

> Task :models:metrics:kspKotlin FAILED
e: java.lang.UnsupportedClassVersionError: csdisco/athena/revenue/core/metrics/processor/MetricsAnnotationProcessorProvider has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 52.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.jetbrains.kotlin.cli.jvm.plugins.ServiceLoaderLite.loadImplementations(ServiceLoaderLite.kt:51)
        at org.jetbrains.kotlin.cli.jvm.plugins.ServiceLoaderLite.loadImplementations(ServiceLoaderLite.kt:44)
        at com.google.devtools.ksp.KotlinSymbolProcessingExtension.loadProviders(KotlinSymbolProcessingExtension.kt:68)
        at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:161)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:120)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:96)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:262)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:53)
        at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:113)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:253)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:100)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:58)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:170)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:52)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:92)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:44)
        at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:98)
        at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1618)
        at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
        at sun.rmi.transport.Transport$1.run(Transport.java:200)
        at sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

tapchicoma

11/22/2021, 7:40 PM

Sounds like a bug in Kotlin compiler 🤔 Could you make a repro project and file an issue?

Alex Spence

11/22/2021, 7:41 PM

🙏 1

Alex Spence

11/22/2021, 7:41 PM

in ksp or the kotlin repo ?

tapchicoma

11/22/2021, 7:42 PM

nope, it is here: https://github.com/google/ksp