Is there a way to set `--release` for the `KotlinC...

# gradle

[Vampire]

Is there a way to set --release for the KotlinCompile task and / or set the Java Toolchain for a specific KotlinCompile task like for Java-centric tasks like JavaCompile?

I tried to provide a better and future-proof solution for #2218 of kotlinx.serialization (https://github.com/Kotlin/kotlinx.serialization/issues/2218).

For that I set the java { toolchain { ... } } to Java 8 and just set the specific JavaCompile task that compiles the module-info.java file to Java 11 toolchain with --release 9. (Because there is no Java 9 toolchain for certain macOS architectures.

But the consequence of this is, that the KotlinCompile tasks ignore the module-info.class files. Before, the KotlinCompile task considered the module-info.class files of compile dependencies, so if you for example remove the exports kotlinx.serialization; from `core`'s module-info.java, the compilation of formats/json failed due to the missing exports in core. Now as Java 8 is used as toolchain, the module-info.class of core is ignored and the compilation succeeds and would then only fail at runtime when using JPMS.

So a solution would probably either need to use a Java 11 toolchain for the KotlinCompile task, but somehow being able to specify --release 8, so that the compilation happens against the correct Java 8 API. Alternatively, another KotlinCompile task would be necessary - for exampled hooked to the check lifecycle task - that compiles all files again but with Java 11, so that for the real build it compiles against the Java 8 API, but as a verification for the JPMS configuration it is compiled again with Java 11.

Using a Java 11 toolchain with jvmTarget 8 is not a proper solution as that is basically what is done now. Actually in this specific case it would work as the API that has changed here was changed in Java 13. But there might be other cases now or in the future, so it should really compile against the proper Java 8 API like the --release option is doing for JavaCompile.

[Sam]

The Kotlin compiler does have an -Xjdk-release option; does that help? https://kotlinlang.org/docs/compiler-reference.html#xjdk-release-version

[Vampire]

Sounds promising in absence of something built-in like described above. Will try that with the freeCompilerArgs, thanks.

[Vampire]

Hm, it works for specifying --release, but unfortunately not sufficiently.
If I add that arg to the current build setup, core does not compile anymore as it says maaaany symbols declared in module kotlin.stdlib which is not depended on and unnamed module which is not read.

And if I add it to the setup I changed it to, changing the toolcahin to 11 (or 17 to eventually reproduce the problematic outcome) and setting jvmTarget and -Xjdk-release to 1.8, now produces the intended bytecode, but still does does not complain about the module-info problem 😞

[mbonnin]

The Kotlin multiplatform plugin allows multiple JVM targets. Maybe you could use that?

[Vampire]

Maybe, but I'm practically inexperienced with KMM, so hope for some more information than "maybe you could" 🙂

[Vampire]

Besides that "Dependencies on such a multiplatform library may fail to resolve because it isn't clear which target to choose." sounds like it might be a bad idea 😕

[mbonnin]

kotlin {
    jvm("java11")
    jvm("java8")
}

tasks.withType<KotlinJvmCompile>().configureEach {
    if (name.contains("Java8")) {
        compilerOptions.jvmTarget.set(JvmTarget.JVM_1_8)
    } else {
        compilerOptions.jvmTarget.set(JvmTarget.JVM_11)
    }
}

This works for me

[mbonnin]

If you're going that route, you're going to bump into https://youtrack.jetbrains.com/issue/KT-56019 but nothing a good workaround can't solve

[Vampire]

Thanks, maybe I'll try it that way, try to do some attribute to trick resolution and see what the reviewer says 😄

[Vampire]

Ah, yeah, that issue is probably what the doc sentence I quoted referred to

[tapchicoma]

There is KotlinApiPlugin plugin which exposes methods to create KotlinJvmCompile task

[Vampire]

Hm, thanks @tapchicoma.
Can you elaborate / help a bit more how it is supposed to be used?
Like this, besides that Project#plugins shouldn't be used?

apply<KotlinBaseApiPlugin>()
val verifyModuleTaskName = "verify${compileModuleTaskName.removePrefix("compile").capitalize()}"
val verifyModuleTask = plugins.findPlugin(KotlinBaseApiPlugin::class)!!.registerKotlinJvmCompileTask(verifyModuleTaskName)
verifyModuleTask {
    group = VERIFICATION_GROUP
    libraries.from(compileKotlinTask.libraries)
    source(compileKotlinTask.sources)
    source(moduleInfoSourceFile)
    kotlinOptions {
        jvmTarget = "9"
        freeCompilerArgs += "-Xjdk-release=9"
    }
}
tasks.named("check") {
    dependsOn(verifyModuleTask)
}

Besides that it complains about missing value of ownModuleName.
So assuming above might be correct, what else would I need to set for it to get it going?

[tapchicoma]

I think you also need to wire from compileKotlinTask compilerOptions.moduleName (ownModuleName could be the same as moduleName)

[Vampire]

But generally the approach is correct?

[Vampire]

I did try with moduleName.set(verifyModuleTaskName), but that didn't make the error go away

[Vampire]

moduleName.set(compileKotlinTask.moduleName) also does not help

[Vampire]

Oh, wait, you said in the options

[Vampire]

But

kotlinOptions {
    moduleName = compileKotlinTask.moduleName.get()
}

also does not change a thing

[tapchicoma]

you also need to wire `ownModuleName`: ownModuleName.set(compileKotlinTask.compilerOptions.moduleName)

[tapchicoma]

KotlinJvmTask.moduleName is deprecated in favor of KotlinJvmTask.compilerOptions.moduleName

[Vampire]

On what? IJ does not find ownModuleName, neither on the task, nor the options

[tapchicoma]

Which KGP version project is using?

[Vampire]

It is the kotlinx.serialization build

[Vampire]

So should be 1.8.21

[Vampire]

On mobile right now

[tapchicoma]

let me check what should be the approach in this case

[tapchicoma]

aha, so ownModuleName is not a part of public api in KotlinJvmCompile task. Unfortunately in this case you need to fallback to use our internal org.jetbrains.kotlin.gradle.tasks.KotlinCompile implementation

[Vampire]

So there is no way to add a custom compile task without using internals? That is quite sad 😭

[tapchicoma]

ownModuleName is scheduled to be removed in 2.0. Though this is good point that created task could not be used as is. Could you open an issue?

[Vampire]

Sure: https://youtrack.jetbrains.com/issue/KT-60541

[Vampire]

And while we are at it: https://youtrack.jetbrains.com/issue/KT-60542 🙂

[Vampire]

So, what I have now is this, does that look "proper"?

apply<KotlinBaseApiPlugin>()
val verifyModuleTaskName = "verify${compileModuleTaskName.removePrefix("compile").capitalize()}"
val verifyModuleTask = plugins.findPlugin(KotlinBaseApiPlugin::class)!!.registerKotlinJvmCompileTask(verifyModuleTaskName)
verifyModuleTask {
    group = VERIFICATION_GROUP
    libraries.from(compileKotlinTask.libraries)
    source(compileKotlinTask.sources)
    source(sourceFile)
    destinationDirectory.set(temporaryDir)
    moduleName.set(compileKotlinTask.moduleName)
    multiPlatformEnabled.set(compileKotlinTask.multiPlatformEnabled)
    kotlinOptions {
        jvmTarget = "9"
        freeCompilerArgs += "-Xjdk-release=9"
    }
    this as KotlinCompile
    ownModuleName.set(compileKotlinTask.moduleName)
}
tasks.named("check") {
    dependsOn(verifyModuleTask)
}

And if yes, how do I make it work? 😄
Because now I get several

e: file///.../kotlinx.serialization/formats/json/commonMain/src/kotlinx/serialization/json/Json.kt69:6 Declaration annotated with '@OptionalExpectation' can only be used in common module sources

Comparing the `--info`output of compileKotlinTask and verifyKotlinJvmModule suggests they are compiling the same files, except for the additional module info file.

[Vampire]

This is really much harder than I thought it should be. 😄

[tapchicoma]

aaah, it is also an MPP project. Yeah - this will be harder 😅

[Vampire]

🙈

[Vampire]

Heeelp me 🙂

[tapchicoma]

Soooo, first you also need to wire common sources and enable MPP-compilation. The problem that related common sources input is internal.... So following hack may work:

verifyModuleTask {
    @Suppress("INVISIBLE_MEMBER")
    commonSourceSet.from(compileKotlinTask.commonSourceSet)
    multiplatformEnabled.set(true)
}

[Vampire]

Yay, seems to work fine from a cursory look, thanks.
So current version:

apply<KotlinBaseApiPlugin>()
val verifyModuleTaskName = "verify${compileModuleTaskName.removePrefix("compile").capitalize()}"
val verifyModuleTask = plugins.findPlugin(KotlinBaseApiPlugin::class)!!.registerKotlinJvmCompileTask(verifyModuleTaskName)
verifyModuleTask {
    group = VERIFICATION_GROUP
    libraries.from(compileKotlinTask.libraries)
    source(compileKotlinTask.sources)
    source(sourceFile)
    destinationDirectory.set(temporaryDir)
    moduleName.set(compileKotlinTask.moduleName)
    multiPlatformEnabled.set(compileKotlinTask.multiPlatformEnabled)
    kotlinOptions {
        jvmTarget = "9"
        freeCompilerArgs += "-Xjdk-release=9"
    }
    this as KotlinCompile
    ownModuleName.set(compileKotlinTask.moduleName)
    @Suppress("INVISIBLE_MEMBER")
    commonSourceSet.from(compileKotlinTask.commonSourceSet)
}
tasks.named("check") {
    dependsOn(verifyModuleTask)
}

It detects if I remove exports kotlinx.serialization; from core and it detects if I remove requires transitive kotlinx.serialization.core; from formats/json.

[Vampire]

Interesting to know that you ignore internal at will 😄

[Vampire]

Should the commonSourceSet requirement be a separate issue, is it intended, or can it be considered part of KT-60541?

[tapchicoma]

This api was not intended to be used in MPP projects 😅

[Vampire]

Change intention then 😄

[Vampire]

Or is there an MPP alternative?

[Vampire]

Or should there be a feature request for an MPP version of it?

[Vampire]

I added it to KT-60541 for now

[Vampire]

Thanks for all your help

[tapchicoma]

No, there is no MPP alternative. Initially this API was exposed for AGP plugin so it could configure Kotlin compilation withing itself without requirement to apply kotlin-android plugin. We will consider fixing it 👍

[Vampire]

Can I disable incremental compilation?
There might be a bug with it and module info java files

[tapchicoma]

yes, set KotlinCompile.incremental = false

[Vampire]

Feared so, so another instant of internal API needed 🙈

[Vampire]

Damn, it only half-helps.

With and without incremental, just changing `core`'s module-info.java does not make the task out-of-date, it stays UP-TO-DATE and only forcing --rerun makes it run and fail accordingly.

And with incremental, just changing `format/json`'s module-info.java, the task does execute, but does not fail although it should, due to the incrementality. Using --rerun again makes the task run non-incrementally and fail properl.y

So two more bugs I'd say, that changes in the module-info.class of dependencies does not make the task out-of-date and the task needs to run non-incrementally automatically if the compiled module-info.java changes. Will create those too.

[Vampire]

Any good idea how to mitigate those, especially the module-info.class in dependency not making the task out-of-date?

[tapchicoma]

You could disable up-to-date checks at all for the task

[tapchicoma]

It is available in DefaultTask

[Vampire]

Sure, I meant a non-big-hammer method 😄

[tapchicoma]

aha, you need also call source(kotlinCompileTask.javaSources) so task with track changes in .java files

[tapchicoma]

calling source(..) separate sources into 3 task inputs: sources (.kt, .kts files), javaSources (.java files), scriptSources (custom Kotlin script files)

[Vampire]

I don't think that will help.
kotlinCompileTask in my case does not contain the module-info.java anyway.
I already add the module-info.java to the new task as the is the main purpose of the task.
The --info output also says that the module-info.java is compiled along.
It just does not make the Kotlin files recompiled when only the module-info.java changes.

And I would not expect that change to care about the module-info.class files in the dependencies anyway.

[Vampire]

What I have now is incremental = false for the incrementality problem and for the module-info.class in the dependencies

inputs.files(
    libraries.asFileTree.elements.map { libs ->
        libs
            .filter { it.asFile.exists() }
            .map {
                zipTree(it.asFile).filter { it.name == "module-info.class" }
            }
    }
).withPropertyName("modularInfosOfLibraries")

[Vampire]

This seems to mitigate both problems

[Vampire]

I also added source(compileKotlinTask.javaSources) to be sure in case there are other Java files in that source task.

But I don't see compileKotlinTask.scriptSources

[Vampire]

Ah, it's again not public API, but internal in KotlinCompile, so

@Suppress("INVISIBLE_MEMBER")
source(compileKotlinTask.scriptSources)

[Vampire]

Ok, there we are:
https://youtrack.jetbrains.com/issue/KT-60582
https://youtrack.jetbrains.com/issue/KT-60583