👋 I'm trying to understand a strange behaviour of the Kotlin Compiler daemon. In our multi-module project, we depend on an included build that contains our convention plugins:

// settings.gradle.kts

pluginManagement {
    includeBuild("build-logic")
}

When running a classic build, a new Kotlin Compiler daemon is spawn to compile the

build-logic

Kotlin classes, as expected. But when Gradle moves to the actual project compilation, a new Kotlin Compiler daemon is spawn again, and the previous one is not reused, nor killed. This is unexpected to me. By looking at the JVM arguments and system properties of both daemons (with VisualVM), they are almost identical. The only difference is on

java.class.path/sun.java.command

where on one hand it references Kotlin jars in version

1.9.0

versus

1.9.10

on the other. Is this the reason why the daemon is not reused? And if yes, how can this be changed? By looking at these docs:

Avoid specifying a version for the

kotlin-dsl

plugin

And the corresponding changelogs seems empty. So I'm not sure what to do about this, and would love to get some help 🙂

If that is the case, then it is probably because you productively use Kotlin 1.9.10, while the Gradle built-in Kotlin version for the build logic is 1.9.0. You could downgrade your project Kotlin version to 1.9.0, then it might be reused. Like it is, you use two different Kotlin version, so you have two different compiler daemons.

Indeed, we rely on 1.9.10 in our project. Forcing the

kotlin-dsl

to 4.1.2 made the daemon reused and does not seem to introduce any regression 👍

You should never do that though

😬

This can break many things. Gradlew controls the Kotlin version for build logic. It also forces the shipped Kotlin libs. So you end up with a mix of 1.9.0 and 1.9.10 for the build logic and can get quite some subtle or not so subtle problems.

Understood. If I downgrade the build logic (included build and main projects) to use Kotlin 1.9.0 would I still be able to use Kotlin 1.9.10 for the app code, or do they all need to be aligned?

You did have 1.9.0 in the build logic before. And yes, you can just fine use 1.9.10 for the production code. You just do not get compile daemon reuse.

I think I am missing one piece of the puzzle here 😅. To me, there are at least 3 places where I can configure Kotlin versions: • the included build (aka my

build-logic

) which is requested by

kotlin-dsl

plugin (1.9.0 in this case) • the main build (aka, all by

gradle.kts

files of all my other modules) which comes from the root

build.gradle.kts

(1.9.10 in this case)

plugins {
  alias(libs.plugins.kotlin.jvm) apply false
}

• the app source code (aka

kotlin-stdlib*

) which comes from the applied plugin Or are these last two items always identical?

Sure, what you apply there is the plugin that cares about building your production code

Ok, then I'm still not sure how I can downgrade the build version but at the same time keeping a more recent version in source code 🤔

You don't need to downgrade anything.

build-logic

uses 1.9.0 as defined through

kotlin-dsl

. Executing the build scripts of the main build uses 1.9.0 as defined through Gradle. The 1.9.10 plugin you apply makes your production classes compiled with 1.9.10.

Executing the build scripts of the main build uses 1.9.0 as defined through Gradle.

I think this is the part that I don't get 😅

The Kotlin plugin version you apply has nothing to do with the runtime environment the buidscript is running in. The buildscripts are running with the bundled Kotlin version. The Kotlin plugin runs on that and sets up tasks that compile and run your production code with 1.9.10.

Ok, but where does the embedded Kotlin version running the main build scripts comes from? Gradle? Because according to this table, Gradle 8.3 uses Kotlin 1.9.0. But we I run my build, the Kotlin Compile Daemon mentions Kotlin 1.9.10 jars.

You said you have two Kotlin compile daemons, one for the build logic using 1.9.0, one for the production code using 1.9.10, which is exactly what I described. Yes, it is comding from Gradle, the Kotlin runtime for the build logic is built-in in the Gradle version you are using.

I think I get it, but my main goal was to avoid having 2 daemons in the first place (for memory constraints reasons).

As a workaround you can use Gradle 8.4 rc 1/2, which uses Kotlin 1.9.10 🤷🏻‍♂️

my main goal was to avoid having 2 daemons

Well, for that you have to align the versions and thus downgrade your production Kotlin version to 1.9.0, or use Gradle 8.4, yes. But imho you really shouldn't couple these versions.

Then there might be alternative workarounds that I am not aware of, because we keep raising the amount of ram needed, but it's hard to keep up with that pace (we are currently maxing out at 32g...). One could be to extract this

build-logic

into a dedicated project, and publish it somewhere to not have to compile it from the main project? Another one that I'm thinking about is, would it be possible to override some kind of Kotlin daemon timeout (time after which it is killed if unused)? Because at the moment, it is kept alive for at least the whole duration of the build, and hangs onto ~1 to 2g of RAM.

--no-daemon

does not seem to affect it, and

gradle.properties

inside the

build-logic

are ignored 🤷

Yes for 1. No idea for 2.

--no-daemon

is just about the Gradle build daemon, could well be that the Kotlin compile deamon survives that.

There seems to be a

--daemon-autoshutdownIdleSeconds=7200

given as argument to the Kotlin Compile Daemon. But this is probably not configurable (at the moment) https://youtrack.jetbrains.com/issue/KT-50510/Gradle-Add-the-possibility-to-change-daemon-autoshutdownIdleSeconds Anyway, thanks a lot for all these informations 🙏

You could choose the in-process compiler execution strategy for the build-logic module, then it won’t require the daemon at all. That strategy doesn’t yet support incremental compilation, however if you don’t change the module too frequently, you’ll be well. https://kotlinlang.org/docs/gradle-compilation-and-caches.html#defining-kotlin-compiler-execution-strategy

But iirc, this is discouraged 🙂

Right, the current implementation of the in-process strategy is suboptimal, however we have plans to improve it

What do you mean by suboptimal? This is meant for CI only, so I suppose incremental compilation might no be that useful. But if it results in more flakiness or even more RAM consumption at the benefit of only not spawning an extra VM, that might not be worth.

Currently it’s creating a new class loader for each compilation task, that may cause higher memory consumption and much slower compilation because JVM shall load classes and perform JIT compilation for the compiler classes almost from scratch. So I won’t recommend using it for the main build as I consider it consists of at least a few modules. However, it’s ok to use it for

buildSrc

or the single

build-logic

module. It shouldn’t cause any problems in this case if you’re ok with not having incremental compilation for that module.

Indeed we have ~1k modules on the main project 😅