https://kotlinlang.org
Join Slack
:wave: I'm trying to understand a strange behaviour of the Kotlin Compiler daemon. In our multi-modu...
# gradle
s
👋 I'm trying to understand a strange behaviour of the Kotlin Compiler daemon. In our multi-module project, we depend on an included build that contains our convention plugins:
Copy code
// settings.gradle.kts

pluginManagement {
    includeBuild("build-logic")
}
When running a classic build, a new Kotlin Compiler daemon is spawn to compile the 
build-logic
Kotlin classes, as expected. But when Gradle moves to the actual project compilation, a new Kotlin Compiler daemon is spawn again, and the previous one is not reused, nor killed. This is unexpected to me. By looking at the JVM arguments and system properties of both daemons (with VisualVM), they are almost identical. The only difference is on 
java.class.path/sun.java.command
where on one hand it references Kotlin jars in version 
1.9.0
versus 
1.9.10
on the other. Is this the reason why the daemon is not reused? And if yes, how can this be changed? By looking at these docs:
Avoid specifying a version for the 
kotlin-dsl
plugin
And the corresponding changelogs seems empty. So I'm not sure what to do about this, and would love to get some help 🙂
v
If that is the case, then it is probably because you productively use Kotlin 1.9.10, while the Gradle built-in Kotlin version for the build logic is 1.9.0. You could downgrade your project Kotlin version to 1.9.0, then it might be reused. Like it is, you use two different Kotlin version, so you have two different compiler daemons.
s
Indeed, we rely on 1.9.10 in our project. Forcing the 
kotlin-dsl
to 4.1.2 made the daemon reused and does not seem to introduce any regression 👍
v
You should never do that though
s
😬
could this break the build-cache or the configuration-cache?
v
This can break many things. Gradlew controls the Kotlin version for build logic. It also forces the shipped Kotlin libs. So you end up with a mix of 1.9.0 and 1.9.10 for the build logic and can get quite some subtle or not so subtle problems.
s
Understood. If I downgrade the build logic (included build and main projects) to use Kotlin 1.9.0 would I still be able to use Kotlin 1.9.10 for the app code, or do they all need to be aligned?
v
You did have 1.9.0 in the build logic before. And yes, you can just fine use 1.9.10 for the production code. You just do not get compile daemon reuse.
Build logic and production code are two separate things, they don't need to be aligned.
s
I think I am missing one piece of the puzzle here 😅. To me, there are at least 3 places where I can configure Kotlin versions: • the included build (aka my 
build-logic
) which is requested by 
kotlin-dsl
plugin (1.9.0 in this case) • the main build (aka, all by 
gradle.kts
files of all my other modules) which comes from the root 
build.gradle.kts
(1.9.10 in this case)
Copy code
plugins {
  alias(libs.plugins.kotlin.jvm) apply false
}
• the app source code (aka 
kotlin-stdlib*
) which comes from the applied plugin Or are these last two items always identical?
v
Sure, what you apply there is the plugin that cares about building your production code
s
Ok, then I'm still not sure how I can downgrade the build version but at the same time keeping a more recent version in source code 🤔
v
You don't need to downgrade anything. 
build-logic
uses 1.9.0 as defined through 
kotlin-dsl
. Executing the build scripts of the main build uses 1.9.0 as defined through Gradle. The 1.9.10 plugin you apply makes your production classes compiled with 1.9.10.
s
Executing the build scripts of the main build uses 1.9.0 as defined through Gradle.
I think this is the part that I don't get 😅
v
The Kotlin plugin version you apply has nothing to do with the runtime environment the buidscript is running in. The buildscripts are running with the bundled Kotlin version. The Kotlin plugin runs on that and sets up tasks that compile and run your production code with 1.9.10.
You can compare this to running Gradle with a Java 8 JDK, but you JVM toolchains to compile your production code with Java 21.
While Gradle and the build logic is running with Java 8, it sets up tasks that compiles, tests, runs, ... your actual production files using a Java 21 JDK
s
Ok, but where does the embedded Kotlin version running the main build scripts comes from? Gradle? Because according to this table, Gradle 8.3 uses Kotlin 1.9.0. But we I run my build, the Kotlin Compile Daemon mentions Kotlin 1.9.10 jars.
v
You said you have two Kotlin compile daemons, one for the build logic using 1.9.0, one for the production code using 1.9.10, which is exactly what I described. Yes, it is comding from Gradle, the Kotlin runtime for the build logic is built-in in the Gradle version you are using.
s
I think I get it, but my main goal was to avoid having 2 daemons in the first place (for memory constraints reasons).
h
As a workaround you can use Gradle 8.4 rc 1/2, which uses Kotlin 1.9.10 🤷🏻‍♂️
v
my main goal was to avoid having 2 daemons
Well, for that you have to align the versions and thus downgrade your production Kotlin version to 1.9.0, or use Gradle 8.4, yes. But imho you really shouldn't couple these versions.
s
Then there might be alternative workarounds that I am not aware of, because we keep raising the amount of ram needed, but it's hard to keep up with that pace (we are currently maxing out at 32g...). One could be to extract this 
build-logic
into a dedicated project, and publish it somewhere to not have to compile it from the main project? Another one that I'm thinking about is, would it be possible to override some kind of Kotlin daemon timeout (time after which it is killed if unused)? Because at the moment, it is kept alive for at least the whole duration of the build, and hangs onto ~1 to 2g of RAM. 
--no-daemon
does not seem to affect it, and 
gradle.properties
inside the 
build-logic
are ignored 🤷
v
Yes for 1. No idea for 2. 
--no-daemon
is just about the Gradle build daemon, could well be that the Kotlin compile deamon survives that.
s
There seems to be a 
--daemon-autoshutdownIdleSeconds=7200
given as argument to the Kotlin Compile Daemon. But this is probably not configurable (at the moment) https://youtrack.jetbrains.com/issue/KT-50510/Gradle-Add-the-possibility-to-change-daemon-autoshutdownIdleSeconds Anyway, thanks a lot for all these informations 🙏
👌 1
a
You could choose the in-process compiler execution strategy for the build-logic module, then it won’t require the daemon at all. That strategy doesn’t yet support incremental compilation, however if you don’t change the module too frequently, you’ll be well. https://kotlinlang.org/docs/gradle-compilation-and-caches.html#defining-kotlin-compiler-execution-strategy
v
But iirc, this is discouraged 🙂
a
Right, the current implementation of the in-process strategy is suboptimal, however we have plans to improve it
👌 1
s
What do you mean by suboptimal? This is meant for CI only, so I suppose incremental compilation might no be that useful. But if it results in more flakiness or even more RAM consumption at the benefit of only not spawning an extra VM, that might not be worth.
a
Currently it’s creating a new class loader for each compilation task, that may cause higher memory consumption and much slower compilation because JVM shall load classes and perform JIT compilation for the compiler classes almost from scratch. So I won’t recommend using it for the main build as I consider it consists of at least a few modules. However, it’s ok to use it for 
buildSrc
or the single 
build-logic
module. It shouldn’t cause any problems in this case if you’re ok with not having incremental compilation for that module.
s
Indeed we have ~1k modules on the main project 😅
Good to know anyway 👍
The embedded Kotlin has been upgraded to 1.9.10
https://docs.gradle.org/8.4/release-notes.html 👌
49 Views
Open in Slack
PreviousNext
Powered by

kotlinlang

A modern programming language that makes developers happier.

Powered by