JB people, can you please explain performance differences between running Gradle build with separate...

# build-tools

[artem_zin]

JB people, can you please explain performance differences between running Gradle build with separate Kotlin Compiler Daemon vs in-process compilation?

What is the lifecycle of in-process compilation? Isn’t Gradle daemon enough to keep compiler “hot” (class loading, JIT)?

Thanks!

---

Reason why I’m asking is because we’re facing issues with Kotlin Compiler Daemon with a single Kotlin module in project when we build it with --parallel, see <https://youtrack.jetbrains.com/issue/KT-15562#comment=27-2494744>

[gildor]

Just for test tried to build our project (23 modules) without kotlin daemon. Build time for clean builds roughly 2 times slower with disabled Kotlin Daemon.
Do you have the same degradation of build speed?

[artem_zin]

No, I can’t notice any difference because we only have one Kotlin module atm

[artem_zin]

@gildor but you didn’t kill Gradle Daemon right?

[gildor]

yes, Gradle Daemon works with parallel build and 8 threads

[artem_zin]

mhm, that’s a pretty significant change then, damn it

[artem_zin]

Do you use --parallel?

[artem_zin]

i mean in general 🙂

[gildor]

In our case ~50% of code is Kotlin and most of modules are pure Kotlin + kapt

[gildor]

yes, we have org.gradle.parallel=true

[artem_zin]

have you noticed build problems due to that? I saw you +1oned related issues on YT

[gildor]

We had such problem on CI, but it fixed after server reboot (uptime 420+ days before it)

[artem_zin]

I don’t think reboot matters here (unless you share Gradle/Kotlin daemons between builds)

[gildor]

We had some problems with memory, swap was full all the time and sometimes build fails with this error

[artem_zin]

it seems to be purely concurrency issue

[gildor]

Yes, maybe. Not sure what is the reason. But before and after we don’t have such problem with unstable build

[artem_zin]

ah, then you most likely have problem of Kotlin Compiler Daemon not starting due to unavailable ram or it dies with OOM

[gildor]

yes, exactly

[artem_zin]

that’s discussed in the issues, try to tune JVM memory options or just get a better CI nodes 😄

[gildor]

Already fixed, as I said

[gildor]

We used this node for about 2 years without issues and still use it 🙂

[gildor]

but we got this problem after move android build slave to a separate docker image (before build on master)

[artem_zin]

in our case it’s super ridiculous, we have ~200 modules and only one Kotlin module, 30 CPU cores and 32 GB ram

So it sounds more like concurrency issue because Gradle can call Kotlin Gradle Plugin concurrently due to --parallel

[gildor]

yeah, our node much smaller, only 8 Gb RAM and 2 core

[bringoff]

We had this issue with almost every build for the past several days. Parallel builds and daemon disabling didn’t help. But when I added -Pkotlin.incremental=false there aren’t any failed builds yet. But I keet daemon and parallel disabled as we don’t see much difference in build time (oldshool single module project).

[bringoff]

Nope, just failed. Will try to change execution strategy

[artem_zin]

cc @alexey.tsvetkov (yo 🙂 ) maybe you have some insights?

[alexey.tsvetkov]

Currently "in-process" compilation creates a new isolated classloader for each call to the compiler. That gives a significant performance penalty since compiler classes are loaded and JIT-compiled every time a "compile*Kotlin" task executes. Also it does not support incremental compilation.
We plan to investigate the classloader caching and incremental compilation support in future.

[artem_zin]

@alexey.tsvetkov that was exactly what I wanted to know, thank you!