I'm building my Kotlin app using Gradle in a multi...

# gradle

[Rob Elliot]

I'm building my Kotlin app using Gradle in a multi-stage Dockerfile.

Consequently I don't think there's any point in using the kotlin compiler daemon because it only does one compile - might as well do it in the gradle process.

Can I get any benefits from incremental compilation by caching directories between builds?

[Sam]

Yes. The Kotlin compiler's incremental compilation works with the Gradle build cache. Set up Gradle to keep the build cache in a local directory, and then persist that directory between builds.

[Sam]

By the way, you can cache downloaded dependencies too 💡

[Rob Elliot]

I'm mounting the whole of ~/.gradle (and $project_dir/.gradle) as a cache mount:

RUN --mount=type=cache,target=~/myproject/.gradle \
    --mount=type=cache,target=~/.gradle \
    ./gradlew assemble

which should mean both build cache and dependencies are maintained across builds.

[Rob Elliot]

Thanks for the confirmation. I'm slightly surprised compileKotlin is still taking 30 seconds or so for a single file change.

[Sam]

Make sure the build cache is actually enabled too, of course. The build outputs should give some insight into what's taking the time, what's been loaded from the cache, etc.

[tapchicoma]

JiC: there is a way to have experimental JVM incremental compilation support inside Gradle daemon process which could be better for docker builds due to less memory management and less processes. For this you need to add following Gradle properties:

kotlin.compiler.execution.strategy=in-process
kotlin.compiler.runViaBuildToolsApi=true

This should also support Gradle build cache feature.

[Rob Elliot]

Thanks!

[Rob Elliot]

With ./gradlew --info classes I'm seeing:

#25 5.282 Task ':compileKotlin' is not up-to-date because:
#25 5.282   Output property 'classpathSnapshotProperties.classpathSnapshotDir' file /home/worker/work/build/kotlin/compileKotlin/classpath-snapshot has been removed.
#25 5.282   Output property 'classpathSnapshotProperties.classpathSnapshotDir' file /home/worker/work/build/kotlin/compileKotlin/classpath-snapshot/shrunk-classpath-snapshot.bin has been removed.
#25 5.282   Output property 'destinationDirectory' file /home/worker/work/build/classes/kotlin/main has been removed.
#25 5.282   and more...
#25 5.282   and more...
#25 5.282   and more...
#25 5.282 The input changes require a full rebuild for incremental task ':compileKotlin'.

[tapchicoma]

yeah, Gradle expects that outputs of previous compilation stay intact

[tapchicoma]

build cache will help to not recompile modules where code has not changed

[Rob Elliot]

I suppose I could make it:

RUN --mount=type=cache,target=~/myproject/.gradle \
    --mount=type=cache,target=~/myproject/build \
    --mount=type=cache,target=~/.gradle \
    ./gradlew assemble

[Rob Elliot]

Looks like this may be the sweet spot:

ENV GRADLE_PROPS="\
-Pkotlin.compiler.execution.strategy=in-process \
-Pkotlin.compiler.runViaBuildToolsApi=true \
"

RUN --mount=type=cache,target=~/myproject/.gradle \
    --mount=type=cache,target=~/myproject/build/kotlin \
    --mount=type=cache,target=~/myproject/build/classes/kotlin \
    --mount=type=cache,target=~/.gradle/caches/build-cache-1 \
    --mount=type=cache,target=~/.gradle/caches/transforms-4 \
    ./gradlew "$GRADLE_PROPS" assemble

Honestly, if Gradle and JetBrains had actively set out to make it as hard as possible to get an efficient multi-stage docker build I'm not sure they could have made it any harder.

(I also love that next time they increment build-cache-1 to build-cache-2 or transforms-4 to transforms-5 my optimisation will silently stop working... why couldn't they have nested those under a directory?!)

[tapchicoma]

I highly double Gradle was designed in mind of Doker multi-stage builds and I would propose you to open an issue for them to write about best practices for such case

[Rob Elliot]

Yes, I know, shouldn't really moan, just seem to have spent an absurd amount of time wrangling with this

[tapchicoma]

imho still worth to open a documentation issue

[wasyl]

For kotlin, is it also worth reusing <projectDir>/.kotlin directory that was recently added?

[tapchicoma]

Only for KMP projects