The memory usage of the kotlin compiler daemon for...

# kapt

[Sergei Dryganets]

The memory usage of the kotlin compiler daemon for our project reached 10Gb threshold.
If we set it lower kotlin daemon dies out with OOM. 

It dies during the kapt processing. That’s why I’m trying to reach out to this channel.

Wonder if anyone else experienced similar issues.

The heap dump looks pretty similar to the one in this issue (https://youtrack.jetbrains.com/issue/KT-32962). But there is no leak in our case. There is only one instance of IncrementalProcessingEnvironment. Also with 10gb heap limit the memory goes down to normal level after annotation processing finished.

We had a suspicion that it had something to do with the generation of the @Metadata annotations on the stubs (A big chunk of heap is consumed by the flatbuffer strings). We did a quick hack to strip those annotations out of stubs generated by kapt but it doesn’t seem to be helping much.

We have a BUCK build configured and it doesn’t have such problem (it doesn't use kotlin daemon though).

Right now we have to hypothesizes:

1. There is something wrong with our gradle script or kotlin gradle plugin. It would be awesome if someone could give a pointer how to enable the logs of the kotlin daemon launched from the gradle plugin, ie how to tweak the verbosity level of the daemon and where to find logs on Mac/Linux.
2. It might be the case that kapt extension keeps more resources than it is needed for the processing. Currently the code for the resources release in kapt looks like this (https://github.com/JetBrains/kotlin/blob/2552540f7165571dd2fdefdef4f09d858e4fa92a/plugins/kapt3/kapt3-compiler/src/org/jetbrains/kotlin/kapt3/Kapt3Extension.kt#L101-L104). We wonder if there is a way to say release some of the resources between different annotation processors runs? 
Most of our codebase is Kotlin, so literally every file in the project has the Metadata annotations and corresponding overhead. Is there a way to ignore those data completely during the kapt processing? What would be consequences of doing it?

Maybe there are some known kapt options that could potentially help mitigate the issue/reduce the memory footprint?

Our gradle.properties for kapt now:

# See: <https://youtrack.jetbrains.com/oauth?state=%2Fissue%2FKT-17621>
kotlin.incremental.usePreciseJavaTracking=true
# Configure the Kotlin Gradle plugin to run tasks in parallel within a project.
# <https://blog.jetbrains.com/kotlin/2019/01/kotlin-1-3-20-released/>
kotlin.parallel.tasks.in.project=true
# There is a cache key inconsistency when enabling this
# Disable incremental kapt locally to avoid this
# <https://youtrack.jetbrains.com/issue/KT-31840>
kapt.incremental.apt=false
# Skips annotation processing completely when there are no changes in kapt stubs and only method
# bodies are changed in dependencies. See <https://blog.jetbrains.com/kotlin/2019/01/kotlin-1-3-20-released/>
kapt.include.compile.classpath=false

[gildor]

Have you tried to disable incremental AP?

[gildor]

Do you use workers for kapt?

[Sergei Dryganets]

The incremental AP is disabled for us.
No we don't use workers.
I tried it today and this moves the memory burden to the gradle daemon. Now it slowly dies out of memory.
On the bright side - Kotlin daemon memory is in the 2Gb range.

Though it is kotlin 1.3.60.
I'll try to give it more memory to check how much is needed for this setup.

We tried this option before and had some reliability issues in the CI, I don't think we had tried it too hard though.

Thanks for this insight Andrey!

[Sergei Dryganets]

So far it looks pretty neutral.
The Kotlin daemon used around 2Gb memory less.
Gradle uses about 1.5Gb extra .

Thats for 10.3.60 that we are using now.