https://kotlinlang.org logo
Join Slack
Powered by
Hi folks, I’ve got a question about the KMP Gradle...
# gradle
k
Hi folks, I’ve got a question about the KMP Gradle plugin. On my team we’re running into memory issues, particularly when doing IDE sync. Our Gradle daemon routinely runs out of memory with a max heap size of 8g. I’ve done some JFR profiling of the daemon while syncing to see where hotspots might be and it looks like some calls to resolve dependencies for Native libraries in the KMP plugin are allocating a lot of memory. Should I be expecting this many allocations for native dependency resolution? Our project is ~660 modules, most of which are KMP modules. We don’t see similar allocation totals for the JVM dependency resolvers. It looks like these allocations bottom out in reading properties files from klibs. What’s a reasonable amount of memory consumption to expect from a 660 module project for KMP?
I’d be happy to post the JFR here if it helps anyone from Jetbrains optimize memory allocations. These memory issues have started to become a really big impediment to our work. Most people can’t sync the IDE at all without getting OOM, even when bumping xmx to 12g.
I've done some more digging, and a lot of the memory consumption for several different call frames seems to come from reading from the properties files here. Are there any optimizations we can make here, like perhaps doing some memorization of these properties files by project?
t
Could you open a new Kotlin issue with all your findings?
k
I'm about to head on vacation unfortunately, but I'll try to get someone else to do that. Thanks!
Is it okay to open an issue that doesn't have any suggested remediation? We're still digging in, and based on our findings I'm not seeing any clear places memorization would help.
t
Is it okay to open an issue that doesn't have any suggested remediation?
Yes, we will check from our side what could be done here
👍 1
r
So we kept going down this path and found that for each module 
IdeMultiplatformImportFactory
was adding ~17 resolvers, running for each of the 10 source sets that are applied by the default target hierarchy (
commonMain/Test
, 
appleMain/Test
, 
iOS***Main/Test
, 
nativeMain/Test
) Heap use dropped from 8-10gig to ~5.5gig by using 
kotlin.mpp.applyDefaultHierarchyTemplate=false
with the experimental 
applyHierarchyTemplate {}
applying only iOS and 
jvm()
targets to most modules, and 
androidTarget()
to UI modules. This came with a bunch of config + build time benefits.
(I should note that almost all source is in 
commonMain
, a couple of projects/modules use 
jvmMain
and a handful use 
androidMain
and 
iosMain
- no other source sets are used).
t
fyi @Anton Lakotka [JB]
k
Thanks for following through on this Rich 😄
t
still could you create a Kotlin issue?:)
👍 1
➕ 1
k
Rich, it might also be helpful to run a JFR profile with only the JVM target enabled with the default hierarchy as a baseline before you file that issue
r
@tapchicoma Indeed, I'm a little snowed under after last week but I've made a note to get a reproducer up.
Rich, it might also be helpful to run a JFR profile with only the JVM target enabled with the default hierarchy as a baseline before you file that issue
Yeah, IIRC I did this and you got saw the expected drop, heap use was peaking at like 3.5/4g.
a
Can you also roughly draw size of your projects? i.e. how many subproject do you have, how many source sets per subproject and how many dependencies per source set. I missed this > Our project is ~660 modules, most of which are KMP modules. That's enough. We're going to add this into our performance tests to measure memory consumption as well. From what I see in screenshots there is lack of cache for Commonized Native Distribution. i.e. for each sourceSet we independently re-run the same IdeCommonizedNativePlatformDependencyResolver over and over. In fact it should be enough to read those dependencies once per Import session.
r
Would you still like an issue raised from us @Anton Lakotka [JB] or is this OK? This is the change we made in terms of showing the impact (note we only put code in 
iosMain()
for any native targets)..
k
Rich, I’d like to still file an issue that includes a JFR profile of our project and helps illustrate the problem.
r
We've gone ahead and raised one here. Thanks again.
thank you color 1
k
Following up on this issue. We’ve worked around this by specifying a custom KMP target hierarchy, but that’s only bought us some time. We’re wondering when we should expect this issue to be prioritized? Some of our developers are running into Gradle daemon OOM during IDE sync again.
t
we will try to re-prioritize it
K 2
thank you color 2
r
Thank you so much Yahor 🙏
k
Thank you so much 🎉
Critical
severity! Thank you so much 🤯
6 Views
Open in Slack
PreviousNext
Powered by