https://kotlinlang.org logo
#ksp
Title
# ksp
v

Vladimir Tagakov

02/22/2023, 2:44 AM
Hi! We moved from javac based annotation processors to everything written with KSP and we are seeing ridiculously big difference in memory footprint. At the first glance it seems like a memory leak in either KSP or kotlinc. Our processor doesn’t have any fields, top-level fields, objects, companion objects or anything static involved. Everything we reference has a scope of
process
function and should be freed after exiting. So it is highly unlikely that there is a memory leak on our side 😞 We are comparing Kotlin + javac annotation processing with Kotlin + KSP annotation processing and the difference in peak memory consumption for the whole build is: • For Kotlin+Javac: 39.5 Gb • For Kotlin+KSP: 62.5 Gb Both setups are using
Bazel
workers, to verify that it is a memory leak we changed strategy to
local
(which is basically starts new instance of kotlinc for every module instead of using long running daemons) and the memory usage felt back to normal values (still slightly more with KSP but tolerable) Can someone help me with triaging this issue and give me a clue how to run some diagnostics so I’ll be able to create a proper bug report? Was fixed by: https://github.com/google/ksp/pull/1067
y

yigit

02/22/2023, 2:46 AM
I'm not familiar with bazel but if you can share a memory dump, that could help us interstate. It is very hard to know whether it is KSP or kotlinc
v

Vladimir Tagakov

02/22/2023, 2:47 AM
I’m thinking on how to make it. I can run KSP locally for a single module and dump its heap, is it sufficient?
y

yigit

02/22/2023, 2:48 AM
You probably want to run it multiple times and share the heap dump from the first and last
So we can see what gets preserved between compilations etc
v

Vladimir Tagakov

02/22/2023, 2:49 AM
Oh, gotcha. So you want me to dump the heap of the kotlinc worker after a several compilations. If yes, I’ll try to do it tomorrow.
y

yigit

02/22/2023, 4:08 AM
Yep. Btw i don't work on KSP (at least not anymore) but that's what I think i would need to figure out what is leaking. Take a look at both heap dumps to see which objects kept growing.
v

Vladimir Tagakov

02/22/2023, 4:22 AM
Ugh… I’m sorry. I found this: https://github.com/google/ksp/pull/1067 And it seems like we are still on the version before the fix because we unable to update to the latest kotlin yet. So it is highly likely that our issue will be resolved after update. In the meanwhile I’ll try to verify that it helps by backporting the fix to our version. I am still not 100% sure that it will help because I could not consider our processor to be “leaky” but figuring it out is relatively harder than just backporting the fix and checking. Thank you, Yigit for the support 🙂 brb with the findings
Confirmed! Backport helped with the memory issues, thank you and sorry for the false alarm!
j

Jiaxiang

02/22/2023, 7:55 AM
Good to know that the fix worked for you! On the other hand, source dependencies are necessary for establishing correct incremental relationships, since your use case does not involve incremental it should be ok, but could be a note that I should take on our side.
15 Views