https://kotlinlang.org logo
Join SlackCommunities
Powered by
Dokka 2 is really hungry for resources :wink: Had ...
# dokka
r
Dokka 2 is really hungry for resources 😉 Had to set max heap size to 12G and close all other applications to generate documentation for my project with about 40 modules (It took 6 minutes with load average ~40 and made my 32GB thinkpad unresponsive).
It generated 676MB of html code.
🫣 1
o
In theory, Dokka 2.0.0 generator should use a bit less memory (at rendering stage) 🙂 But! Because DGPv2 now uses Gradle Workers, Dokka execution is highly parallel and spawns multiple processes... Which heap you were increasing? Dokka execution or Gradle's? There is also a Troubleshooting section which could help. In addition to that, you can try to run DGP tasks with 
org.gradle.workers.max=2
 to limit parallelism, but it will affect not only Dokka, so it's more of a last resort, if needed BTW, is your project opensource?
r
Yes. https://github.com/rjaros/kilua But I haven't pushed upgrade to Dokka 2 yet.
thank you color 1
Which heap you were increasing? Dokka execution or Gradle's?
I have added:
Copy code
dokkaGeneratorIsolation.set(ProcessIsolation {
    maxHeapSize.set("12g")
})
to all modules, and yes, it seems all modules are generated in parallel.
o
Does it fail with e.g 4g? if so, what is the exception?
Also, for projects with a lot of modules, 
ClassloaderIsolation
might be a better choice regarding memory consumption - because it will not spawn separate processes, and so will use just Gradle heap
r
I'm doing some tests. I've changed to 
ClassloadedIsolation
and with Gradle heap set to 
6GB
it fails after a few minutes with:
Copy code
> Task :modules:kilua-trix:dokkaGenerateModuleHtml FAILED
The Daemon will expire after the build after running out of JVM heap space.
The project memory settings are likely not configured or are configured to an insufficient value.
The daemon will restart for the next build, which may increase subsequent build times.
These settings can be adjusted by setting 'org.gradle.jvmargs' in 'gradle.properties'.
The currently configured max heap space is '6 GiB' and the configured max metaspace is 'unknown'.
but there are still other tasks running, CPU at 100%, load avg ~10.
I have to kill this run, because it will probably not finish anytime soon.
Will try increasing heap size by 2GB until it works.
java.lang.OutOfMemoryError: Java heap space
on 8GB
Lost patience after 17 minutes run with 10GB. No exceptions but no results either.
Same with 12GB. I'm afraid there is some kind of deadlock there, because I can't even kill the java daemon process without 
-9
signal.
I will try with 16GB once more
Still doesn't work
😞 1
o
I will take a look on your project a bit later Could you publish a branch where you were able to generate documentaion?
r
I'll let you know
thank you color 1
I've reverted to 
ProcessIsolation
and it works with 8g.
I've pushed my changes. Current Dokka configuration is in the `buildSrc`: https://github.com/rjaros/kilua/blob/main/buildSrc/src/main/kotlin/Shared.kt#L207
o
Okay, so I took a look on your project. And yes, Dokka needs rather big amount of memory to process one of the modules: 
kilua
module - it's just big Previously Dokka were executed inside Gradle daemon and wasn't able to run in parallel in most cases - and so 6GB (in 
gradle.properties
) were enough to generate documentation for every module separately That's why now with 
ClassloadedIsolation
you are getting OOM with the same or even bigger memory there - because Dokka run in parallel So yes, for your use-case it's better to use 
ProcessIsolation
with 8GB because of this big module. You could also set those 8GB only for that 
kilua
module - may be it will use less memory overall (by default we set 2GB for worker) Also, just keeping you informed: thanks to your project, I found some places which could optimize Dokka memory consumption 🙂 In any case, thanks for trying DGPv2!
r
Thank you for taking care of my case! 🙂
14 Views
Open in Slack
PreviousNext
Powered by