https://kotlinlang.org
Join Slack
:wave: Hey folks, I'm trying to understand how I can troubleshoot v slow `compileKotlin` task times ...
# gradle
a
šŸ‘‹ Hey folks, I'm trying to understand how I can troubleshoot v slow 
compileKotlin
task times -- it takes up to a minimum of 3 minutes each time a Kotlin file is changed in the main sourceset šŸ˜¢ ā€¢ Build caching is enabled -- with a Gradle build cache node in use (cache is fresh and definitely getting hits, used the cache debug flag to verify) ā€¢ Heap alloc is pretty high ā—¦ 
org.gradle.jvmargs=-Xmx8g -Dkotlin.daemon.jvm.options=-Xmx6g
ā€¢ It's a single module project @ around 80k LOC Please let me know what some options are to try and troubleshoot the task closer -- I tried using the profile & scan flags, but it didn't really break down what the task was spending its time doing
j
Have you tried to run check a build scan?
a
Yessir
Didn't really reveal why the compileKotlin task itself was taking time; it was useful to identify that 
compileKotlin
is the longest running task
v
You could have a deeper look using https://github.com/gradle/gradle-profiler probably
But maybe the Kotlin compilation simply needs that long
e
if it's a single module, it should be doable to build up a argument list to run the command line 
kotlinc
for comparison
but that sounds incredibly long for such a small project, are you doing anything like https://youtrack.jetbrains.com/issue/KT-41965 that could be causing non-linear behavior in the compiler?
a
Bjorn -- not sure if that profiler gives more insight into the compilation step, but will be useful to make changes and profile to see if it improves the situation Ephemient -- The 
kotlinc
idea is pretty interesting, didn't consider it -- can definitely give it a stab for reference. That was an interesting thread to read; I have no idea how that author narrowed it down to that method -- he doesn't seem to discuss it there I just checked, and we don't have any implementations of Comparable that we do ourselves
a
what happens if you comment out the ktlint build config? So there's no ktlint interfering at all And how much do you use Lombok?
t
if it's a single module, it should be doable to build up a argument list to run the command line kotlinc for comparison
It is possible to get them from debug logs
generally if you could provide a repro - I am interested to look into it
8Gb for Gradle and 6Gb for Kotlin is too high for 80 LOC project šŸ¤”
From build scan I see Gradle daemon consumes ~300Mb. Kotlin daemon memory usage should be printed in Kotlin daemon logs. Soon such metric should be available in build metrics produced by KGP
btw @Andrew Louis could you enable build reports and share them?
a
It is possible to get them from debug logs
Oh nice, will try it and report back later today Re - build reports - yes, will do! Let me enable it and share shortly Regarding a repro, not sure if this is possible at the moment, but will loop back. Can do an attempt at generate something minimal, but I'm not confident I'll be able to do an open repro (company codebase) šŸ¤” Will have to look at some options here and get back to you
t
please share at least a build report - it may lead investigation into proper direction
a
@tapchicoma built with the 
build_scan
report output: A couple of weird observations: ā€¢ There's something pretty funny there -- 
Connect to Kotlin daemon: 47365ms
?? ā€¢ Also, it's flagged as 
NON_INCREMENTAL
blob thinking upside down
t
NON_INCREMENTAL
could be because you run 
clean
before or changed build classpath
a
Gotcha, let me try again and see if that goes away
t
Could you try to run the build on non-GraalVM JDK?
a
šŸ‘ will do
Ah the last one was non Graal (see link) ; I believe I was trying out Graal temporarily to see if it improved the situation
t
yeah, accidentally looked into previous build scan
just in case - could you also try to run the build with JDK 17.0.6?
and are you using native Mac/ARM JDK?
a
Copy code
openjdk version "16.0.2" 2021-07-20
OpenJDK Runtime Environment Zulu16.32+15-CA (build 16.0.2+7)
OpenJDK 64-Bit Server VM Zulu16.32+15-CA (build 16.0.2+7, mixed mode)
I believe so
And yeah, let me just generate one last set of build scans, and then will try 17.0.6 šŸ‘
a
If you invoke 
java -XshowSettings:properties -version 2> >(grep "os.arch")
in terminal, will it print 
os.arch = x86_64
or 
os.arch = aarch64
?
a
aarch64 -- makes sense for an M1
a
Iā€™m asking because you could use both of them on M1 Mac, but the one built for 
x86_64
is expected to be slower
a
Ah makes sense -- seems like I'm using the arm one
v
Side question, what advantage do you see with using 
>(grep ...)
instead of 
| grep ...
?
a
Java outputs the properties into stderr
e
2>&1 | grep
then
v
Oh, that 
2> >(grep ...)
feeds fd 2 to the subshell? Interesting. I would have done 
2>&1 | grep ...
. TIL, thanks
e
it works but the shell job doesn't include the 
>()
process substitution so sometimes its output will be delayed until after the next terminal prompt has already come up :-/
v
ah, good to know, thx
a
@Yahor - with JDK 17, seems like a big boost - 1 minute 30 on the first run of 
compileKotlin
https://scans.gradle.com/s/uajg3nzz2paug
Also I missed a dangerous typo up above; I meant 
80k LOC
not 
80 LOC
šŸ˜‚ in case that threw anyone
v
It for sure did
e
ohhhh that makes a big difference :D
v
Not me, as I didn't even read it, but yeah, others referred to 80 LOC šŸ˜„
a
Hahah šŸ’€ apologies for the confusion folks, updated counts after an espresso: main source set: 47k LOC codegen generated kotlin files: 90k LOC
Is the 2-3 minutes within reason in that light?
t
build with JDK 17 looks much better.
Is the 2-3 minutes within reason in that light?
For clean build I would say yes
r
Tried these? ā€¢ Use the k2 compiler We have been running k2 for a kotlin/jvm project for a long time locally and then we disable it in CI. A nice tradeoff if you do not want to use it for release builds
Copy code
val isCiServer = System.getenv().containsKey("CI")
Copy code
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile>().configureEach {
    kotlinOptions.useK2 = !isCiServer
}
or in 1.8.20+
Copy code
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile>().configureEach {
    kotlinOptions {
       if(!isCiServer) {
           languageVersion = "2.0"
       }
    }
}
ā€¢ Use the incremental compilation support, it will be default in 1.8.20 https://blog.jetbrains.com/kotlin/2022/07/a-new-approach-to-incremental-compilation-in-kotlin/ ā€¢ Use the multithreaded compilation mode -Xbackend-threads=0
s
Andrew Louis [8:16 PM]
@Yahor - with JDK 17, seems like a big boost - 1 minute 30 on the first run of 
compileKotlin
Please use JDK built for aarch64, not x86 You are running it through Rosetta, which affects performance noticeably
a
Hmm, how could you tell I was running it through Rosetta?
s
Probably I'm wrong, JDK version in the build scan doesn't report JDK arch, but rather 64-Bit Server regardless of the architecture šŸ˜ž
t
@Andrew Louis if possible could you also produce build scan with build report while using Kotlin 1.8.10 in the repo?
a
Confirmed that I'm using the aarch64 JVM, the 
Kind
column in Activity Monitor shows Apple for my java processes
Yessir will do Yahor
Hmm not much change on the initial/clean build: https://scans.gradle.com/s/36gghtzscpk52/custom-values On the incremental, seems much snappier than the 1.7.x https://scans.gradle.com/s/36gghtzscpk52/custom-values
t
@Andrew Louis yeah, clean build looks slightly worth. Could you also try to run clean build with Kotlin 1.8.20-Beta and then additionally enable K2:
Copy code
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompilationTask<*>>().configureEach {
    compilerOptions.languageVersion.set(org.jetbrains.kotlin.gradle.dsl.KotlinVersion.KOTLIN_2_0)
}
a
Got 1 min 30 on 1.8.20-Beta without K2 https://scans.gradle.com/s/aolrqhqgnsydm -- which was comparable to our fastest clean build so far However, I couldn't get a build to succeed with K2 enabled sadly; I have some jooq generated code that's resulting in some compilation failures; For some reason a java class does not fit as the argument of type 
java.lang.Class<T>
:
Copy code
OrgSettings.kt:82:226 Argument type mismatch: actual type is java/lang/Class<java/lang/Module> but java/lang/Class<ft<U & Any, U?>>! was expected
t
@Andrew Louis could you open a Kotlin issue for this K2 problem?
a
Yessir will do!
35 Views
Open in Slack
PreviousNext
Powered by

kotlinlang

A modern programming language that makes developers happier.

Powered by