Not sure if this belongs here, or <#C19FD9681|grad...

# kotlin-native

[ankushg]

Not sure if this belongs here, or #gradle, but I'm trying to understand why our build times for release artifacts are so painfully slow.

We have a multi-module, multi-platform build, and it looks like linking iOS Frameworks is taking 2/3 of our total build time.

After running a build scan, it looks like our linkReleaseFramework<Target> tasks (KotlinNativeLink) are starting late in the build and executing serially, even though there are plenty of idle workers available.

Things got much worse once we added M1 support since there was now yet another framework to link 😕

The KotlinNativeLink tasks don't seem to depend on each other in any way.

Are these tasks supposed to be able to run in parallel? If so, how can I find out why they aren't for our builds? If not, is there a YT ticket for this?

Screenshot/description of build scan timeline in 🧵

[ankushg]

Screenshot of our build timeline in case it's helpful

It looks like:
• iosArm64 doesn't start linking until after JS and JVM have already finished their entire compilations
• iosSimulatorArm64 only starts linking once iosArm64 has completed (even though 3/4 of the workers are sitting idle)
• iosX64 doesn't start linking until iosSimulatorArm64 has completed either -- still with three idle workers

[ankushg]

Ah I think I see what I'm missing: From Gradle's parallel execution docs (emphasis mine)

you can force Gradle to execute tasks in parallel as long as those tasks are in different projects.

so there doesn't seem to be a way to parallelize those tasks since they're all part of the same project 😕

[rnett]

There's work on that with the configuration cache, see https://docs.gradle.org/current/userguide/configuration_cache.html#config_cache:intro:performance_improvements

[rnett]

Not sure how much Kotlin supports it yet though

[ankushg]

Yeah, there's no Kotlin listed in the official support list for configuration cache stuff, and it doesn't seem to impact clean CI builds either

There also used to be a kotlin.parallel.tasks.*in.project* flag, but it was deprecated and we're supposed to use the normal gradle --parallel flag (which is what the docs say requires the tasks to be in different projects)

[rnett]

You'd probably have more luck in #gradle or the gradle slack, I'm surprised this is the first I'm hearing about it tbh

[tapchicoma]

Kotlin Gradle plugins should fully support configuration cache. Note - the feature itself is still experimental. If you will find some plugins are not working - please open a new issue.

[tapchicoma]

But KGP/Native part implementation is kind of separated for other parts and does not rely/use kotlin.parallel.tasks.in.project flag value and does not utilize Gradle workers for link* tasks.

Please open a new issue to improve this part.

[ankushg]

@tapchicoma created https://youtrack.jetbrains.com/issue/KT-49385 -- not sure if I covered everything in there

[Paul Woitaschek]

Well if you remove them, multiplatform is.

[Paul Woitaschek]

I have a gradle property to control which native targets to include to speed up the builds and that brings the side effect of making it config cache compatible

[ankushg]

Yeah, the issue for us is that our release XCFramework compilation time is specifically what we're trying to improve.

Removing native tasks isn't really going to help with that because we need all of them 🙃

[Paul Woitaschek]

For us the config phase is only a fraction of the compile time

[Paul Woitaschek]

Most of the time is spent on linking

[Paul Woitaschek]

40 minutes linking, 40 seconds config

[ankushg]

Yeah same, but apparently with a properly configuration task, linking can also be parallelized across targets

https://docs.gradle.org/current/userguide/configuration_cache.html#config_cache:intro:performance_improvements

All tasks run in parallel by default.

[Paul Woitaschek]

There is an issue for that I think

[Paul Woitaschek]

In Kotlin

[ankushg]

Yup, that's what I linked to above

[Paul Woitaschek]

Try disabling hmpp, that improved our build time iirc

[mbonnin]

Damn, I was hoping to enable hmpp soon 😅

[Paul Woitaschek]

Only disable it for the native build

[Paul Woitaschek]

We build only on ci and publish from there

[Paul Woitaschek]

Development happens most of the time with all native targets excluded, config cache and hmpp disabled

[Paul Woitaschek]

Gives you a few unresolved references but we mostly work in common main anyways so everything being magnitudes faster outweights that

[Paul Woitaschek]

Iirc I had a conversation here a few days ago where a kotlin dev had a simple idea to make the hmpp indexing way faster. We have many source set layers in between (commonJavaMain, appleMain, iosMain, watchosMain) and only with the optimizations described above, smooth development is possible

[Paul Woitaschek]

https://kotlinlang.slack.com/archives/C3PQML5NU/p1634314283057400?thread_ts=1634136542.497300&amp;cid=C3PQML5NU