Anyone have any tips on how to speed up github actions when building kotlin native binaries? I am burning a lot of minutes on my macos and windows builds (particularly macos b/c of their minutes multiplier). Seems like a good chunk of the time goes into downloading konan dependencies

Downloading and extracting Konan takes only few seconds (add a BuildScan to check it). Building the native library does take ages. On macOS, you can't build native targets in parallel and there is no real workaround yet: https://youtrack.jetbrains.com/issue/KT-49385/Kotlin-Native-Parallelize-link-tasks-for-different-targets

anecdotally, there's no way this is true, at least on a stock github action runner. I'm watching some builds right now (macos/windows specifically) where download and extract of konan deps pushes way past the 4 minute mark

The question is, is caching it faster? You still need to download and extract the cache. But yeah, it took more time than a few seconds, sorry.

Please wait while Kotlin/Native compiler 1.8.21 is being installed.
Download <https://download.jetbrains.com/kotlin/native/builds/releases/1.8.21/macos-x86_64/kotlin-native-prebuilt-macos-x86_64-1.8.21.tar.gz> (308.22 MB)
Download kotlin-native-prebuilt-macos-x86_64-1.8.21.tar.gz finished, took 24 ms
Unpack Kotlin/Native compiler to /Users/runner/.konan/kotlin-native-prebuilt-macos-x86_64-1.8.21
Unpack Kotlin/Native compiler to /Users/runner/.konan/kotlin-native-prebuilt-macos-x86_64-1.8.21 finished, took 24 s 260 ms

https://github.com/cashapp/sqldelight/actions/runs/5041216230/jobs/9040718326#step:7:30

that is a question for sure 🙂

BTW: This is your run, 4 + 21 seconds:

Please wait while Kotlin/Native compiler 1.8.21 is being installed.
Download <https://download.jetbrains.com/kotlin/native/builds/releases/1.8.21/macos-x86_64/kotlin-native-prebuilt-macos-x86_64-1.8.21.tar.gz> (308.22 MB)
Download kotlin-native-prebuilt-macos-x86_64-1.8.21.tar.gz finished, took 4 s 237 ms
Unpack Kotlin/Native compiler to /Users/runner/.konan/kotlin-native-prebuilt-macos-x86_64-1.8.21
Unpack Kotlin/Native compiler to /Users/runner/.konan/kotlin-native-prebuilt-macos-x86_64-1.8.21 finished, took 21 s 851 ms

Yes, I am calling shenanigans on that. Maybe there is additional steps that gradle is not taking into account in that number, that would require a build scan to introspect, but something is taking 4+ minutes, prior to the binary creation step

are you using the

gradle/gradle-build-action

? Do you have Gradle build cache enabled?

i am using this one https://github.com/burrunan/gradle-cache-action

you can also add a step to cache the Kotlin Konan dir. Here’s what I use:

- name: Cache Kotlin Konan
        id: cache-kotlin-konan
        uses: actions/cache@v3
        with:
          path: |
            ~/.konan/**/*
          key: kotlin-konan-${{ runner.os }}

ah yes will likely need to add that directory, good call!

I really recommend you to use a build scan, this makes analyzing way easier.

is there a way to get build scans to work in CI? every time I use a build scan gradle always sends me thru this super annoying loop of command line opt ins and email hyperlinks

Of course: https://github.com/cashapp/sqldelight/blob/9ea9c095f66922da874ae7e57dc9ea30f667e49f/settings.gradle#L11

there’s no way to workaround the need to click the link and give it your email, unless you pay for Gradle Enterprise

Just remove the jdk resolver plugin

ah yes I should specify, I'm a lowly solo dev in this endeavor... gradle enterprise is not going to be an option for me 🥲

me too :) But you can still enable the scans and then you can click the link and generate the scan result if you need to, for example if you’re asking for help on improving build speed :blob-wink:

There is also a free option, but they are publicly accessible, which sounds fine with open source projects and using public GitHub actions.

only been one run, but fixing the cache key issue dropped my macos builds from 7 minutes to 3 and my windows build from 6 to 3

add

org.gradle.caching=true

to

gradle.properties

- it gives a really big improvement

no i have that already, what i'm saying is that in github actions, the os matrix was causing my cache key to collide b/c i was not properly accounting for the os names

ah okay, I couldn’t see it https://github.com/unredundant/neonctl/blob/chore--fiddling-with-builds/gradle.properties

i usually start from a template

Gradle does lots of caching, and what Build Cache solves is saving & sharing that cache so that even on a fresh checkout, the cache is still available in the

$HOME/.gradle

dir

thanks for the help! @Adam S & @hfhbd

and in theory you could share the cache via a HTTP server or a AWS S3 bucket or something, so if I checked out your repo and ran

./gradlew check

it would fetch the result from the build cache, see that the inputs haven’t changed, and then conclude that the tests are correct without actually running them.

my employer made some pretty crazy CI speed gains by doing something similar with their monolith (though in python land, not w/ gradle)

that’s cool