https://kotlinlang.org logo
Join Slack
Powered by
Please try out the Gradle build cache: 1. Create ...
# kotest-contributors
a
Please try out the Gradle build cache: 1. Create an init script in your GRADLE_USER_HOME directory (or if you don't have GRADLE_USER_HOME env var, in 
~/.gradle/
) 
nano $GRADLE_USER_HOME/init.d/kotest-build-cache.init.gradle.kts
2. Paste this build cache config:
Copy code
gradle.settingsEvaluated {
   if (rootProject.name == "kotest") {
      buildCache {
         remote<HttpBuildCache> {
            if (url != null) {
               logger.warn("Kotest remote build cache is already configured. You might want to remove \$GRADLE_USER_HOME/init.d/kotest-build-cache.init.gradle.kts.")
            } else {
               logger.lifecycle("Configuring Kotest remote build cache!")
               url = uri("<http://5.161.201.26:5071/cache>")
               isAllowUntrustedServer = true
               isAllowInsecureProtocol = true
               isPush = false
            }
         }
      }
   }
}
Once we've configured a nice URL instead of a raw IP address, then https://github.com/kotest/kotest/pull/4141 can be merged and this file can be removed and build cache will Just Work. Until then, I'm curious whether you notice a difference in build times.
o
Will do.
🚀 1
Oh, I'm sorry, I'd love to but I can't do that on my machine. Using the above would mean a supply chain risk as I would run executable code from an untrusted, publically accessible source locally. Or am I missing something?
An idea: Is the build cache just a normal HTTP-write-accessible URL? Then I could maybe try this with a local server.
a
hmmm I really have to think about that. I guess that technically yes, you'd be downloading executable code. But practically, isn't that what happens already? If you checkout a PR and run the tests, then you're running executable code?
o
Yes. But I trust the repo plus the process plus the people. And it is source code I can review if in doubt.
a
just to clarify, pushing data to the build cache node is restricted
currently only I and GitHub CI have pushed data (and technically Emil could, because he has access to the server)
o
If it's just you, I'd consider it safe. What's the mechanism blocking pushes from unknown folks?
a
btw I recognise that it's an important question, so I do want to take it seriously and figure out how best to make it secure and trustable
The build cache node has an admin console, and I can create users with username & password credentials. So I created read/write access for GitHub CI, and put them into GitHub secrets
👍 1
Emil and I can connect to the server with SSH keys, and the build cache node is protected with a username/password
and the build cache node is running with rootless Docker
the build cache node uses the official image https://hub.docker.com/r/gradle/build-cache-node/
o
I'll do some research to familiarize myself with it. Actually, my impression regarding supply chain risks is this: So far, in the JVM/Maven universe we are somewhat lucky. We can usually trust our dependencies, and many people seem to go through changelogs before updating anything. In the JS/NPM world, people don't, they just update to whatever (transitive) dependency is available. Malicious artifacts happen, like this one, which I was lucky enough to miss, but it was close (2-hour time window IIRC). Then additional checks were introduced, but people still have to use them properly.
a
And yet Kotest has disabled yarn.lock blob thinking upside down https://github.com/kotest/kotest/blob/5234a310776eda23c1ee7b1d079d6105381a74c4/.gitignore#L48
o
True, and we should probably change that. And also look what we can do for the Kotest Gradle plugin test project, where we have no easy way of manually running 
gradlew kotlinUpgradeYarnLock
, and a changing yarn.lock file would block the build. Currently it will produce a warning. Locally, I manage yarn.lock and NPM dependencies for all projects with a JS part (Kotest included). That sometimes explains why my reviews get delayed, e.g. if new versions of Kotlin/JS arrive.
So I did some research: A CI-fed build cache, which is configured as you explained it, is totally OK.
Until then, I'm curious whether you notice a difference in build times.
That's a bit hard with local testing, right? When my local build cache is up to date, 
gradlew assemble
completes in 16 seconds. Gradle will use the local build cache first and delegate to the remote build cache for build outputs that don't have a local match. Normally, my local build cache will always be more recent than CI outputs. So I'd have to wait for a while without doing any Kotest development, then have some CI action update the remote build cache, then resume local development. In this case, I'll benefit from the remote build cache. I guess what would benefit most is if we allow PR actions to cache their outputs. Then subsequent PR runs could be way faster.
a
great, thanks for confirming 👍
if we allow PR actions to cache their outputs
That will be the case, when the PR is merged.
at the moment the impact will be limited
👍 1
e
Will the build cache work if we use different JVMs?
a
not always :( But it will still help. Gradle 8.8 has a beta feature that might help https://docs.gradle.org/8.8/release-notes.html#gradle-daemon-jvm-configurable-via-toolchain, so Gradle will use a stable JDK. The feature isn't ideal yet, because doesn't auto-provision the required JDK, so I'm on the fence about enabling it.
e
Okay. Do you know if there's any easy way of ensuring we have the same JDK as the GH pipelines locally?
We're also mixing temurin and adopt on GH. I think it would make sense to move all to temurin since adopt is discontinued?
a
moving to Temurin makes sense - although from a Gradle build POV I don't think the vendor makes a difference
I think so long as the action has a setup-java step, then Gradle will use that version, but I'm not 100% sure
the JDK that Gradle uses can be set explicitly. In my 
$GRADLE_USER_HOME/gradle.properties
I specify a fixed version:
Copy code
# Try to help with Build Cache re-use by always using the same JDK
org.gradle.java.home=/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/
e
What's left for the build cache PR? 🙂
Copy code
$ nslookup <http://gradle-build-cache.kotest.io|gradle-build-cache.kotest.io>                                                                                                                                                                                                                                                                      19s 13:38:35
Server:		2001:2042:fe60:5700::1
Address:	2001:2042:fe60:5700::1#53

Non-authoritative answer:
Name:	<http://gradle-build-cache.kotest.io|gradle-build-cache.kotest.io>
Address: 5.161.201.26
SSL cert?
a
yes
12 Views
Open in Slack
PreviousNext
Powered by