I started seeing this message after updating Gradl...
# gradle
e
I started seeing this message after updating Gradle to 8.0-rc-1 and Kotlin to 1.8 (and setting
kotlin.daemon.useFallbackStrategy=false
). Is this talking about the Kotlin compiler daemon or the Gradle daemon? Bonus points if anyone knows what the issue could be 😅
Copy code
Unable to clear jar cache after compilation, maybe daemon is already down: java.rmi.ConnectException: Connection refused to host: 127.0.0.1; nested exception is: 
	java.net.ConnectException: Connection refused
There are no errors in the build. Tasks are executing, and suddenly that message appears and the build fails.
v
It's the Kotlin compile daemon that is meant.
Maybe look at the Kotlin compile daemon logs if there are any to see whether there is something suspicious, or maybe the system logs to see whether it got killed for example by OOM killer.
e
I don't think I can access those logs because it is on GitHub Actions
But I'm assuming it's the OOM killer and the fallback strategy has been masking that for a while
v
Add the https://github.com/marketplace/actions/debugging-with-ssh action to be able to SSH into the machine, and add a step that runs
if: always()
after the failing step and keeps the workflow running in case the debugging-with-ssh action does not do that already anyway.
e
Oh cool, I'll check that out thanks
t
yeah, looks like OOM in Kotlin daemon. Please check the Kotlin daemon logs inside
/tmp
directory on the runner
e
Going to do that shortly. I tested lowering the
Xmx
to
3G
from
6G
for both the gradle daemon and the compiler daemon and now I get this failure almost immediately:
Copy code
Failed to compile with Kotlin daemon: java.lang.RuntimeException: Could not connect to Kotlin compile daemon
	at org.jetbrains.kotlin.compilerRunner.GradleKotlinCompilerWork.compileWithDaemon(GradleKotlinCompilerWork.kt:192)
	at org.jetbrains.kotlin.compilerRunner.GradleKotlinCompilerWork.compileWithDaemonOrFallbackImpl(GradleKotlinCompilerWork.kt:150)
	at org.jetbrains.kotlin.compilerRunner.GradleKotlinCompilerWork.run(GradleKotlinCompilerWork.kt:123)
	at org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction.execute(GradleCompilerRunnerWithWorkers.kt:74)
	at org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:63)
	at org.gradle.workers.internal.NoIsolationWorkerFactory$1$1.create(NoIsolationWorkerFactory.java:66)
	at org.gradle.workers.internal.NoIsolationWorkerFactory$1$1.create(NoIsolationWorkerFactory.java:62)
	at org.gradle.internal.classloader.ClassLoaderUtils.executeInClassloader(ClassLoaderUtils.java:100)
	at org.gradle.workers.internal.NoIsolationWorkerFactory$1.lambda$execute$0(NoIsolationWorkerFactory.java:62)
	at org.gradle.workers.internal.AbstractWorker$1.call(AbstractWorker.java:44)
	at org.gradle.workers.internal.AbstractWorker$1.call(AbstractWorker.java:41)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
	at org.gradle.workers.internal.AbstractWorker.executeWrappedInBuildOperation(AbstractWorker.java:41)
	at org.gradle.workers.internal.NoIsolationWorkerFactory$1.execute(NoIsolationWorkerFactory.java:59)
	at org.gradle.workers.internal.DefaultWorkerExecutor.lambda$submitWork$0(DefaultWorkerExecutor.java:169)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.runExecution(DefaultConditionalExecutionQueue.java:187)
	at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.access$700(DefaultConditionalExecutionQueue.java:120)
	at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner$1.run(DefaultConditionalExecutionQueue.java:162)
	at org.gradle.internal.Factories$1.create(Factories.java:31)
	at org.gradle.internal.work.DefaultWorkerLeaseService.withLocks(DefaultWorkerLeaseService.java:249)
	at org.gradle.internal.work.DefaultWorkerLeaseService.runAsWorkerThread(DefaultWorkerLeaseService.java:109)
	at org.gradle.internal.work.DefaultWorkerLeaseService.runAsWorkerThread(DefaultWorkerLeaseService.java:114)
	at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.runBatch(DefaultConditionalExecutionQueue.java:157)
	at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.run(DefaultConditionalExecutionQueue.java:126)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Using fallback strategy: Compile without Kotlin daemon
Try ./gradlew --stop if this issue persists.
warning: ATTENTION!


This mode is not recommended for production use,
as no stability/compatibility guarantees are given on
compiler or generated code. Use it at your own risk!
@tapchicoma it looks like there is no kotlin daemon log. It actually looks like the daemon never started at all.
After much debugging, there are two issues: 1. On GitHub Actions Ubuntu 20.04 runner, an Xmx of 6g for both Gradle and Kotlin is too high, because the runner has 7g of RAM. I solved this by lowering the Xmx, and now it works fine with the fallback strategy disabled (although still takes roughly the same amount of time) 2. On GitHub Actions Ubuntu 22.04 runner (which
ubuntu-latest
now points to), no matter what I do with the Xmx it seems like the Kotlin daemon never runs after a lot of fat trimming I was able to get this to work seemingly consistently but at the cost of increased build time. When it does fail there are no logs in
/tmp
and the OOM reaper didn't log anything about killing any process. Might be related to https://github.com/actions/runner-images/issues/6709
t
It actually looks like the daemon never started at all.
That is really strange. Could you get logs with Gradle
--debug
and share them?
e
I'm a little burnt out of Kotlin/Gradle right now...it's been a stressful few days of updating to 1.8/8.0 😅 I'll get to this as soon as I can, but likely won't be until mid next week
t
we have several cases reported with OOM in Metaspace. Probably you are affected as well
e
I was looking at jconsole with some local builds and didn't notice any issues with metaspace. I'll try again with some different scenarios. The GitHub Actions thing might have been transient, because I can't get it to happen again.
Guess I wasn't paying attention. There's something going on with metaspace. Is there a youtrack/github issue I can follow?
Left it running for longer; definitely something going on with metaspace. Also looks like the Kotlin daemon (but not the Gradle daemon) has some kind of leak with "CodeHeap non profiled nmethods" but I don't know too much about that (
ReservedCodeCacheSize
is set to
240m
)
t
e
@tapchicoma would this cause the corresponding metaspace leak in the gradle daemon? I was running detekt, which I believe uses the Kotlin compiler in process.
t
yeah, most probably
2581 Views