I started seeing this message after updating Gradle to 8 0 r

Question

I started seeing this message after updating Gradle to 8 0 rc 1 and Kotlin to 1 8 and setting `kotlin daemon useFallbackStrategy=false` Is this talking about the Kotlin compiler daemon or the Gradle daemon Bonus points if anyone knows what the issue could be sweat smile ```Unable to clear jar cache after compilation maybe daemon is already down java rmi ConnectException Connection refused to host 127 0 0 1 nested exception is java net ConnectException Connection refused```

eygraber · Answer

There are no errors in the build Tasks are executing and suddenly that message appears and the build fails

Vampire · Answer

It s the Kotlin compile daemon that is meant

Vampire · Answer

Maybe look at the Kotlin compile daemon logs if there are any to see whether there is something suspicious or maybe the system logs to see whether it got killed for example by OOM killer

eygraber · Answer

I don t think I can access those logs because it is on GitHub Actions

eygraber · Answer

But I m assuming it s the OOM killer and the fallback strategy has been masking that for a while

Vampire · Answer

Add the <https github com marketplace actions debugging with ssh> action to be able to SSH into the machine and add a step that runs `if always ` after the failing step and keeps the workflow running in case the debugging with ssh action does not do that already anyway

eygraber · Answer

Oh cool I ll check that out thanks

tapchicoma · Answer

yeah looks like OOM in Kotlin daemon Please check the Kotlin daemon logs inside ` tmp` directory on the runner

eygraber · Answer

Going to do that shortly I tested lowering the `Xmx` to `3G` from `6G` for both the gradle daemon and the compiler daemon and now I get this failure almost immediately ```Failed to compile with Kotlin daemon java lang RuntimeException Could not connect to Kotlin compile daemon at org jetbrains kotlin compilerRunner GradleKotlinCompilerWork compileWithDaemon GradleKotlinCompilerWork kt 192 at org jetbrains kotlin compilerRunner GradleKotlinCompilerWork compileWithDaemonOrFallbackImpl GradleKotlinCompilerWork kt 150 at org jetbrains kotlin compilerRunner GradleKotlinCompilerWork run GradleKotlinCompilerWork kt 123 at org jetbrains kotlin compilerRunner GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction execute GradleCompilerRunnerWithWorkers kt 74 at org gradle workers internal DefaultWorkerServer execute DefaultWorkerServer java 63 at org gradle workers internal NoIsolationWorkerFactory$1$1 create NoIsolationWorkerFactory java 66 at org gradle workers internal NoIsolationWorkerFactory$1$1 create NoIsolationWorkerFactory java 62 at org gradle internal classloader ClassLoaderUtils executeInClassloader ClassLoaderUtils java 100 at org gradle workers internal NoIsolationWorkerFactory$1 lambda$execute$0 NoIsolationWorkerFactory java 62 at org gradle workers internal AbstractWorker$1 call AbstractWorker java 44 at org gradle workers internal AbstractWorker$1 call AbstractWorker java 41 at org gradle internal operations DefaultBuildOperationRunner$CallableBuildOperationWorker execute DefaultBuildOperationRunner java 204 at org gradle internal operations DefaultBuildOperationRunner$CallableBuildOperationWorker execute DefaultBuildOperationRunner java 199 at org gradle internal operations DefaultBuildOperationRunner$2 execute DefaultBuildOperationRunner java 66 at org gradle internal operations DefaultBuildOperationRunner$2 execute DefaultBuildOperationRunner java 59 at org gradle internal operations DefaultBuildOperationRunner execute DefaultBuildOperationRunner java 157 at org gradle internal operations DefaultBuildOperationRunner execute DefaultBuildOperationRunner java 59 at org gradle internal operations DefaultBuildOperationRunner call DefaultBuildOperationRunner java 53 at org gradle internal operations DefaultBuildOperationExecutor call DefaultBuildOperationExecutor java 73 at org gradle workers internal AbstractWorker executeWrappedInBuildOperation AbstractWorker java 41 at org gradle workers internal NoIsolationWorkerFactory$1 execute NoIsolationWorkerFactory java 59 at org gradle workers internal DefaultWorkerExecutor lambda$submitWork$0 DefaultWorkerExecutor java 169 at java base java util concurrent FutureTask run FutureTask java 264 at org gradle internal work DefaultConditionalExecutionQueue$ExecutionRunner runExecution DefaultConditionalExecutionQueue java 187 at org gradle internal work DefaultConditionalExecutionQueue$ExecutionRunner access$700 DefaultConditionalExecutionQueue java 120 at org gradle internal work DefaultConditionalExecutionQueue$ExecutionRunner$1 run DefaultConditionalExecutionQueue java 162 at org gradle internal Factories$1 create Factories java 31 at org gradle internal work DefaultWorkerLeaseService withLocks DefaultWorkerLeaseService java 249 at org gradle internal work DefaultWorkerLeaseService runAsWorkerThread DefaultWorkerLeaseService java 109 at org gradle internal work DefaultWorkerLeaseService runAsWorkerThread DefaultWorkerLeaseService java 114 at org gradle internal work DefaultConditionalExecutionQueue$ExecutionRunner runBatch DefaultConditionalExecutionQueue java 157 at org gradle internal work DefaultConditionalExecutionQueue$ExecutionRunner run DefaultConditionalExecutionQueue java 126 at java base java util concurrent Executors$RunnableAdapter call Executors java 539 at java base java util concurrent FutureTask run FutureTask java 264 at org gradle internal concurrent ExecutorPolicy$CatchAndRecordFailures onExecute ExecutorPolicy java 64 at org gradle internal concurrent ManagedExecutorImpl$1 run ManagedExecutorImpl java 49 at java base java util concurrent ThreadPoolExecutor runWorker ThreadPoolExecutor java 1136 at java base java util concurrent ThreadPoolExecutor$Worker run ThreadPoolExecutor java 635 at java base java lang Thread run Thread java 833 Using fallback strategy Compile without Kotlin daemon Try gradlew stop if this issue persists warning ATTENTION This mode is not recommended for production use as no stability compatibility guarantees are given on compiler or generated code Use it at your own risk ```

eygraber · Answer

< tapchicoma> it looks like there is no kotlin daemon log It actually looks like the daemon never started at all

eygraber · Answer

After much debugging there are two issues 1 On GitHub Actions Ubuntu 20 04 runner an Xmx of 6g for both Gradle and Kotlin is too high because the runner has 7g of RAM I solved this by lowering the Xmx and now it works fine with the fallback strategy disabled although still takes roughly the same amount of time 2 On GitHub Actions Ubuntu 22 04 runner which `ubuntu latest` now points to ~no matter what I do with the Xmx it seems like the Kotlin daemon never runs~ after a lot of fat trimming I was able to get this to work seemingly consistently but at the cost of increased build time When it does fail there are no logs in ` tmp` and the OOM reaper didn t log anything about killing any process Might be related to <https github com actions runner images issues 6709>

tapchicoma · Answer

gt It actually looks like the daemon never started at all That is really strange Could you get logs with Gradle ` debug` and share them

eygraber · Answer

I m a little burnt out of Kotlin Gradle right now it s been a stressful few days of updating to 1 8 8 0 sweat smile I ll get to this as soon as I can but likely won t be until mid next week

tapchicoma · Answer

we have several cases reported with OOM in Metaspace Probably you are affected as well

eygraber · Answer

I was looking at jconsole with some local builds and didn t notice any issues with metaspace I ll try again with some different scenarios The GitHub Actions thing might have been transient because I can t get it to happen again

eygraber · Answer

Guess I wasn t paying attention There s something going on with metaspace Is there a youtrack github issue I can follow

eygraber · Answer

Left it running for longer definitely something going on with metaspace Also looks like the Kotlin daemon but not the Gradle daemon has some kind of leak with CodeHeap non profiled nmethods but I don t know too much about that `ReservedCodeCacheSize` is set to `240m`

tapchicoma · Answer

You could track this issue <https youtrack jetbrains com issue KT 56093>

eygraber · Answer

< tapchicoma> would this cause the corresponding metaspace leak in the gradle daemon I was running detekt which I believe uses the Kotlin compiler in process

tapchicoma · Answer

yeah most probably