<X-posting> from <#C01D6HTPATV|compose-desktop> A...

# gradle

[eygraber]

X-posting from #compose-desktop

After I got the solution to my original question about failing Compose UI tests that are only happening on CI (GitHub Actions) my tests started acting flaky again in a new PR.

Running with --no-build-cache --no-configuration-cache --rerun-tasks causes the flakiness to go away, but when I remove those flags it comes back.

I deleted all of the caches in GitHub Actions, and disabled all CI jobs except the tests, but the build scan still shows some cache hits. I'm assuming that's where the flakiness is coming from, but I have no idea where Gradle is finding this cache.

Anyone know where the flakiness / cache is coming from?

[Adam S]

I don't think the test failure would be related to caching...

[Adam S]

could it be that that the test is sensitive to the resources available on the machine? Meaning, if there CPU is stressed, then the test is more likely to fail? Or maybe there's some shared resources that the tests rely on, that is getting deleted?

--no-configuration-cache will stop tasks from running in parallel, so each individual test will have more system resources. GitHub runner are on the lower end of specs, which usually isn't a problem, but it could be, if there are many high-intensity tests?

[Adam S]

Could it be that the val intents = ArrayList<Intent>() isn't getting updated correctly? I haven't looked in detail, so I'm just guessing, but should it be a threadsafe collection instead?

https://github.com/eygraber/vice/blob/ce0a88c4352bc48811290ee66cf2ad031021c99e/vice-core/src/jvmTest/kotlin/com/eygraber/vice/filter/ThrottlingIntentUiTest.kt#L470

[eygraber]

I thought it might be related to the resources that are available, but the flakiness is gone if I disable the build and configuration cache, and rerun the tasks.

I've never seen --no-configuration-cache stop tasks from running in parallel. Configuration is single threaded, but task execution shouldn't be affected by that.

The whole flow should be happening on the main thread (see how ViceContainer handles that) so it shouldn't need to be thread safe.

[Adam S]

All tasks run in parallel by default, subject to dependency constraints.

https://docs.gradle.org/8.9/userguide/configuration_cache.html#config_cache:intro:performance_improvements

[Adam S]

but actually CC is off in the build scan you shared 🤔

[wasyl]

I agree this failure doesn't really feel like it's got something to do with caching — the field exists, there's no compilation failure etc.

If you have a way of verifying if anything fixes the issue I'd explore --max-workers=1 flag — while your code might be thread safe, there may be something surprising in Compose/skiko/junit internals

[wasyl]

thought it might be related to the resources that are available, but the flakiness is gone if I disable the build and configuration cache, and rerun the tasks.

Do you have a scan of a successful run?

[eygraber]

--max-workers=1 failed as well.

Here's a working one https://scans.gradle.com/s/oayugsnroamyu/performance/build-cache

The only difference is that the cache isn't used at all.

[Adam S]

if you run ./gradlew clean then ./gradlew check --configuration-cache --rerun-tasks locally, do you get test failures?

[eygraber]

Nope, everything passes

[eygraber]

Also regarding your point about configuration cache and parallel

All tasks run in parallel by default, subject to dependency constraints.

I have parallel builds enabled explicitly

[Adam S]

Nope, everything passes

Funny... when I run ./gradlew check --configuration-cache --rerun-tasks most of ThrottlingIntentUiTest fails!

[eygraber]

🤔

[eygraber]

I'll try a completely clean environment

[Adam S]

and this is curious. When I update TestContainer to use a Mutex to protect the intents list, a lot of the tests fail. Why should that be? 🤔

val intents = mutableListOf<Intent>()
    private val lock = Mutex()

    override val compositor = object : ViceCompositor<Intent, Any> {
      @Composable
      override fun composite() = Any()

      override suspend fun onIntent(intent: Intent): Unit = lock.withLock {
        intents += intent
      }
    }

[eygraber]

Very odd. Both my regular env and the clean env work with allTests (with the Mutex change and without).

check works without the Mutex change, and with the change it has flaky behavior (but still sometimes passes).

[eygraber]

I printed the Thread name in onIntent to sanity check that everything was happening on the main thread (it is) and now it started exhibiting flaky behavior with allTests and check 😵‍💫

[eygraber]

Ah it looks like the issue is that the test where I am doing the assertion is on a different thread than the main thread so it sometimes doesn't see the updates to the list

[eygraber]

Not my favorite solution but it seems to work

[Adam S]

weird! Good that you figured it out