https://kotlinlang.org logo
Join Slack
Powered by
Is there a way that would make a GitHub CI master ...
# kotest-contributors
o
Is there a way that would make a GitHub CI master merge omit testing if there are still valid test results from the corresponding PR test?
e
Not afaik, when using the merge queue
The idea of using the merge queue was to get rid of the issues with keeping PRs up to date. Merge queue will build and test the PR code with master and spit back the PR if there’s issues. If we got rid of the flakiness I don’t think should be a big deal? Perhaps build cache could help too? We should have a valid cache of the build when running the tests the second time?
o
Does the merge queue avoid the double testing?
e
No
o
Caches are currently read-only in PR tests, so there are no updates propagating to master. If we make them read-write, there would still be no propagation as caches are tied to their specific branch. At least that's what happens by default.
I was thinking along the lines of putting hashes of code tested along with test results in the GH cache. Caching the complete build cache for each PR would probably not work given our cache size limit.
e
I think the main problem would be that there’s be too much cache data, so we would lose the precious master cache due to eviction
o
Yep. Overlap in typing, that's why hashes and results only.
👍 1
e
I think @Adam S might have improvements incoming which would help 😉
o
Seen. Looks promising!
s
It'll be faster when we have three projects instead of one 😉
o
That's for sure. Also sure: For now, replicating the tons of build logic updates would kill us. 😉
s
Heh
I'm waiting to PR a bunch of changes to make the plugin awesome but waiting until its split out
a
Something I noticed that could make CI faster: The 'validate-secondary' step depends on 'validate-primary'. https://github.com/kotest/kotest/blob/63c18dc4ea640c08478f4f4461a509790afb30e1/.github/workflows/PR.yml#L73 Does secondary have to wait until primary is finished? If they were launched in parallel then that would save ~20 minutes
I'm very much looking forward to the plugin changes Sam!
s
It's a game changer
I can't wait to show yall
a
does it change how JS browser tests are launched? Or does it still use Karma under the hood?
s
It's focused on jvm atm
But I have a separate WIP for js
Once we get going we can tie it together
o
Something I noticed that could make CI faster: The 'validate-secondary' step depends on 'validate-primary'.
The idea was to have a form of fail fast, save some energy, and not wait for results from the slowest runner each time. Once those flaky tests are history, we would no longer be interested in waiting for results from those slow runners, as we'd expect them to pass like the primary one in almost all cases.
a
okay, that makes sense. I think GitHub doesn't really have a good way to define that though...
possibly by configuring 'required checks'?
o
We could try and see how GH reporting looks like on the PR tab. Wouldn't it then say overall "Successful" even though secondary runners were unsuccessful?
a
yes I believe so
or, at least it will allow auto-merge to complete
o
OT: Can 
eventually
use the coroutine test scheduler's virtual time already?
a
Open in Slack
PreviousNext
Powered by