A modern programming language that makes developers happier.

kotlinlang

<@UB9K6R4JH> I'm looking at kotest's <https://github.com/kotest/kotest/tree/master/.github/workflows|GitHub workflows> and opportunities to make things easier to maintain with <https://github.com/typesafegithub/github-workflows-kt|github-workflows-kt>. So far I noticed a pretty complex release pipeline, consisting of multiple subworkflows. There's a lot of duplication there. Do you think I could try using the lib to make things simpler there? I thought about merging the `release_(XYZ).yaml` workflows into one, with multiple jobs. Unless there's a good reason to keep them separate (calling them manually? is it done in practice?), then I could keep the current structure and simply express things in a single Kotlin script while removing lots of duplication

CC <@U01681FLZC2> since I see you've been working on CI config a lot!

I don't remember calling them manually, only when they fail and we go "just run all failed jobs again".
Indeed you're right on assuming it's complex and has tons of duplication. I think we just sewed what worked to release on multiple channels and called it a day.

I do like the idea of moving the workflows to github-workflows-kt but I'm very biased -- I really enjoy it and contribute to it in some way or another.

For that let me also ping <@U12AGS8JG> to give his position
And also <@UPFEDEC8J> who's been suffering with me with flaky github actions

yeah, with release workflows it's like "it works, let's not touch it" :smile:  but I learned that (1) sooner or later some changes are needed there so you need to touch it, (2) given that it runs rarely (apart from publishing SNAPSHOT versions) it's beneficial to ensure to some point that it's correct. That's why I think it may be a good place to clean up :muscle:

the main problem the CI config was attempting so solve is that performance was bad. Small changes triggered full rebuilds. So the intention was to avoid running all tests every time, unless the faster (slightly) tests on Linux passed.

Builds were slow because the GHA cache is only 10gb, and Kotest filled that, so there was effectively no cache.

But now Kotest has a remote Build Cache instance, so there are a lot more cache hits and builds are faster. I think a lot of the complication could be removed. By default, just run all tests, on all machines, and fail-fast. (There are some optimisations/exclusions to this rule, but not many).

I think another consideration is sometimes publishing is flakey. So Kotest needed to specifically republish missing artifacts. That's why there are many individual workflows for only publishing one target.

Maybe that could be reworked, to be simpler, deduplicated. E.g. Workflows can have arguments. So

• just have one 'republish' workflow with a 'Kotlin target' parameter.
• Then re-run the build and the workflow will run `gradle publish${kotlinTarget}ToMavenCentral` or whatever.

I don't have a strong opinion about Yaml vs workflows-kt. But before a big refactor I think it could help to write down a high level spec of what is necessary. E.g.

• Need a PR-check workflow. Must check ABI first, and if successful, then runs tests on all targets with a fail-fast matrix on all machines. Must de-duplicate tests (e.g only run JVM tests on Linux, not on all targets).
• Need a publish-all workflow. Must...
• Need a publish-custom workflow to run manually.

thanks for this context! I'll mull it over and propose some simple first change