https://kotlinlang.org logo
Join SlackCommunities
Powered by
Would it be possible to replace the integrity chec...
# github-workflows-kt
l
Would it be possible to replace the integrity check job that delays all Workflows, with a git hook that runs on workflow for change? Another idea is to just check a checksum or hash with plain shell code rather than running a full Kotlin compile action on CI on every run.
v
Removing the integrity check by default is imho a bad idea, as you really have to understand the consequences of that. But if you don't want it, you can anytime switch it off for your workflows. Doing a hash-based check might be a good idea per-se. But the problem is, that you do not know what to hash. It is not only the one YAML file and one Kotlin file. The Kotlin file can import multiple other files and can have dependencies, even ones with dynamic version or changing version, ...
👍 1
p
@louiscad I've been thinking about caching compilation results for the scripts, then the consistency check job would try to restore this cache. In practice it would cut down the consistency job's time to just the JVM startup which is a few seconds, not twenty-something like with compilation
Note: caching won't work correctly as long as the issue with recompiling if imports change is not fixed. A lot of the balls are now on the JetBrains side, to fix several pain points in the lib
l
Can GitHub Workflows KT know when the workflow isn't just about the kts file, but also about imported files? I'd think so, worst case, by introspecting the kts file. In the cases where no other files are imported (the majority of cases I guess, at least, that's my case), it could use hash/checksum instead, so one-file workflows are fast in CI
I will speak about the library in Paris on the 1st of February, I'd love to be able to tell people that it won't slow down their CI. Currently, using github-workflows-kt for workflows that normally take only a few seconds is a hard sell 😕
v
Well, you can always disable the consistency check if you really want to and thus safe the time for it for the price of possible inconsistencies
l
I don't want to get inconsistencies, wasn't it clear from the beginning?
I want to replace the technique with a cheaper one when it's possible
v
Yeah, but I think there will not be a reliable replacement. If you for example use a pre-commit hook, it is only guaranteed for people that use it. If someone makes a fresh clone, or does a contribution it might be missing. A post-receive hook could maybe work when having a canonical repository, but on GitHub you cannot add custom ones like on SourceForge.
Maybe a hash based check that is manually configured which files to hash could work, so that people can activate it if they want and need. I think you can never reliably auto-detect. One Kotlin file could also generate 0 or multiple workflows. Libraries the script depends on - including this lib - could theoretically change in the remote side, ...
l
The hash check could be the workflow validation job/step
Not a git hook
Just like now, but instead of recompiling, it's checking the hash (when there are no Kotlin files imported)
v
Yes, I got that. You also suggested a gift hook above. That was independent from the hash topic
p
Kotlin's built-in script caching (the broken one) also uses hashing to determine changes. It would be really cool, and the right way, to fix it at the source. I'd like to avoid reimplementing it on the library side
l
Do you know what's broken there?
p
They hash only source of the entry point script, without traversing imported scripts
l
I see, so it's working for single-file workflows
p
Exactly
l
But it's working only on my machine, not on GitHub đŸ„č
If I did some research to support hashing based validation when the workflow isn't importing other scripts of Kotlin files, would you accept it to become a new feature?
v
For it to work on GitHub you need to use the cache action to save and restore the directory where the cache lives, otherwise a fresh run has no cache available
And if the hash based check is added then please as I said only opt-in, and optimally configurable which files are to be checked
👍 2
l
The cache location of Kotlin scripts is still not stable (cc @ilya.chernikov), and might be moved, so it's a time-ticking "bomb", unless the caching code plans for that.
p
Yeah, it's challenging to build on moving foundations, but if I didn't believe things will stabilize and get fixed one day, I'd probably let go already
Open in Slack
PreviousNext
Powered by