https://kotlinlang.org logo
Join Slack
Powered by
You might remember the issue <https://github.com/t...
# github-workflows-kt
v
You might remember the issue https://github.com/typesafegithub/github-workflows-kt/issues/627 where String with value 
"null"
and 
""
were serialized wrongly as 
null
. We have the same problem with Strings that look like other scalars like ints, floats, and booleans. A String with value 
"4.0"
for example is serialized as 
4.0
which then is interpreted as the float 
4.0
and thus given as 
4
where used, for example in 
${{ matrix.variant }}
. Should we finally just single-quote all single-line Strings, or introduce some more parsing to determine whether it needs quotes or not?
p
ugh, I’m lovin’ it… could you create a bug for it?
I need to spend some time to understand the root cause of this
v
The root cause is
Copy code
if (this is String) {
    if (lines().size > 1) {
        ScalarStyle.LITERAL
    } else if (isEmpty() || (this == "null")) {
        ScalarStyle.SINGLE_QUOTED
    } else {
        ScalarStyle.PLAIN
    }
}
which strives to produce better readable YAML output by not quoting all single-line strings. But for the price of issues like these two.
p
I see - I’d be keen to sacrifice readability here to get correct and simper code
cool you caught it when the lib is still before v1, it would mean a breaking change
v
Quite tedious change with all the embedded integ tests 😕
I want to have an immediate hotfix as reimbursement 😄
p
Thanks! 🙇Merged. To avoid unscheduled release, can you use the Jitpack snapshot (yeah, the coords in the docs need to be updated) from main branch to iterate on your change (I presume this issue doesn't occur in any existing workflow that you rely on)? If not, let me know, I'll prepare a release tomorrow morning
v
It happened in the Spock build where I want to establish the lib currently.
I wondered why the Workflow job names changed in an unexpected way ("4" instead of "4.0", "3" instead of "3.0"). The build itself indeed worked still, as it is lenient enough, as it parses the variant parameter to a 
BigDecimal
anyway. :-D
p
Sorry, a bit confused here, I need a link where this is happening
v
p
Still don't get why you'd expect 3.0 or 4.0 somewhere, I need a more precise pointer
v
There are "variants" for Groovy 2.5, 3.0, and 4.0. In the latest state on that PR, they are defined in 
matrix.groovy
where the build script reads them and also the workflow kts files read them from.
They are written as 
2.5
, 
3.0
, and 
4.0
to the workflow YAML. But before they were Strings in quotes, so for example produced the job 
Verify Branches and PRs / Build and Verify (ubuntu-latest, 3.0, 8) (pull_request)
and giving 
3.0
as project property to the Gradle build. Now with the lib it produces the job 
Verify Branches and PRs / Build and Verify (ubuntu-latest, 3, 8) (pull_request)
and gives 
3
as project property to the Gradle build.
In the actual YAML you still have 
2.5
, 
3.0
, and 
4.0
, but after the YAML is parsed and you use the value using 
${{ matrix.variant }}
it was converted to a number and is only given as 
3
.
Btw. JitPack building is broken: https://jitpack.io/com/github/typesafegithub/github-workflows-kt/main-d92aeda42b-1/build.log
Copy code
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':library:publishMavenJavaPublicationToMavenLocal'.
> Failed to publish publication 'mavenJava' to repository 'mavenLocal'
   > Invalid publication 'mavenJava': artifact file does not exist: '/home/jitpack/build/library/build/libs/library-0.41.0-sources.jar.asc'

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.

* Get more help at <https://help.gradle.org>

BUILD FAILED in 2m 19s
24 actionable tasks: 17 executed, 7 up-to-date

Publishing build scan...
<https://gradle.com/s/ri5jagwk6cixw>

Build tool exit code: 0
Looking for artifacts...
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Dhttps.protocols=TLSv1.2
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Dhttps.protocols=TLSv1.2
Looking for pom.xml in build directory and ~/.m2
Found artifact: io.github.typesafegithub:github-workflows-kt:0.41.0
2023-04-19T17:06:31.992197934Z
Exit code: 0

⚠️ ERROR: No build artifacts found
Expected artifacts in: $HOME/.m2/repository/io/github/typesafegithub/github-workflows-kt/0.41.0
It's probably the same problem I also had when trying to test with 
mavenLocal()
, you require signing even though it is not a release and thus it fails.
With latest 
main
it behaves as expected again.
p
We disabled signing for JitPack, see here. Feel free to create a bug for it
v
You disabled the signing task, but the publication is invalid as you specified it should be singed but then suppressed the task
Actually, as you have Maven Central deploying up anyway (or even otherwise), I'd actually recommend using the Sonatype snapshot repository and auto-deploy snapshot builds there. JitPack is slow, unreliable, and broken by design.
p
Thanks for the idea! I'll definitely consider it: https://github.com/typesafegithub/github-workflows-kt/issues/774
BTW, Jitpack building worked some time ago, I wonder why it broke
v
Thanks for the idea! I'll definitely consider it
For the meantime: https://github.com/typesafegithub/github-workflows-kt/pull/775
BTW, Jitpack building worked some time ago, I wonder why it broke
Gradle update from 7.4.1 to 8.1 I guess
p
Thanks!
v
yw
Btw. when anyway reworking the publishing, you should consider applying the https://github.com/gradle-nexus/publish-plugin, especially when using Sonatype to publish to Maven Central it is quite flaky without it and if wanted it can also help completely automating the release without needing to finish it on the Nexus UI.
p
Qq: have you tested the fix for JitPack?
v
p
Thanks! For the future, it's helpful to write such things in PR description :)
v
I opened the PR first, as that pushes the branch 😉
p
Btw. when anyway reworking the publishing, you should consider applying the https://github.com/gradle-nexus/publish-plugin, especially when using Sonatype to publish to Maven Central it is quite flaky without it and if wanted it can also help completely automating the release without needing to finish it on the Nexus UI.
we already use it https://github.com/typesafegithub/github-workflows-kt/blob/25657f3272f0c74ffb40364af07957f0d50ee6ad/build.gradle.kts#L5
at the moment I don’t need to use Nexus UI to finish anything, publishing to MC happens automatically when pushing a tag
the release process is almost automatic, what’s missing is preparing release notes for GitHub releases, pinging on Slack, and optionally figuring out the right version based on conventional commits, but for now I’d like to control the version manually
v
Oh, right, I didn't see it. Just looked in 
dsl-publishing
, didn't see it there and seen manual repository configuring. Controlling the version manually is absolutely in my favor. Or at least just doing it on explicit task execution and updating and committing a file. I hate it if versions are calculated from VCS history. This makes builds less reproducible, they don't work from an exported sources archive, they don't even work in a Git worktree. I usually update the patch version directly after release, also adding 
-SNAPSHOT
, and bump the minor or major version with the commit that requires this bump. This then builds the proper snapshot builds for the proper next version. And before release I remove the snapshot suffix. The work around the release I usually do using the gradle-release plugin. You currently always build the last released version, even though there were additional changes. This can be quite confusing, especially when using maven local.
p
The work around the release I usually do using the gradle-release plugin.
I’m keen to try out this plugin one day!
You currently always build the last released version, even though there were additional changes. This can be quite confusing, especially when using maven local.
I don’t understand where’s the problem now. If one wants to use maven local, they by default work on the latest commit, and if desired, can check out a tag for a given release.
(probably worth having a separate thread for it because it’s so off-topic form the original message 😅 )
v
Maven local does not mean it is automatically used. Also it is unclean, confusing, and inconsistent, besides that it makes you more work. E.g. imagine I have a latest commit checked out which has still version 
0.41.0
. I deploy it to Maven local to test some changes. So I test it in a workflow still having 
0.41.0
declared while it actually is 
0.42.0-SNAPSHOT
. I'm fine with my changes, create a PR and you merge it, next day you create a new release so I update my version and be good. Two weeks later I open another project that still uses 
0.41.0
. I try to build the workflows and it fails (due to the changes in Maven local I forgot by then). Now I open a bug entry for you, or that project, or adapt to the changes according to IDE and push and thereby breaking it for the others, .......
This is worse in Maven builds or main.kts scripts as there maven local is used automatically, opposed to a Gradle build where you should practically never use Maven local as it is broken by design actually. But even there would be bad. :-)
p
ok, so if I understand you correctly, it’s about explicitly marking the version as non-released one (
x.y.z-SNAPSHOT
) instead of using 
x.y.z
every time that looks like a released version, right?
v
Yes, that and also to increase the version in time when the changes require the change. This you could probabyl easily verify using the conventional commit messages. If with 
!:
the major version (or minor while at 
0.
) needs to be higher than in the last tag, if no 
feat
, then the patch version needs to be higher, if also 
feat
then the minor version needs to be higher.
But if doing such a check, then please handle the case gracefully that it cannot be determined, like when running out of VCS or in a Git worktree.
p
increase the version in time when the changes require the change.
do you mean bumping the version in the very same commit/PR that makes a certain change? so a functional change + major/minor/patch version bump?
v
Yes, that's how I usually do it. If I do a breaking change, I change the major version with that commit if it was not increased already, because as soon as this commit is included, it effectively is a snapshot of the next major version.
And directly after release the release plugin increases the patch version. And as soon as I add a new funtionality the first time after release, I increase the minor version.
Always given I don't forget it of course as I don't use conventional commits. 😄
p
Hmm, this approach has pros and cons. Pros: versions are semantically correct, snapshots are properly versioned. Cons: each commit is polluted with version change, one has to think about it. I know that "pollution" sounds negatively but I guess thats my current take on it :D thanks, I have to think about it
v
Not "each" commit is polluted, only the first feature-commit and the first breaking-commit after a release. And I personally don't see it as pollution, because the changes in that commit require that the version is increased to be correct, so the change belongs to that commit imho. But you could also split it of course, first make the commit that increases the version right before the commit introducing the first changes that needs it. 🙂
p
Not “each” commit is polluted, only the first feature-commit and the first breaking-commit after a release.
ok, got it now! so it also assumes that each version bump besides the “patch” part results in a release, right?
v
No, why? You can make 5 bugfixes and then release a bugfix release, or not and add 2 features and release then. When you increase the version number is not related to when you create the release. Unless you do want to establish some continuous publishing that releases on each commit of course or similar.
You would not (or at least I do not) increase the minor or major version with each feature-change / breaking-change. You increase it the first time you do such a change after a release and you release when you are ready.
You merely say "this commit introduces a feature (or breaking) change, so the next release created needs to have version X".
example commit history top-down: • [work, work, work with version being 1.0.0-SNAPSHOT] • Remove -SNAPSHOT (built and tagged as v1.0.0) • Set version to 1.0.1-SNAPSHOT • Fix bug a • Fix bug b • Implement new feature X and thus increase version to 1.1.0-SNAPSHOT • Implement new feature Y • Do breaking change O and thus increase version to 2.0.0-SNAPSHOT • Implement new feature Z • Remove -SNAPSHOT (built and tagged as v2.0.0) • Set version to 2.0.1-SNAPSHOT
p
so you do “Set version to 1.0.1-SNAPSHOT” as a minimal version bump after v1.0.0 which says “there’s going to be at least a patch-like change, and if something bigger, let’s bump more significant version numbers”
it makes sense to me! I’ll try to follow this, and we’ll need to add 
-SNAPSHOT
to the current version. Will take care of it “later” 🙂 (after work)
are you aware of any tool (for git/GitHub) that would help enforce such versioning based on conventional commits?
this approach is very algorithmic, provided that one correctly uses conventional commits
v
so you do “Set version to 1.0.1-SNAPSHOT” as a minimal version bump after v1.0.0 which says “there’s going to be at least a patch-like change, and if something bigger, let’s bump more significant version numbers”
Exactly
Will take care of it “later” 🙂 (after work)
👌 And then you can also easily publish to Sonatype snapshot repository on each push to 
main
. 🙂
are you aware of any tool (for git/GitHub) that would help enforce such versioning based on conventional commits?
this approach is very algorithmic, provided that one correctly uses conventional commits
Yes, that's what I mentioned, that you should be able to verify it by the commit messages, but no I'm not aware of any such tool.
You could maybe let https://github.com/marketplace/actions/semver-conventional-commits run to determine the correct version and then verify it actually is that version or something like that.
If there are multiple commits you wouldn't get whether it is correct for each commit though, just for the tip, as you cannot run an action for each commit afaik
p
side-note: my goal is to avoid merging via rebasing, I want to allow only squashes. I know that in case of PRs that you recently created, it would result in like 6 PRs, but there are tools for stacked PRs, e.g. Graphite. In general, this whole versioning/releasing issue is interesting, but I’d like to focus on something related to the core functionality first, and polish releasing/versioning when getting closer to v1
v
Hm, looks nice, that Graphite thing. Also for Spock work as Leonard also prefers doing squash merges unless forced otherwise, and I currently indeed have a PR that builds upon another PR.
p
ok, let’s try that in the future PRs 👍
(not necessarily Graphite, but single-responsibility PRs)
v
Oh dang, what a strange decision, for the CLI you need to have Node.js installed locally. As if there were only JS developers in the world. o_O
p
v
• Now just add a workflow that publishes the snapshot version on each push • Then you can throw out JitPack if you like • Also please make signing optional if 
SNAPSHOT
version so that you can publish to maven local without hassle • Shouldn't it already be at 
0.42.0-SNAPSHOT
as you already introduced 5 new 
feat
commits? • The releasing instructions talk about a 
develop
branch, but there is no such branch. Or do you intend to change your process to Git Flow? • Please add to the instructions to use annotated tags and probably also update the existing tags to be annotated ones. Lightweight tags are for short-lived temporary tags, annotated tags are for persistent things like version tags. There are for example quite some commands or tools which consider annotated tags but ignore lightweight tags unless told otherwise if possible. For example try to do 
git describe
.
p
Thanks a lot for review! I'll apply it gradually
@Vampire do you use researchgate/gradle-release or some other one?
v
Yes, the researchgate one is the one I currently use for projects.
p
ok 👌 and do I get correctly that it’s used interactively (locally), not on CI?
v
The atlassian one I personally don't like, because it is one of those not having a version in a file, but calculating it from Git which has imho quite some drawbacks.
You can use it locally, but you can also use it on CI
p
ok, cool - will try to integrate with it in one of the next releases
v
It is Gradle tasks, so as long as you provide the necessary information like signing information and so on, you can perfectly well do it on CI, either triggered by some tag being pushed, or by some branch being pushed, or by manually triggering a workflow, ...
p
my biggest concern is that I need to understand really well what it’s going to do with my branches/tags/versions. I guess it’s just about going through the docs and playing on some side project
I need to gain trust in it 🙂
v
It depends on what you configure, but by default it basically makes a commit that removes 
-SNAPSHOT
, creates tag on it, makes another commit that increases the patch-version and re-adds 
-SNAPSHOT
.
You for example need to pimp the version increasing of course as you need it in multiple places.
But you can probably easily play with it by disabling the automatic pushing of the commits and tag.
8 Views
Open in Slack
PreviousNext
Powered by