Wild thoughts about supporting reference of other ...

# github-workflows-kt

[louiscad]

Wild thoughts about supporting reference of other steps/jobs:
I think we could have a type wrapping a reference to a previous job or step, with or and and functions/extensions, plus maybe other operators, and that would allow the library to keep track of which values are referenced, and automatically add the right needs clauses.

[Piotr Krzemiński]

Seems to be possible only partially, because sometimes the dependency is there for a side-effect, not explicit result of a job. But yeah, inferring it is something we could look at somewhat in the future. A real-life example would help a lot as well :)

[louiscad]

We could have both. Implicit, always enabled because it's a mistake to forget the needs clause, and GitHub Actions is bad because it runs and passes null 😒
And explicit when you just want stuff to run one after the other without passing data from one job to the other

[louiscad]

Examples are publishing to MavenCentral

[louiscad]

Here's a simple one: https://github.com/Beepiz/BleGattCoroutines/blob/177dab479c638d9f89c2183b7e6aaa0d9ed04c1e/.github/workflows/publish-to-maven-central.yml

[louiscad]

(publishes only for Android)

[louiscad]

Here's a more complex one that publishes a multiplatform library and checks the build before publishing. https://github.com/LouisCAD/Splitties/blob/1ea7e072ae7fba5b989f226dd8371cd22d0916ed/.github/workflows/publish-to-maven-central.yml

[Nikky]

why does windows-checking depend on linux-upload and such?
and.. how could it infer this..? what we could do is add all the needs of jobs it depends on too (if that not happeneing already) to make sure github executes things in strictly the correct order

[louiscad]

why does windows-checking depend on linux-upload and such?

Because it builds with stuff published from Linux as well.

[louiscad]

First and last line of this snippet are linked, and the first line could be inferred from the last: https://github.com/LouisCAD/Splitties/blob/1ea7e072ae7fba5b989f226dd8371cd22d0916ed/.github/workflows/publish-to-maven-central.yml#L27-L41

[Nikky]

the only thing you should need to specify is the dependency of linux-upload needs create-staging-repository , linux-checking needs linux-upload and `finalize`needs linux-checking (and the other platforms, but don't want to type so much)
the dsl could make the dependencies a exhaustive list of eg. create-staging-repository, windows-checking, macos-checking, linux-checking for the finalize step

[Nikky]

and adding needs based on usage of outputs would implicitely add them to the dependencies too.. yeah

[Nikky]

making a dsl for this would be a bit annoying without keeping references to steps of other jobs around.. because they are created in their own scopes

[louiscad]

With contracts callsInPlace(ExactlyOnce), you can solve that

[Nikky]

how ?

[louiscad]

You can write that:

val someJob: GitHubActionsJob
workflow {
    someJob = job(name = "whatever") {
        // Whatever
    }
    job(name = "something else") {
        someJob.whatever()
        whatever(someJob.output["something?"])
    }
}

[louiscad]

Granted the workflow function has the right callsInPlace contract.

[Nikky]

i wonder if we can make this work instead

val someJob = object : Job("someJob") {
   val someStep by step()
}

val nextJob = object : Job("NextJob") {
   val nextStep by step(
       env = linkedMapOf(
           "value" to expr(
             needs[someJob].steps[someJob.someStep].outputs.get["something"]
           )
        )
   )
}

would be nice if we can refer to steps in other jobs by name somehow.. but ... no clue how to make kotlin expose variables in other objects or scopes

[Nikky]

we could remove some scoping.. and use builders for steps that expose needs that as a side effect adds the dependency to the job they are attached to

but it looks terrible:

val workFlow = workflow(
    name = "Docker build & push",
    on = listOf(
        Push(branches = listOf("main")),
        PullRequest(branches = listOf("main"))
    ),
    sourceFile = Paths.get(".github/workflows/workflow.main.kts"),
    targetFile = Paths.get(".github/workflows/workflow.yml"),
) {
    val someJob = job(name = "someJob", runsOn = UbuntuLatest)
    val someStep = someJob.step(
        name = "someStep",
        action = SomeAction()
    )
    val nextJob = job(name = "nextJob", runsOn = UbuntuLatest)
    val nextStep = nextJob.uses(
        name = "nextStep",
        action = NextAction()
    ) { // this: UseBuilder ->
        env = mapOf(
            "arg" to expr(steps[someStep].outputs.get["someOutput"])
        )
    }
}

writeWorkflows(
    addConsistencyCheck = true,
    workflows = listOf(
        workflow
    ),
    args
)

.. unless ofc there is also cross workflow interactions that we need to model..

could be improved by using delegates maybe `val some job name by job() {}`

[Nikky]

maybe if i am insane enough i could draft this up in a PR..