https://kotlinlang.org logo
Channels
Powered by
l

louiscad

02/27/2022, 5:28 PM
Wild thoughts about supporting reference of other steps/jobs: I think we could have a type wrapping a reference to a previous job or step, with 
or
and 
and
functions/extensions, plus maybe other operators, and that would allow the library to keep track of which values are referenced, and automatically add the right 
needs
clauses.
💡 1
p

Piotr Krzemiński

02/27/2022, 5:57 PM
Seems to be possible only partially, because sometimes the dependency is there for a side-effect, not explicit result of a job. But yeah, inferring it is something we could look at somewhat in the future. A real-life example would help a lot as well :)
l

louiscad

02/27/2022, 5:59 PM
We could have both. Implicit, always enabled because it's a mistake to forget the needs clause, and GitHub Actions is bad because it runs and passes null 😒 And explicit when you just want stuff to run one after the other without passing data from one job to the other
Examples are publishing to MavenCentral
(publishes only for Android)
Here's a more complex one that publishes a multiplatform library and checks the build before publishing. https://github.com/LouisCAD/Splitties/blob/1ea7e072ae7fba5b989f226dd8371cd22d0916ed/.github/workflows/publish-to-maven-central.yml
n

Nikky

02/27/2022, 6:18 PM
why does windows-checking depend on linux-upload and such? and.. how could it infer this..? what we could do is add all the 
needs
of jobs it depends on too (if that not happeneing already) to make sure github executes things in strictly the correct order
l

louiscad

02/27/2022, 6:19 PM
why does windows-checking depend on linux-upload and such?
Because it builds with stuff published from Linux as well.
First and last line of this snippet are linked, and the first line could be inferred from the last: https://github.com/LouisCAD/Splitties/blob/1ea7e072ae7fba5b989f226dd8371cd22d0916ed/.github/workflows/publish-to-maven-central.yml#L27-L41
n

Nikky

02/27/2022, 6:24 PM
the only thing you should need to specify is the dependency of 
linux-upload needs create-staging-repository
, 
linux-checking needs linux-upload
and `finalize`needs 
linux-checking
(and the other platforms, but don't want to type so much) the dsl could make the dependencies a exhaustive list of eg. 
create-staging-repository, windows-checking, macos-checking, linux-checking
for the 
finalize
step
and adding needs based on usage of outputs would implicitely add them to the dependencies too.. yeah
making a dsl for this would be a bit annoying without keeping references to steps of other jobs around.. because they are created in their own scopes
l

louiscad

02/27/2022, 6:30 PM
With contracts 
callsInPlace(ExactlyOnce)
, you can solve that
n

Nikky

02/27/2022, 6:30 PM
how ?
l

louiscad

02/27/2022, 6:33 PM
You can write that:
Copy code
val someJob: GitHubActionsJob
workflow {
    someJob = job(name = "whatever") {
        // Whatever
    }
    job(name = "something else") {
        someJob.whatever()
        whatever(someJob.output["something?"])
    }
}
Granted the 
workflow
function has the right 
callsInPlace
contract.
n

Nikky

02/27/2022, 6:41 PM
i wonder if we can make this work instead
Copy code
val someJob = object : Job("someJob") {
   val someStep by step()
}

val nextJob = object : Job("NextJob") {
   val nextStep by step(
       env = linkedMapOf(
           "value" to expr(
             needs[someJob].steps[someJob.someStep].outputs.get["something"]
           )
        )
   )
}
would be nice if we can refer to steps in other jobs by name somehow.. but ... no clue how to make kotlin expose variables in other objects or scopes
we could remove some scoping.. and use builders for steps that expose 
needs
that as a side effect adds the dependency to the job they are attached to but it looks terrible:
Copy code
val workFlow = workflow(
    name = "Docker build & push",
    on = listOf(
        Push(branches = listOf("main")),
        PullRequest(branches = listOf("main"))
    ),
    sourceFile = Paths.get(".github/workflows/workflow.main.kts"),
    targetFile = Paths.get(".github/workflows/workflow.yml"),
) {
    val someJob = job(name = "someJob", runsOn = UbuntuLatest)
    val someStep = someJob.step(
        name = "someStep",
        action = SomeAction()
    )
    val nextJob = job(name = "nextJob", runsOn = UbuntuLatest)
    val nextStep = nextJob.uses(
        name = "nextStep",
        action = NextAction()
    ) { // this: UseBuilder ->
        env = mapOf(
            "arg" to expr(steps[someStep].outputs.get["someOutput"])
        )
    }
}

writeWorkflows(
    addConsistencyCheck = true,
    workflows = listOf(
        workflow
    ),
    args
)
.. unless ofc there is also cross workflow interactions that we need to model.. could be improved by using delegates maybe `val 
some job name
by job() {}`
maybe if i am insane enough i could draft this up in a PR..
2 Views
Powered by