@nkiesel There is no need to overcomplicate

waiFor

just

waitFor(predecessors: Tasks) = predecessors.forEach { it.await() }

works just as fine

@elizarov will try that later today

You don't have to have all the tasks created in advance in the proper order. You can start tasks lazily and keep a map of the actually created tasks that is lazily filled as they are instantiated. No need to do toposort in advance with such design.

Hmm. I could first create the graph but not start (i.e. start lazy) and then the final await on all of them triggers their start? I that the idea? How would I trigger the start? From the coroutines docu I remember I would have to have some code that asks for their results (even if I ignore them because the tasks store the real results somewhere else (e.g. file system)). if I understand correctly, this would mean that I would update the predecessor list for tasks as I walk though the task list and find their dependencies. Would also mean that I would "pre-create" tasks I have not seen yet but that the current task depends on and then update the task once I really process it. One downside is that processing only starts after the whole graph was constructed. Might not be a real issue. After all, I expect task execution to be expensive (else I would not create coroutines for them).

When you start, you have some object that defines a code that that task shall execute. Let’s call it “a recipe”. You have a function

execute(recipe)

that takes a recipe and performs whatever action is needed. Each recipe has a list of predecessor recipes. I suggest to keep a

Map<Recipe, Job>

that is lazily initialised. When a task has predecessors it looks up or creates a jobs for them, then waits on their jobs before executing its own recipe.

I meant what you called "...looks up or creates a job". Problem I was anticipating is that the recipe for the predecessor is not known yet (only the name of the recipe is known). Of course not a problem if I already have a map from recipe name to recipe.

What I mean is that receipe is not a job yet. You either look up its job (by name, for example) or create it with

async(context) { awaitPredecessors(); execute(receipe) }

But I then also must make sure that the execution does not start immediately, because subsequent jobs might still be added to the predecessors. Perhaps by having a "start" job that every other job depends on. Once the graph is completed, finishing the start job would trigger execution of the rest.

You shall load all the receipes first (and build predecessor graphs), then trigger the execution of your “root” job (whose results you need).

Let me try to code that and then resume this discussion