https://kotlinlang.org logo
Title
e

elizarov

06/04/2017, 1:53 PM
@nkiesel There is no need to overcomplicate
waiFor
just
waitFor(predecessors: Tasks) = predecessors.forEach { it.await() }
works just as fine
n

nkiesel

06/04/2017, 4:25 PM
@elizarov will try that later today
Yes, works as expected. Thanks. I actually changed it to
suspend fun Tasks.await() = forEach { it.await() }
. One problem with this approach is that I need to create tasks in their topological order / partially sorted. Would be nicer if I could hide this (i.e. somehow allow callers to add nodes and edges of the dependency graph in any order). Any ideas? or Is there another coroutine construct that would no rely on the topological order?
e

elizarov

06/05/2017, 8:12 PM
You don't have to have all the tasks created in advance in the proper order. You can start tasks lazily and keep a map of the actually created tasks that is lazily filled as they are instantiated. No need to do toposort in advance with such design.
n

nkiesel

06/06/2017, 7:35 AM
Hmm. I could first create the graph but not start (i.e. start lazy) and then the final await on all of them triggers their start? I that the idea? How would I trigger the start? From the coroutines docu I remember I would have to have some code that asks for their results (even if I ignore them because the tasks store the real results somewhere else (e.g. file system)). if I understand correctly, this would mean that I would update the predecessor list for tasks as I walk though the task list and find their dependencies. Would also mean that I would "pre-create" tasks I have not seen yet but that the current task depends on and then update the task once I really process it. One downside is that processing only starts after the whole graph was constructed. Might not be a real issue. After all, I expect task execution to be expensive (else I would not create coroutines for them).
e

elizarov

06/06/2017, 7:52 AM
When you start, you have some object that defines a code that that task shall execute. Let’s call it “a recipe”. You have a function
execute(recipe)
that takes a recipe and performs whatever action is needed. Each recipe has a list of predecessor recipes. I suggest to keep a
Map<Recipe, Job>
that is lazily initialised. When a task has predecessors it looks up or creates a jobs for them, then waits on their jobs before executing its own recipe.
No “precreation” is needed. Only tasks that are transitively needed to execute the initial recipe will get created.
n

nkiesel

06/06/2017, 9:17 AM
I meant what you called "...looks up or creates a job". Problem I was anticipating is that the recipe for the predecessor is not known yet (only the name of the recipe is known). Of course not a problem if I already have a map from recipe name to recipe.
e

elizarov

06/06/2017, 9:18 AM
What I mean is that receipe is not a job yet. You either look up its job (by name, for example) or create it with
async(context) { awaitPredecessors(); execute(receipe) }
n

nkiesel

06/06/2017, 11:21 PM
But I then also must make sure that the execution does not start immediately, because subsequent jobs might still be added to the predecessors. Perhaps by having a "start" job that every other job depends on. Once the graph is completed, finishing the start job would trigger execution of the rest.
e

elizarov

06/07/2017, 5:54 AM
You shall load all the receipes first (and build predecessor graphs), then trigger the execution of your “root” job (whose results you need).
n

nkiesel

06/07/2017, 6:37 AM
Let me try to code that and then resume this discussion