https://kotlinlang.org logo
#coroutines
Title
# coroutines
b

bj0

08/14/2018, 5:16 PM
you can probably handle the selection of which task to run using a select clause, which bias the first clause (generate tasks) over the second (combine task)
r

Ruckus

08/14/2018, 5:20 PM
Could you elaborate on that? Maybe I'm mistaken, but wouldn't that allow only one generate task at a time (or let generate and combine run at the same time)?
b

bj0

08/14/2018, 5:28 PM
well that would depend on what you do after the select clause is successful. the select will only choose if it's time to run a generate or combine, once you have that decision you have to decide what to do with it
r

Ruckus

08/14/2018, 5:32 PM
I must be missing something fundamental, because it's still not clicking for me. Thanks anyway.
b

bj0

08/14/2018, 5:34 PM
well it's hard to get more specific without knowing more about how you want the tasks to operate, for instance, if you want the generators to run concurrently why do they even need a queue?
r

Ruckus

08/14/2018, 5:36 PM
Good point, I should edit my original question to include that. It's because there may occasionally be a large number to run, and they are quite CPU intensive, so I'd like to only run a maximum number of them at a given time (probably # of cores - 1).
b

bj0

08/14/2018, 5:38 PM
ah, in that case I would probably create a specific threadpool for that task with a set number of threads to limit it
better than manually dealing with limiting
r

Ruckus

08/14/2018, 5:42 PM
Yes, I imagined it would be something like that. I was thinking I'd need some sort of "worker thread" equivalent that would run generate tasks from a queue, and some sort of lock to switch between it and the combine task, but I'm not sure what the best way to do that is (while also avoiding a busy
while(true)
loop when there are no tasks to run).
(I'm not sure what the correct terminology is, sorry)
b

bj0

08/14/2018, 5:43 PM
is the combine task supposed to run on the same threadpool as the generate tasks?
when you create a threadpool context and launch your coroutines using it, it effectively creates it's own queue and runs them from it internally, so you don't have to
r

Ruckus

08/14/2018, 5:46 PM
I don't think it needs to (and it possibly shouldn't since it should be exclusive), I just need to make sure I'm either running either generate tasks or a combine task, not both.
b

bj0

08/14/2018, 5:53 PM
after re-reading it, its simpler than i thought, I had thought combine tasks could be launched by themselves, but they always follow a group of generate tasks?
in that case there's no need for select
r

Ruckus

08/14/2018, 6:01 PM
Yes, a simple analogy would be like a video editor. I can work on multiple layers, and changes to a layer are rendered to an intermediate file. The combine task would then take the rendered results and merge them into one video file. Changing any layer would add that layer to the render queue, and when the render queue empties, it fires the combine task. While the combine task is running, I can still edit layers, and they'll still be added to the queue, but they won't render till the combine pass is done.
(I'm not actually working with video, but it's a similar enough to be comparable)
From what I've read in
coroutines-guide.md
, I was thinking an actor would be what I'm looking for, but I'm not sure how to check if there are no more generate tasks in the channel, run the combine task, then go back to waiting for generate tasks, without closing the channel. Does that sound like I'm (roughly) on the right track, or is there a better way that comes to mind? (I'm also not sure if actor would easily allow for running multiple generates concurrently.)
b

bj0

08/14/2018, 6:17 PM
yea you don't want an actor if you want them to run concurrently
r

Ruckus

08/14/2018, 6:18 PM
Okay, good to know
b

bj0

08/14/2018, 6:18 PM
i mean you could use an actor, and then launch the tasks from within the actor, but you're really only using the actor as a channel in that case
r

Ruckus

08/14/2018, 6:18 PM
Makes sense
b

bj0

08/14/2018, 6:19 PM
Copy code
suspend fun run(generateInputs: Channel<Int>) {

    val genContext = newFixedThreadPoolContext(5, "gen pool")

    suspend fun genTask(input: Int) {
        println("launching $input")
        delay(input.toLong(), TimeUnit.SECONDS)
        println("finished $input")
    }

    for (input in generateInputs) {

        val job = launch(genContext) { genTask(input) }

        while (!job.isCompleted && !generateInputs.isClosedForReceive) {
            select<Unit> {
                generateInputs.onReceiveOrNull {
                    if (it == null) {
                        println("chan closed")
                        job.join()
                    } else
                        launch(genContext, parent = job) { genTask(it) }
                }

                job.onJoin { println("joined") }
            }
        }

        println("combine")
//        doCombineTask()
    }
}
thats what i came up with based on your description
and you can probably drop the second condition in the
while
loop,
the
run
function can be turned into an actor and it would be essentially the same
r

Ruckus

08/14/2018, 6:40 PM
Nice, it seems to work pretty well. No I just have to break it apart and understand it 🙂 Thanks a ton!
7 Views