you can probably handle the selection of which task to run u

Question

you can probably handle the selection of which task to run using a select clause which bias the first clause generate tasks over the second combine task

Ruckus · Answer

Could you elaborate on that Maybe I m mistaken but wouldn t that allow only one generate task at a time or let generate and combine run at the same time

bj0 · Answer

well that would depend on what you do after the select clause is successful the select will only choose if it s time to run a generate or combine once you have that decision you have to decide what to do with it

Ruckus · Answer

I must be missing something fundamental because it s still not clicking for me Thanks anyway

bj0 · Answer

well it s hard to get more specific without knowing more about how you want the tasks to operate for instance if you want the generators to run concurrently why do they even need a queue

Ruckus · Answer

Good point I should edit my original question to include that It s because there may occasionally be a large number to run and they are quite CPU intensive so I d like to only run a maximum number of them at a given time probably of cores 1

bj0 · Answer

ah in that case I would probably create a specific threadpool for that task with a set number of threads to limit it

bj0 · Answer

better than manually dealing with limiting

Ruckus · Answer

Yes I imagined it would be something like that I was thinking I d need some sort of worker thread equivalent that would run generate tasks from a queue and some sort of lock to switch between it and the combine task but I m not sure what the best way to do that is while also avoiding a busy `while true ` loop when there are no tasks to run

Ruckus · Answer

I m not sure what the correct terminology is sorry

bj0 · Answer

is the combine task supposed to run on the same threadpool as the generate tasks

bj0 · Answer

when you create a threadpool context and launch your coroutines using it it effectively creates it s own queue and runs them from it internally so you don t have to

Ruckus · Answer

I don t think it needs to and it possibly shouldn t since it should be exclusive I just need to make sure I m either running either generate tasks or a combine task not both

bj0 · Answer

after re reading it its simpler than i thought I had thought combine tasks could be launched by themselves but they always follow a group of generate tasks

bj0 · Answer

in that case there s no need for select

Ruckus · Answer

Yes a simple analogy would be like a video editor I can work on multiple layers and changes to a layer are rendered to an intermediate file The combine task would then take the rendered results and merge them into one video file Changing any layer would add that layer to the render queue and when the render queue empties it fires the combine task While the combine task is running I can still edit layers and they ll still be added to the queue but they won t render till the combine pass is done

Ruckus · Answer

I m not actually working with video but it s a similar enough to be comparable

Ruckus · Answer

From what I ve read in `coroutines guide md` I was thinking an actor would be what I m looking for but I m not sure how to check if there are no more generate tasks in the channel run the combine task then go back to waiting for generate tasks without closing the channel Does that sound like I m roughly on the right track or is there a better way that comes to mind I m also not sure if actor would easily allow for running multiple generates concurrently

bj0 · Answer

yea you don t want an actor if you want them to run concurrently

Ruckus · Answer

Okay good to know

bj0 · Answer

i mean you could use an actor and then launch the tasks from within the actor but you re really only using the actor as a channel in that case

Ruckus · Answer

Makes sense

bj0 · Answer

``` suspend fun run generateInputs Channel lt Int gt val genContext = newFixedThreadPoolContext 5 gen pool suspend fun genTask input Int println launching $input delay input toLong TimeUnit SECONDS println finished $input for input in generateInputs val job = launch genContext genTask input while job isCompleted amp amp generateInputs isClosedForReceive select lt Unit gt generateInputs onReceiveOrNull if it == null println chan closed job join else launch genContext parent = job genTask it job onJoin println joined println combine doCombineTask ```

bj0 · Answer

thats what i came up with based on your description

bj0 · Answer

and you can probably drop the second condition in the `while` loop

bj0 · Answer

the `run` function can be turned into an actor and it would be essentially the same

Ruckus · Answer

Nice it seems to work pretty well No I just have to break it apart and understand it slightly smiling face Thanks a ton