Hi, I've an idea for a Kotlin library. I'm seeking...

# coroutines

[Dipendra Singh]

Hi, I've an idea for a Kotlin library. I'm seeking feedback on whether it's viable.Have prepared a README describing the features here: https://github.com/KCharger/KDag

[CLOVIS]

The idea seems interesting, but I don't know exactly in which cases this would be useful. I rarely use channels in general, so maybe I'm not the target audience? I'm curious if you have examples of real world projects in which this would be useful

[CLOVIS]

My approach to libraries in general, however, is: if you need it, other people probably need it as well. You'll never learn more about a user's needs than when they're yourself

[Dipendra Singh]

I'm working on a project in which I receive a stream of data on a websocket which I push to a channel for processing, my consumer pulls data from this channel and sends it to multiple other channels for further processing and so on.
On top of that my application is performance sensitive and I need to log internal state, so a blocking logger won't work for me. So I'm using channels to send log messages as well.
Eventually the websocket stream ends, and I need to shutdown my program.
I would like to close the program only when all the processing has completed and channels are empty. Which I can't achieve by simply cancelling the consumer coroutines as it will lead to data loss.
So to achieve this currently I close all the channels manually in correct order while ensuring all producers of the channel have completed before closing it so no new data is expected.
This manual closing of channels is error prone as I can easily mess up the order.
Also I have to visually ensure that there are no cycles in my processing by mistake.
When trying to come up with a solution what I realized is my processing is like a DAG(Directed Acyclic Graph) where upstream channels are feeding downstream channels and if I can capture this relationship I can automate closing of the channels while ensuring all data is processed and also detect if I've introduced any cycles in processing graph by mistake.

[CLOVIS]

Thanks for the explanation! I haven't come across a need for something like this in my projects, but I do think it would be a useful tool to have in the future.

[Daniel Pitts]

FWIW, I've seen DAGs used elsewhere. I haven't used them myself, so I don't know the pros/cons of them, but having a good Kotlin abstraction over them couldn't hurt.

[Dipendra Singh]

The most popular library I could find which does the same thing is the python library Apache Airflow: https://github.com/apache/airflow/tree/main

[Daniel Pitts]

Yeah, that's the one I've seen used.