A modern programming language that makes developers happier.

kotlinlang

hey there. I have a list of a few million urls and i want to fetch the head response for them. What would be the best approach if I want to limit the amount of urls that can be fetched concurrently? Of course I could have a sequence that produces the requests, chunk them and process one chunk at a time but this feels like my default coming-from-a-java-world solution instead of something tailored to kotlin / around co-routines.

start a coroutine which iterates the list and dumps them into a channel. start N coroutines which take from the channel and perform the HEAD request. N is your concurrency number.

sorry I only come up with this now but this solution wouldn't allow me to stop the processing as soon as one request fails right?

But I like the solution. I'll just give it a try :)

It will if you want to. Just do not catch exceptions, so as soon as one request fails it will cancel the whole scope