i have a bit of parallel processing along the line...

# coroutines

[Nikky]

i have a bit of parallel processing
along the lines of

val targetFiles = Collections.synchronizedMap(mutableMapOf<String, File>())
val features = Collections.synchronizedList(mutableListOf<SKFeatureComposite>())

val jobs = mutableListOf<Job>()
for(...) {
   jobs += launch {
      //do work here
      targetFile[id] = someFIle
      if(feature != null) {
         features += feature
      }
   }
}

jobs.forEach { it.join() }

is there cleaner ways to handle collecting data that lets me juggle less mutable collections ?

[Dominaezzz]

You might want to consider using async instead of launch, then return the feature, someFile and id. Then collect the results in the forEach. To avoid the synchronization when using those collections.

[Dominaezzz]

Or use `Channel`s maybe but that might be unnecessary.

[Nikky]

well feature might not be there.. so i would have to use a nullable and split it up again later

[Nikky]

the annoying part is that its not really a case for .map {}

[Dominaezzz]

I guess for the case of feature, you could use a Channel.

[Dominaezzz]

Then all the "locking" would be asynchronous.

[Nikky]

is there any resources you could recommend to learn about Channels ?

[Dominaezzz]

https://github.com/Kotlin/kotlin-coroutines/blob/master/kotlin-coroutines-informal.md#channels

[Dominaezzz]

That should do it.

[Nikky]

xo i would use 2 seperate channels here, send to each when i get data and then collect it in synchronous code in the end ?

[Nikky]

seems too easy
/shrug

[Dominaezzz]

That's because it is. 🙂

[Dominaezzz]

https://github.com/Kotlin/kotlinx.coroutines/blob/master/coroutines-guide.md#channels

[Dominaezzz]

That's a more detailed resource if you need one.

[Nikky]

thanks for the help.. this is quite helpful

[Nikky]

seems to be almost working,
for some reason the channel blocks when i call .toMap() or .toList() on it however

[Nikky]

apparently the iterator never finishes.. how do i tell it that the channel is done then ?

[Nikky]

ok nvm i found out what was wrong.. i used close but tried to iterate over the channel multiple times.. first time seems to consume all data

[Dominaezzz]

Yeah, it more of a pipe than an actual collection.

[Daniel Tam]

well for your job list, you can surely do targetFiles.map {}?

[Daniel Tam]

and then you can call joinAll on the resulting list of jobs

[Daniel Tam]

but i'm not exactly sure what variables you need inside each launch { } block so I'm not exactly sure how you should build the list

[Daniel Tam]

if it only depends on the loop variable you can do something like List<Job>(some size here) { index -> }

[Nikky]

this is based on files that were read from disk, so i aleady have a list of entries, in this case mabe i could have mapped some parts of it to deferred wrappers,, but i'd rather learn how to make it owrk for the harder cases.. which i did.. channels are quite handy