Hi all, I am developing a Android library with one API exposed to encrypt a file. It returns the Flo...

# coroutines

[Nandu]

Hi all, I am developing a Android library with one API exposed to encrypt a file. It returns the Flow. On the client app I have a list of files for which I need to encrypt. Can you please let me know how to chain all the calls for the list of files to be encrypted and run in parallel and get all the encrypted content once so that I can do the further processing.

[gildor]

Could you show some pseudo code to understand what you are trying to achieve
From your message it looks like you need merge(), map(), toList()
But hard to say exactly

[Nandu]

I have got 2 API’s returning flowsfirst one first() returns a list of file uri’s and 2nd API second() encrypts the file and returns the new uri of the encrypted file. So in my app code looks something like this
first().collect{ uriList ->
for(uri in uriList){
second(uri).collect{ encryptedUri ->
encryptedUrilList.add(encryptedUri)
if(encryptedUriList.size == unencryptedUriList){
call the app callback to show the notification
}
}
}
}
What I am looking is how can I start encrypting all files in parllel and need to wait until all the files are encrypted and get the encrypted uri list so that I can show completed message to the user. With the above approach I am maintaining class level list and checking if the encrypted list size has reached the same size then I am calling the call back to show the notification to the user.

[gildor]

From your explanation it looks that you need this:
https://pl.kotl.in/z7VSRgqBs

[gildor]

Though when I see your API it looks that you actually don’t need Flow in this case, because it looks that both first() and second() is just one time operation so no need to use Flow and code will be more obvious if just rewrite it to suspend functions:
https://pl.kotl.in/GCU-5W8Wo

There are valid cases when you need Flow for this case, so for example when first() emits new lists all the time (for example on file system change it emits new lists of files), but for encryptUri it’s not very clear why you need flow, because as I see it just returns encrypted Uri (so 1 output for 1 input) so it more logical even in this case to use suspend function and use Flow only when it can be more than 1 output for each input

[gildor]

The only when you should be careful is to make sure that first() and second() are itself are non blocking and maybe use Default/IO dispatchers for heavy or blocking operations to avoid blocking
Now I just simulate work with delay which is suspend non blocking, so you can see that every item encrypting in parallel from logs

[gildor]

Btw, from your old code this part is not thread safe:

if(encryptedUriList.size == unencryptedUriList){

and you will end up with race condition which cause that last check for encryptedUriList.size == unencryptedUriList is not passed and your callback is never called

encryptedUrilList.add is also not thread safe and you may corrupt your list (some items will be lost) or get ArrayIndexOutOfBounds

[Nandu]

@gildor thanks a lot for clarifying this to me and now it makes more sense to me to use just the suspend function. I really appreciate all your time and efforts to make me understand it better way.

[gildor]

Yep, it’s important to understand that suspend functions used to encode asyncronous operations in coroutines, they used under the hood of Flow/Channels too, so it’s a building block and Flow is Reactive Streams-like abstraction, which build on top of suspend functions, but itself is a framework for managing data streams. If you task can be better represented as data stream, it’s better to use Flow, but Flow is not required itself for asyncronous/parallel operations, depending on task it’s easier to use coroutines directly or Flow, same way as normal functions/method used even in Reactive Streams (though suspend functions just more powerful in terms of asynchronous code support)