https://kotlinlang.org logo
#arrow
Title
# arrow
d

dave08

11/20/2023, 2:55 PM
If
parMap
by default uses an
EmptyCoroutineContext
doesn't it mean that it's not running in parallel at all? If so, it's a bit confusing, since in the docs, it uses the one w/o parameters... it should be
parMap(<http://Dispatchers.IO|Dispatchers.IO>...) { .. }
or something?
j

Javier

11/20/2023, 3:24 PM
empty doesn’t map to the default dispatcher?
d

dave08

11/20/2023, 3:25 PM
Copy code
/**
 * An empty coroutine context.
 */
@SinceKotlin("1.3")
public object EmptyCoroutineContext : CoroutineContext, Serializable {
    private const val serialVersionUID: Long = 0
    private fun readResolve(): Any = EmptyCoroutineContext

    public override fun <E : Element> get(key: Key<E>): E? = null
    public override fun <R> fold(initial: R, operation: (R, Element) -> R): R = initial
    public override fun plus(context: CoroutineContext): CoroutineContext = context
    public override fun minusKey(key: Key<*>): CoroutineContext = this
    public override fun hashCode(): Int = 0
    public override fun toString(): String = "EmptyCoroutineContext"
}
That's from the coroutines code...
And parMap uses async(coroutineContext...) behind the scenes.
What's the default for async(...) if you use EmptyCoroutineContext?
j

Javier

11/20/2023, 3:29 PM
Coroutine context is inherited from a CoroutineScope, additional context elements can be specified with context argument. If the context does not have any dispatcher nor any other ContinuationInterceptor, then Dispatchers.Default is used. The parent job is inherited from a CoroutineScope as well, but it can also be overridden with corresponding context element.
d

dave08

11/20/2023, 3:32 PM
Oh, so it uses the context of where the suspend fun is being run... but in Android isn't that Dispatchers.Main?
Unless specified...
If so, there should maybe be a warning in the docs about this... 🤕, since in most cases, what we're using parMap (or parZip?) for are things we wouldn't want to run on the main thread...
j

Javier

11/20/2023, 3:41 PM
To be honest I don’t specify any dispatcher at all and everything works in parallel with no issues for me. At the end Retrofit or whatever library is using IO under the hood.
But that statement is not avoid parMap, but just calling two launch or async inside a VM
d

dave08

11/20/2023, 3:43 PM
Yeah, I've seen some that don't specify anything and others that do... I wonder what's really going on there...
I guess any conclusions there would be the same for parMap...
j

Javier

11/20/2023, 3:46 PM
Probably even if both
launch
happens in the main thread, they are instantaneous, and the long task is done in the deeper dispatcher, which will be the one used by the library, so
IO
, or a custom one like Room does for example
I mean, wrapping a dispatcher does nothing, if a library is using internally
IO
to do a request, if you wrap the request with a different dispatcher, your dispatcher will not be used and
IO
will be used anyway
d

dave08

11/20/2023, 3:49 PM
Dispatcher.Main
is a pool? Otherwise, it shouldn't run parallel as if you created a dispatcher with one thread...
c

CLOVIS

11/20/2023, 3:49 PM
EmptyCoroutineContext
means "use the exact same context as what this function is already running in".
<http://Dispatcher.IO|Dispatcher.IO>
means "use the exact same context as what the function is already running in, but replace the dispatcher by Dispatcher.IO" You can debug what the current context is at that point in the code with
println(currentCoroutineContext())
d

dave08

11/20/2023, 3:49 PM
Then the requests will be one after the other, with no gain?
j

Javier

11/20/2023, 3:50 PM
Not parallel but instantaneously, which is the time to run two functions in a row if those functions do nothing? At the end, the long task would be done in IO
d

dave08

11/20/2023, 3:51 PM
EmptyCoroutineContext
means "use the exact same context as what this function is already running in".
<http://Dispatcher.IO|Dispatcher.IO>
means "use the exact same context as what the function is already running in, but replace the dispatcher by Dispatcher.IO"
Yeah, that's what I understood from @Javier, but more explicit. I think something like that should be in the docs to avoid confusion...
c

CLOVIS

11/20/2023, 3:52 PM
It's how every coroutines launcher works (including
launch
and
async
in the coroutines lib), you can see they all have it as a default value
d

dave08

11/20/2023, 3:55 PM
Well, isn't the arrow version "high level concurrency" -- I wonder if everybody using it would understand this, since there's a version of parMap without that parameter... which the IDE uses, so it's not as obvious...
Yeah, it's there too, I see what you mean @CLOVIS...
I guess I just wasn't expecting that to be the default behaviour for some reason.
c

CLOVIS

11/20/2023, 3:57 PM
What did you expect?
I agree that it's not documented there, but it's a common pattern with coroutines, so it wouldn't be the right place to document it. I think it should be added in the documentation of
EmptyCoroutineContext
, since in my experience that's the only place where it is used.
d

dave08

11/20/2023, 3:59 PM
That it should run in parallel...
*par*Map
on an unlimited pool, or limited by a concurrency limit, if specified... in async {} there isn't any concurrency parameter, you just limit the pool, so it could get confusing.
s

stojan

11/20/2023, 3:59 PM
if you follow the convention of suspend functions being main safe, then it's a perfectly reasonable default. e.g. if you run two retrofit functions as an argument to
parMap
, they will run in parallel as retrofit will run them on OkHttp's thread pool by default
d

dave08

11/20/2023, 4:00 PM
Copy code
public suspend fun <A, B> Iterable<A>.parMap(
  context: CoroutineContext = EmptyCoroutineContext,
  concurrency: Int,
  f: suspend CoroutineScope.(A) -> B
): List<B>
c

CLOVIS

11/20/2023, 4:00 PM
It runs concurrently, always. It only runs in parallel if the dispatcher it is running in supports parallel execution (that's why there is a parameter, in case you want to override it).
d

dave08

11/20/2023, 4:06 PM
So even if parMap is used on the Dispatchers.Main context, it'll still be running retrofit requests off the main thread concurrently? So I guess the only situation when we'd need to specify a dispatcher is with some blocking operations running inside parMap itself...?
j

Javier

11/20/2023, 4:08 PM
I think the only part which will not be in parallel is the time to run N
launch/async
. Those milliseconds would be synchronous. After those milliseconds, the
join/await
will be waiting until all requests are made. They would be happening in IO.
c

CLOVIS

11/20/2023, 4:28 PM
@dave08 just for clarification, can you write an example of the code you're thinking of? Even if it's just a stub with something like
// network request here
.
d

dave08

11/20/2023, 4:34 PM
Well in one place I DO have just that network request, but in another place, I have something a bit more computation oriented, and possibly blocking, but it's a bit too complicated to post here... it's a state machine with calls to android api that might block...
Like to package manager functions.
But maybe in that case, the best thing is just to run those calls in a
withContext(...) { }
in a suspend function and then call that from parMap w/o params
c

CLOVIS

11/20/2023, 4:37 PM
Copy code
withContext(foo) {
    parMap { … }
}
is exactly the same as
Copy code
parMap(foo) { … }
(the second is probably very slightly faster, but I doubt it's even measurable)
d

dave08

11/20/2023, 4:39 PM
I mean:
Copy code
suspend fun someBlockingthing() = withContext(foo) { ... }

parMap { someBlockingthing(); .... }
c

CLOVIS

11/20/2023, 4:40 PM
Then, the difference is that
....
in
parMap
runs in whatever called
parMap
d

dave08

11/20/2023, 4:42 PM
Which is what would happen with Retrofit. If there are a few such functions using withContext(...), then it might be more efficient to use the dispatcher straight on parMap to avoid context switching, though.
c

CLOVIS

11/20/2023, 4:44 PM
Yes, though I doubt it will make much difference.
s

simon.vergauwen

11/21/2023, 10:33 AM
Copy code
withContext(foo) {
  parMap(foo) { }
}
Doesn't result in a performance hit over
parMap(foo)
, but you should only use
withContext
if more than only
parMap
needs to run on
foo
. The reason is that
intercepting
or scheduling is what costs time (throughput/performance), and
parMap
will schedule every element on a new coroutine. Besides that the behavior just follows the same patterns as KotlinX Coroutines, so as everyone mentioned above
EmptyCoroutineContext
is similar to
null
. In the sense that
ctx + EmptyCoroutineContext == ctx
and
EmptyCoroutineContext + ctx == ctx
. So in this way
CoroutineContext
has a
Monoid
instance, similar to the one for (immutable) maps.
It runs concurrently, always. It only runs in parallel if the dispatcher it is running in supports parallel execution
This is a very important nuance, but applies to KotlinX Coroutines exactly the same.
2