Hi I have a question about performing CPU bound operations S

Question

Hi I have a question about performing CPU bound operations Should I be using `Dispatchers Default` or create separate scope with fixed thread pools equal to number of cores what are the pros and cons what are the criteria s to choose between one of this approach for ex Library writer App writer etc I have posted detailed explanation and my use cases here <https stackoverflow com questions 63915078 can i use dispatchers default for blocking cpu bound operations>

louiscad · Answer

Please read the doc of `Dispatchers Default` it answers your question telling that it s designed for that

Pritam Kadam · Answer

Yes I have done good amount of research on the topic Wanted to understand if theres any gotchas I should be aware of For example because this is globally available there could be possibilities that any third part library dependency is using it May be they are accidentally doing IO operations on `Default` etc

louiscad · Answer

A third party library could also call `exitProcess ` or `System exit ` In all cases it needs to be fixed

Pritam Kadam · Answer

And if you create your own Dispatcher then you can cope up with third party libs doing bad operations on default pool in some cases

Pritam Kadam · Answer

I hope you have read my stackoverflow question I am all in to use `Default` dispatcher but we are having debate within the team hence the question

louiscad · Answer

gt I hope you have read my stackoverflow question I did not because the answer to the title was yes

Pritam Kadam · Answer

Ok so jumping to conclusion based on the title does not seem right Theres very little info you can provide in the title

louiscad · Answer

Doing your best to cope with bad libraries is not the right strategy IMO They should avoid being that bad and be fixed or shamed and dropped if they dare to be this bad granted it s on the JVM where you have access to `<http Dispatchers IO|Dispatchers IO>`

Pritam Kadam · Answer

That was one example I am just looking for any gotchas pros cons

rocketraman · Answer

The title of your SO says blocking CPU bound operation but your details don t talk about blocking ops just CPU bound ops Which is it

louiscad · Answer

Gotcha is avoid bad libraries because their power might be limitless

Pritam Kadam · Answer

To give another example having separate scope backed by your own thread pool provides you isolation and you can take control of complete lifecycle maybe supervise

louiscad · Answer

You can make an app or a server lag with just a library even if it doesn t use `Dispatchers Default`

louiscad · Answer

True isolation is process isolation and even that has limits

rocketraman · Answer

If you are CPU bound at the end of the day it doesn t really matter where you run them You re gonna be pegging your CPU regardless

Pritam Kadam · Answer

I completely agree with you < rocketraman> Thats one of the point I brought in the discussion within the team

Pritam Kadam · Answer

and currently went ahead with Dispatchers Default

Pritam Kadam · Answer

But know that we are building library and not application

louiscad · Answer

< rocketraman> Well not really actually What manages running your code will define the share of the CPU your code is allotted versus other code running in the process and on the machine But if there s a bad actor in your code or next to it that interferes with the available share then it needs to be found and hunted down

louiscad · Answer

This responsibility is on the consumers of the library if your library already does its best to not starve the CPU

rocketraman · Answer

True but I don t think his scenario is a bad actor scenario that was just a bad example he gave

rocketraman · Answer

If he runs his CPU bound workload in Dispatcher Default or creates a separate coroutines dispatcher its gonna be the same result

louiscad · Answer

By bad I don t mean malware but bad by misuing shared resources in the runtime

rocketraman · Answer

I know

rocketraman · Answer

Like I said I don t believe that is OPs scenario

Pritam Kadam · Answer

Here is my post from my internal chat that should clarify it wink ```default global are meant for cpu bound operations because they follow no of threads roughly equals to no of cores having another thread pool will not solve a problem if there are 8 CPU bound operations already running and you want to start 9th operation that will have to wait here assumption is machine has 8 cpu cores ```

rocketraman · Answer

< Pritam Kadam> I think you confused people by throwing the word blocking in your OP

Pritam Kadam · Answer

i think blocking CPU bound is correct for example Matrix multiplication BigInteger probablePrime from Roman s example from this blog <https medium com elizarov blocking threads suspending coroutines d33e11bf4761>

Pritam Kadam · Answer

maybe I should just rephrase it to `CPU Intensive` to avoid confusion

Pritam Kadam · Answer

< louiscad> I dont think such meta level statement helps at all in any discussion gt True isolation is process isolation and even that has limits

rocketraman · Answer

CPU bound is clear Roman distinguishes between work that is blocking via being CPU bound and blocking vs IO I know you know this but his use of the term blocking for the former is explanatory When you use that term casually most people will assume IO blocking or blocking by a poorly coded library

louiscad · Answer

< Pritam Kadam> I think it was fair regarding your example of a library that would run blocking I O against `Dispatchers Default` instead of `` or something else in the same process of your library

rocketraman · Answer

The process isolation point is a valid one < Pritam Kadam> I think You can always run CPU bound work in a separate process with cgroups to limit how much of the machine it can use That way the machine is guaranteed to have resources for other work like request handling

louiscad · Answer

Excatly despite the counterpart of having to do some sort of IPC to communicate even if just basic command args and exit code

Pritam Kadam · Answer

That I understand it felt more like sarcasm

louiscad · Answer

Maybe it felt that way but it wasn t slightly smiling face

louiscad · Answer

Maybe my style of writing was oriented by how highly exactly I think of bad libraries that would affect yours or its users badly upside down face But no offense intended and I hope you have a clearer view of the risks and the options now wink

rocketraman · Answer

If your CPU bound work has the option to be broken up into smaller chunks you can `yield` between the chunks to give other coroutines the ability to run It d be interesting to see if that worked well enough to avoid the need for process isolation

Pritam Kadam · Answer

< louiscad> I was just probing to get more views more ideas on the topic And the point about 3rd party libs i have similar opinion here is another snippet from internal discussion this statement was in context of scala s global execution context wink gt 2 As it is available globally anyone could have used it in 3rd party libs for doing IO gt though we should not be using lib with such quality but its hard to find out gt this is fine as long as they are doing cpu ops

Pritam Kadam · Answer

< rocketraman> we are building kotlin dsl backed by coroutines which runs on single thread That way you can freely mutate variables in the callback or any part of the script Now we wanted to recommend pattern for our users to do IO CPU bound ops which definitely cant be done on single thread So we dont know what kind of cpu io ops our users might want to run

rocketraman · Answer

Maybe you want to expose that as part of your DSL then i e have a way for them to essentially do `withContext` but more limited to the options you want to give them

rocketraman · Answer

Depends on how technical your users are I guess

Pritam Kadam · Answer

our users are astronomers so you can expect them to know little things about coroutines and all Our scripting env will hide all these complexities

rocketraman · Answer

You can t hide it all as you need to know where to run stuff Unless your DSL consists solely of higher level operations for which you know the ideal execution context However you stated we dont know what kind of cpu io ops our users might want to run

Pritam Kadam · Answer

Currently we are suggesting to write separate suspend function and wrap that within ` ```withContext Dispatchers Default ``` for cpu bound task and call that function from script

Pritam Kadam · Answer

Bdw its all open source and sample script available here <https github com tmtsoftware esw blob master examples src main kotlin esw ocs scripts examples epics Temperature kts>

rocketraman · Answer

In this context I d probably expose my own dispatcher For now you can simpy alias it to DefaultDispatcher but exposing your own gives you the option to do other things in the future Like run that code in a separate process

Pritam Kadam · Answer

Yes we might end up doing exactly that

Pritam Kadam · Answer

when you say separate process do you mean running that in different JVM forking another jvm from one was not a fun experience I had in the past persevere

rocketraman · Answer

Yes different JVM Don t fork use some IPC mechanism Like gRPC or rsocket or something

Pritam Kadam · Answer

yes might not work for us most probably script and IO CPU bound ops I am talking about will be tightly coupled

rocketraman · Answer

Is it multi tenant ie do you want to make sure one astronomers work does not impact anothers

Pritam Kadam · Answer

No I mean one script will have some data structures immutable variables which they might want to use in these cpu intensive operations back and forth

Pritam Kadam · Answer

and that could be anything