https://kotlinlang.org logo
Join SlackCommunities
Powered by
What would you suggest for massively parallel prog...
# getting-started
l
What would you suggest for massively parallel programming in Kotlin ? (library, framework) single computer (no need for MPI & co) purely asynchronous (no need to synchronize a known state at any moment) context : a bunch of asynchronous “artificial creature” living in a “shared world”. There will be race condition and its an integral part of the simulation. (whatever happens in case of race condition, happens). a thread pool would be good as i want to create 1 thread per creature.
i’m planning kotlin/jvm but kotlin/native could be a possibility if you think that there is a significant advantage to use it in this particular use case
c
For “massively parallel”, I’d think straight-up GPU programming would give you the best results, but you’d have to drop down to lower-level languages like CUDA or OpenCL for that. You might be able to find Kotlin or JVM wrappers for those, though. For purely CPU-based parallel programming, I’d think the core coroutines APIs should give you everything you’d need. Its trivial to split work onto different threads/thread pools, join those tasks together, and offers common communication primitives like Channel, Mutex, and Semaphore. And the async programming style of coroutines being integrated into the language makes your code significantly easier to read and write than using any other parallel programming library like RxJava, or using Threads directly
l
no gpgpu (i’m not good enough at this for something of this complexity) the coroutine documentation isn’t very clear to me about parallel computation. it’s asynchronous, yes. but i understand that it run on a single thread. I want to use as many cpu core as possible (all core, since the problem is “embarrassingly parallel”) look like this chapter is talking about it however https://kotlinlang.org/docs/shared-mutable-state-and-concurrency.html
o
Coroutines are multi-threaded by default, unless you specifically choose a single-threaded scheduler or happen to be on a single-threaded platform (JS).
c
Like I mentioned, it’s pretty trivial to run your code on various thread pools in parallel, it’s just a matter of launching code onto a particular Dispatcher. 
Dispatchers.Main
for single-threaded UI code (platform-dependant), 
Dispatchers.Default
for general CPU-bound parallel work (I believe it’s limited to at most N+1 threads, where N is the number of CPU cores), 
<http://Dispatchers.IO|Dispatchers.IO>
for IO-based work (basically an unlimited number of threads). For example:
Copy code
suspend fun massivelyParallel() {
    withContext(Dispatchers.Default) {
        // anything launched in here will be launched onto the Default thread pool
        
        // launch 10 jobs to all run in parallel. Replace `async` with `launch` for a fire-and-forget model
        val tasksRunningInThreadPool = (1..10).map { 
            async { 
                delay(5000)
            }
        }

        // wait for all 10 jobs to complete before continuing
        tasksRunningInThreadPool.awaitAll()
    }
}
And as @Oliver.O mentioned, JS can only ever be single-threaded, but 
Dispatchers.Default
still runs your code concurrently so it still behaves similar to multithreaded code, even though it’s technically single-threaded
l
thank you all. i’ll learn more about it. i’m not 100% sure (obviously) if coroutine will fit my need but it’s worth learning it anyway. or perhaps i’ll just have to rethink my original “1 thread per creature” plan.
c
The intention with Coroutines is that you’re not thinking in terms of threads, but in the work itself. Which jobs needs to be running in parallel, how do those jobs wait on each other and communicate asynchronously. Threads ultimately become an implementation detail with coroutines, not something you’re intentionally programming for
So to your point of 1 thread per creature, the equivalent in coroutines would be “1 
launch { }
per creature” and you let the coroutines framework figure out the rest. It typically ends up being more efficient than threads overall with fewer wasted system resources, and you don’t really notice the difference unless you’re storing data in 
ThreadLocal
variables
l
yes, it doesn’t fit my original plan. i guess i could do everything in main then offload the heavy computation part to // short lived coroutine. btw, this is what i want to do (the original software/game/simulation is single threaded and really slow)

https://www.youtube.com/watch?v=Ksf6nFsr29Q▾

some creature could run for a few seconds and some other could run for many days (if they’re good enough)
and i would store tons of data in threadlocal variable indeed. only the world view would be shared
well i got another idea that will work with coroutine (if it works the way i think it does). thank you.
m
An actor framework like Akka or Vert.x could also be a good fit to this problem, if I’m understanding this correctly. Neither is specific to Kotlin, but both can be used from Kotlin with no issues.
Very unfortunately Akka recently changed their licensing model so depending on your requirements it might not be an option. That’s a pity, because it is a fantastic concurrency toolkit.
l
i’ll check that too. thank you.
55 Views
Open in Slack
PreviousNext
Powered by