Channels
#multiplatform
Title
k
kpgalligan
11/24/2019, 5:19 PMAnybody use Stately collections? We’re starting to look at replacing them and want to better understand use cases.
e
Erik Christensen
11/24/2019, 6:30 PMCurrently using the shared hash map in Island Time (native only) as the closest thing to a
ConcurrentHashMapb
basher
11/24/2019, 10:01 PMWe use the frozen map and list types
k
kpgalligan
11/24/2019, 10:07 PMPlaying with using multithreaded coroutines to implement shared collections by keeping the “collection” in a single thread, and storing/pulling frozen data. Performs significantly better than Stately collections.
b
basher
11/24/2019, 10:29 PMWill that make stately-collections require kotlinx.coroutines?
k
kpgalligan
11/24/2019, 10:29 PMWell, I think it’ll be a different module. That is the issue
Dependency
e
Erik Christensen
11/24/2019, 10:31 PMInteresting finding. So even with the context switch, it's still faster to read and write to a single owning thread?
k
kpgalligan
11/24/2019, 10:31 PMI’m going through the rest of Stately now. A lot of it is implemented by atomic fu
Yeah, but even just doing that with a worker is faster. What’s way faster is if you do batch stuff.
Copy code
val imap = IsoMap<String, SomeData>(createState { mutableMapOf<String, SomeData>() })
val duration = measureTime {
repeat(100_000) {
imap.put("ttt $it", SomeData("jjj $it"))
}
}
println("Coroutine duation ${duration} count ${imap.size()}")
imap.clear()
val coroutineBatchduration = measureTime {
repeat(100) {outerLoop ->
val dataMap = mutableMapOf<String, SomeData>()
repeat(1000){
val index = (outerLoop * 1000) + it
dataMap.put("ttt $index", SomeData("jjj $index"))
}
imap.putMap(dataMap)
}
}
println("Coroutine batch duation ${coroutineBatchduration} count ${imap.size()}")
imap.clear()
val m = mutableMapOf<String, SomeData>()
val mutableDuration = measureTime {
repeat(100_000) {
m.put("ttt $it", SomeData("jjj $it"))
}
}
println("Mutable duation ${mutableDuration} count ${m.size}")
val fm = frozenHashMap<String, SomeData>()
val frozenDuration = measureTime {
repeat(100_000) {
fm.put("ttt $it", SomeData("jjj $it"))
}
}
println("Frozen duation ${frozenDuration} count ${fm.size}")The first 2 are using coroutines, the second is batching. The 3rd is just using mutable state, and the last is using stately
Copy code
Coroutine duation 18.4s count 100000
Coroutine batch duation 634ms count 100000
Mutable duation 451ms count 100000
Frozen duation 30.5s count 100000So, stately is almost 2x worse for puts, but when using batches you can really speed things up. You have more flexibility. In the general case, you provide a lambda that takes the collection as an arg, and as long as you don’t try to return it, you can do what you want
Copy code
open class IsolateState<T : Any>(private val stateId: Int) {
internal suspend fun <R> access(block: (T) -> R): R = withContext(stateDispatcher) {
block(stateMap.get(stateId) as T)
}
}The map then just does whatever through
access Copy code
class IsoMap<K,V>(stateId: Int) : IsolateState<MutableMap<K,V>>(stateId){
suspend fun clear() = withContext(stateDispatcher) {
access { it.clear() }
}
suspend fun size() = withContext(stateDispatcher) {
access { it.size }
}
suspend fun get(mapKey: K): V? = withContext(stateDispatcher) {
access { it.get(mapKey) }
}
suspend fun put(mapKey: K, data: V) = withContext(stateDispatcher) {
access { it.put(mapKey, data) }
}
suspend fun putMap(map: Map<K, V>) = withContext(stateDispatcher) {
access { it.putAll(map) }
}
}I’m still working through complications, but performance and flexibility are pretty good. As for dependency on kotlinx.coroutines, I imagine most apps will eventually depend on that.
b
basher
11/25/2019, 12:23 AMI'd be curious to know what it is about coroutines that makes it faster
k
kpgalligan
11/25/2019, 12:44 AMNothing about coroutines specifically. It’s the mutable state in the single background thread. Could just do this directly with a worker, and may do that. I think playing with coroutines got me thinking.
👍 1
e
Erik Christensen
11/25/2019, 2:29 AMI would think that using C interop to bypass the K/N rules would lead to the fastest implementation. Like what they talk about in the concurrency docs: https://github.com/JetBrains/kotlin-native/blob/master/CONCURRENCY.md#raw-shared-memory
k
kpgalligan
11/25/2019, 4:18 AMWell, yeah. Don't disagree
I started a c++ implementation. However, I'm less concerned about "fastest" than fast and flexible. Will be a general mutable state container
Also, you could just set the state to "shared"
e
Erik Christensen
11/25/2019, 4:20 AMHaha... well, yes. The devil is in the implementation details though. It's a bit of work to really optimize it.
In any case, current stately implementation is super slow
A lot of stately is going to be replaced with atomic fu, so im kind of thinking now is a good time to rethink the whole thing
e
Erik Christensen
11/25/2019, 4:26 AMSeems fair enough.
q
Qinghua
11/25/2019, 5:23 AMPlaying with using multithreaded coroutines to implement shared collections by keeping the “collection” in a single thread, and storing/pulling frozen data. Performs significantly better than Stately collections.It seems that using a Worker can achieve that read/write mutable shared collection in a single thread. Why it needs
coroutinek
kpgalligan
11/25/2019, 6:25 AM"Nothing about coroutines specifically. It’s the mutable state in the single background thread. Could just do this directly with a worker, and may do that. I think playing with coroutines got me thinking." Said that above
q
Qinghua
11/25/2019, 8:19 AMGive a try using a worker as above demonstrated.
Oh, maybe I see your intention in terms of coroutine.