Kotlin Native concurrent mutable state post series...

# feed

[kpgalligan]

Kotlin Native concurrent mutable state post series. Surprisingly productive Sunday! https://dev.to/touchlab/kotlin-native-transferring-state-4n8i

[Dominaezzz]

About keeping a MutableMap in a DetachedObjectGraph , have you considered using DetachedObjectGraph(TransferMode.UNSAFE, ....) when recreating the DetachedObjectGraph ? You could even use DetachedObjectGraph(TransferMode.SAFE, ....) to limit the graph checks on the individual elements.

[Dominaezzz]

Actually, this seems interesting, I'll try implementing this myself.

[kpgalligan]

You could, but I stay away from all UNSAFE usage. It always ends in tears. For me at least

[kpgalligan]

I would double down on that thinking if publishing a library

[Dominaezzz]

I can imagine, I've already hit a wall, can you have frozen objects in a DetachedObjectGraph ?

[kpgalligan]

Yes

[Dominaezzz]

Also, do DetachedObjectGraph have to be explicitly released?

[kpgalligan]

Yes

[Dominaezzz]

Does the frozen object have to be detached too?

[Dominaezzz]

Nahh, that's a silly question, but I feel it shouldn't haaaaaave to be.

[kpgalligan]

Yes? Detaching creates a +1 on the ref count

[Dominaezzz]

Oh!

[kpgalligan]

I think. On phone, having dinner/drinks, so all of my messages are questionable at this poit

[kpgalligan]

Point

[kpgalligan]

Been thinking about this more. The value in UNSAFE would presumably be not checking the whole object graph. It’s risky, but may be useful in some situations. You can also mark it “shared” under the hood, but I wouldn’t recommend that either. Until I need something that performs better, I’ll be focused on the thread confined state. In the past, anytime I try to work around the concurrency rules, I wind up with a race condition that’s very difficult to diagnose.

[Dominaezzz]

I've written up an implementation. Although it's slightly different from what I described.
https://github.com/Dominaezzz/FrozenStructures/blob/master/src/main/kotlin/DetachedMutableMap.kt

[Dominaezzz]

It basically keeps frozen keys and values in `StableRef`s, this way the MutableMap can stay completely detached.

[Dominaezzz]

Too tired/sleepy to tidy/fix up the locking. Will continue tomorrow.

[kpgalligan]

I’m going to look at it, but the unsafe is definitely freaking me out. I had an issue even using safe that I still haven’t figured out, but since I’m not really using DetachedObjectGraph for anything besides a comparison benchmark, I didn’t dig too deep.

[kpgalligan]

Get anywhere with this? I ran it on mac and testBasicThreads has errors that look like memory related race conditions.

[ RUN      ] DetachedMapTest.testInitFrozen
[       OK ] DetachedMapTest.testInitFrozen (0 ms)
[ RUN      ] DetachedMapTest.testBasicThreads
basic threads time: 1.23ms
kotlin.AssertionError: Expected <62>, actual <1850>.
/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/cpp/Memory.cpp:1091: runtime assert: Must be positive
Abort trap: 6
Kevins-MacBook-Pro-5:FrozenStructures kgalligan$

[Dominaezzz]

Yeah I fixed it and pushed it. The issue was that I forgot to wait for the workers to finish, after terminating them. (I don't use workers very much).

[Dominaezzz]

That runtime assert though... I think the Kotlin runtime doesn't wait for all Workers to finish before terminating, like the JVM.

[Dominaezzz]

But that's a different issue.

[kpgalligan]

[ RUN      ] DetachedMapTest.testInitFrozen
[       OK ] DetachedMapTest.testInitFrozen (0 ms)
[ RUN      ] DetachedMapTest.testBasicThreads
basic threads time: 160ms
kotlin.AssertionError: Expected <1881>, actual <1850>.
/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/cpp/Memory.cpp:1091: runtime assert: Must be positive
Illegal instruction: 4

[kpgalligan]

I would not assume Memory.cpp asserts are benign. Also, the actual count check is wrong, obviously. Not sure which check that is, though.

[Dominaezzz]

Hmm, I'll try and look into that. Can't reproduce on my Linux setup.

[kpgalligan]

Is Linux in a VM? Totally speculating, but if VM it might only run on one core

[Dominaezzz]

Nope, I run in directly on my computer.

[Dominaezzz]

I figured out the issues.
The single-threaded tests failed because iterators create cyclic garbage, solved by calling GC.collect() before detaching.
The multi-threaded test failed because waitForMultipleFutures(workers.map { it.requestTermination() }, 5000) only waits for a single future, very misleading name. So waiting for the workers manually, fixes the race condition in the test.

[Dominaezzz]

I'll push the changes soon.