https://kotlinlang.org logo
Join Slack
Powered by
if I have a mutable object that is "owned" by a wo...
# kotlin-native
k
if I have a mutable object that is "owned" by a worker, is there a way to reference it via the lambda passed to 
execute
?
or is there another preferred method for operating on a mutable object asynchronously (i.e., passing a message to a thread)?
maybe I should just use GCD?
k
How is the object owned by the worker?
k
it is always created by the lambda passed to execute
k
What happens when the lambda finishes?
k
it's a class member so the reference lives on
the worker is also a class member
k
Code will probably make this easier.
The class is being touched by more than one thread, which means it is frozen, as is the “mutable” state
k
Copy code
private val _worker = lazy { Worker.start() }
    private val worker by _worker
    private val buffer by lazy {
        check(_worker.isInitialized() && Worker.current == worker)
        val buffer = MemoryMappedRingBuffer(1024, "")
        buffer.open()
        buffer
    }

    actual fun log(entry: LogEntry) {
        worker.execute(TransferMode.SAFE, { entry.freeze() }) {
            buffer.write(entry.format())
        }
    }

    actual fun log(level: LogLevel, subsystem: Subsystem, message: String, context: LogContext?) {
        worker.execute(TransferMode.SAFE, { LogEntry(level, subsystem, message, context) }) { entry ->
            buffer.write(entry.format())
        }
    }
k
buffer
is created lazily by the lambda, but it’s a member of a class that is presumably referenced by other threads. 
buffer
needs to be thread local or frozen.
a
Lambda must be non-capturing, is not it?
k
it captures 
buffer
it sounds like GCD is a better solution in this case
a
So AFAIK it should not compile
k
that's correct
a
But you can pass a frozen lambda and call it from worker
k
unfortunately that would not support this use case
a
The buffer should be freezable
k
Using gcd to avoid the concurrency rules won't end well
a
E.g. based on AtomicReference or be thread local
k
access to the buffer should be serialized by the concurrency primitives. GCD makes that simple. it sounds like Worker does not.
a
Nope, it's not enough. You could use posix mutex same way. But the point is that reference counting is not atomic.
👆 1
You will end up with unexpected segfaults
k
"GCD makes that simple" that's not true. Happy to be wrong though. Post code
k
i suppose given that GCD doesn't guarantee a thread of execution I am going to run into exceptions on mutation
i feel like i am fighting my tools here
k
Even if it did, you’re missing the problem
I think you’re trying to avoid using the tools the way that they were designed. You can’t have buffer be visible by multiple threads unless it’s frozen
k
i don't want it to be visible to multiple threads
i want just one thread to be able to mutate it, by sending messages from other threads
k
Buffer can be thread local, and the single worker will limit access to a single thread
k
I believe the one to one worker to thread is not guaranteed?
k
Yeah. Keep buffer thread local and only let the worker access it. Create an object, have buffer be a filed, and annotate it @ThreadLocal
k
i'd like to have a way to limit access to a single thread than annotate @ThreadLocal and keep my fingers crossed 😛
if worker ends up executing on a thread pool, with that annotation, very bad things would happen
k
Worker is one thread
k
some really smart guy wrote this post: https://medium.com/@kpgalligan/kotlin-native-stranger-threads-ep-1-1ccccdfe0c99, with this footnote: The docs are pretty clear that you shouldn’t rely on that in the future as it may change, but for the foreseeable future, 1 Worker gets one thread. 😛
it just makes me uneasy as even the API docs are vague about the concurrency details
k
Temporally vague maybe, but how they work today is clear. One worker, one thread. All jobs processed in order
Once multithreaded coroutines happen, if there are thread pools or whatever, you'll need a different plan (if you use them)
k
well, that's something. let's hope if they ever change that that they make it really clear, and that the implied contract between Worker and @ThreadLocal doesn't just blow up
it does seem odd to me, though, that there doesn't seem to be an API that's designed for this purpose
the joys of the bleeding edge I guess
k
There's a lot of code that currently relies on that contract, so it won't just go away
What purpose do you want an API for?
k
asynchronous, serialized access to a mutable object
clear ownership
a
My dreams
k
I mean, that's what thread local is doing. What would the API look like? I'm currently working on a talk about libraries, btw
k
i'd have to give that some thought, but Worker + @ThreadLocal is definitely an awkward API, and it seems more that it happens to work than it was designed for the purpose
k
I'd disagree on the last. It's designed to work that way. Just not a lot of clear docs and samples
k
perhaps. i definitely would not have designed it that way.
k
You'd need to decide are you just modding data? Do you want data returned? Is that returned data frozen?
Do you wait for returned data or does it have a callback or suspend? It's not a simple thing to design necessarily
k
should annotating it @ThreadLocal allow it to compile?
it doesn't, just wondering if there's another step or something
k
I can't see the code or the error messages, which makes things difficult
a
BTW, what are you going to do with the buffer? Are there any readers?
k
it's the same code as above except I added @ThreadLocal to buffer
kotlin.native.concurrent.Worker.execute must take an unbound, non-capturing function or lambda
and yes, the buffer can be read
a
Read by another thread?
k
just the 1
👍 1
a
threadLocal is only applicable to a top level variable with backing field or an object
Pass a frozen lambda to the worker and invoke it from there. Access your top level ThreadLocal buffer from that lambda.
k
Thanks, will give that a shot in a while
k
If that doesn’t work I’d suggest a small project and post it somewhere. Verbal code diffs aren’t super productive.
k
🍻
I was able to get this working but it requires extra object allocations and ends up being a big hack
k
If you need TransferMode.UNSAFE to compile, you’re probably going to have serious issues
k
i need it to run. it compiles and throws an exception otherwise
k
Yeah, because you’re doing something that will blow up
k
because the class is referencing the 
buffer
k
It’s a race condition
k
it's actually not
k
OK. Good luck
k
but I can't see a different way to do the same thing
it's maddening
k
To stress the point made multiple times, non-frozen state has non-atomic reference counting
k
oh lord
k
What’s MemoryMappedRingBuffer?
k
at the moment it's an empty class that doesn't do anything
eventually it will contain a ring buffer that's backed my a memory mapped file
it seems like such an obvious use case for these APIs that I feel like I am missing something obvious to implement it correctly
a
I would never play with UNSAFE mode. Even if you manage it to work (by synchronizing access to your buffer), it still will crash on 100th or 1000th run. If you need to access an object from worker then use ThreadLocal or freeze it. No other options.
k
i was not able to get 
@ThreadLocal
to compile
k
I fixed it and will send
Copy code
@ThreadLocal
internal object TheCache {
    val buffer:MemoryMappedRingBuffer = MemoryMappedRingBuffer(1024, "")

    init {
        buffer.open()
    }
}

object LogEngine {
    private val worker = Worker.start()

    fun log(entry: LogEntry){
        println("log called in main ${NSThread.isMainThread()}")
        worker.execute(TransferMode.SAFE, { entry.freeze() }) {
            TheCache.buffer.write(it.format())
        }
    }
    fun log(){
        worker.execute(TransferMode.SAFE, { LogEntry("bunch of stuff") }) {
            TheCache.buffer.write(it.format())
        }
    }

    fun flush() = worker.execute(TransferMode.SAFE, {}) { Worker.current?.processQueue() }.result
}

class MemoryMappedRingBuffer(i:Int, s:String){
    fun open(){}
    fun write(log:String){
        println("Logging: $log, is in main thread ${NSThread.isMainThread()}, am I frozen? ${isFrozen}")
    }
}

data class LogEntry(val s:String){
    fun format():String = "LogEntry: $s"
}
In swift…
Copy code
let logEngine = LogEngine()
        logEngine.log(entry: LogEntry(s: "Hey we're loggin'"))
You’ll see this in ios log…
Copy code
log called in main true
Logging: LogEntry: Hey we're loggin', is in main thread false, am I frozen? false
k
so it looks like the key is to move the buffer class member to a global
k
You may want 
val buffer:MemoryMappedRingBuffer
to be lazy, so you don’t start one per thread, but the rest of the lazy things are just noise.
Well, that’s one way of doing it. It’s far more important to understand the rules and structures.
k
i didn't want to start the worker until log is called. that's why I used the other lazy
k
For what you’re trying to do, you need buffer thread local. Putting it in an object and making that thread local is a simple way of doing that
Why?
What’s the difference?
Well, whatever.
k
creating and starting a thread isn't free
k
Nope
k
thank you for taking the time to assist with this!
k
I’d guess that checking for lazy on every call isn’t either, but hey
Sure! Just needed to bang it out. It’s simple in my mind, but I’ve been doing it for a while
k
yeah. the actual solution ends up being a workaround. it's not normally how you would implement this.
looks like a companion object works as well
k
Yeah, makes sense
k
k
I shouldn’t ask, but what’s with the guard?
k
it verifies there's only ever one instance of the companion
k
You don’t think that’s a little extra complexity you don’t need?
Anyway, never mind
k
not at all. the added complexity is very small, and it prevents what would be a large issue if it happened
k
Global val’s are only visible to the main thread, so I’m struggling to understand how this functions, but OK
k
i have unit tests and verified that it executes on a non-main thread
k
“global variables, unless specially marked, can be only accessed from the main thread”
I would double check that
k
hmm
Current thread: <NSThread: 0x7facf9c013a0>{number = 2, name = (null)}, main thread: <NSThread: 0x7fad09c064a0>{number = 1, name = (null)}
perhaps because it's an AtomicInt?
k
Interesting. you can see AtomicInt val’s. Learn something new. Not sure why, but need to move on to other stuff…
k
👍 🍻
2 Views
Open in Slack
PreviousNext
Powered by