I am trying to do some threading with K/N. Basical...

# kotlin-native

[David]

I am trying to do some threading with K/N. Basically, I want to thread off some work and call a listener when ready. This code crashes with IllegalStateException. Any suggestions on how to improve?

import kotlin.native.concurrent.*

class KotlinNativeFramework {

    var listener: KotlinNativeFrameworkListener? = null

    fun startWorking() {
        val myInput = DoubleArray(10) { it.toDouble() }.toList()

        val future = Worker.start().execute(TransferMode.SAFE, {
            myInput }) { input ->
            val doSomeWork = DoSomeWork()
            doSomeWork.run(input)
        }

        future.consume {
            result -> listener?.resultsReady(result)
        }
    }
}

interface KotlinNativeFrameworkListener {
    fun resultsReady(result: WorkResult)
}

[olonho]

move producing of myInput to producer lambda

[olonho]

also please read https://github.com/JetBrains/kotlin-native/blob/master/CONCURRENCY.md#workers

[David]

Thanks, I have read the document but not really understood how to solve the situation. I have tried to apply `freeze()`to myInput without success. The problem with a producer lambda is that the input data is created over time, then once in a while I want to take that data and process. The data is not handled at any other place in the code after being sent to the worker, so there should be no risk of concurrent modifications. Also, the amount of data in myInput is rather large so a producer lambda that copies data must be avoided.

[olonho]

so why cannot you just freeze? what is the exact problem you see? frozen object can be liberally passed without giving up ownership. Maybe sharing repo with your code would help us understand how to help you.

[David]

Thanks for quick replies Nikolay! freeze() does help for this error, I apparently had another error “Invalid pointer dequeued from free list” when applying freeze() (in the real code). I have to investigate further to understand how to get this in the example.

[David]

@olonho To take a step back, I have created a simple naive example of how we would traditionally handle sensor data on iOS. Basically, gyro data is coming in on an iOS-thread and sends it to a data processing class. We have now written this data processing class in the KN framework. The KN class is initialized on the main thread but called from the sensor thread, so it is obvious that this should not work. So, what is best practice in this situation? We would like to avoid using the main thread for processing.
See the simple example here:
https://github.com/dtornqvist/kotlin-native-threading-debug

[olonho]

see sample I sketeched here https://github.com/JetBrains/kotlin-native/pull/2135/files#diff-cac804bf85415b6b3e34dd4959d73fff- it has different jobs for different workers. In your example, although it seems gyroSumComputer better be instantiated on the worker’s side

[olonho]

note that worker can have own local data, i.e. global marked with the @ThreadLocal

[olonho]

and so it can get the data from sensor, process it and once you need it - ask on the main thread

[olonho]

somewhat similar behavior is shown in this sample: https://github.com/JetBrains/kotlin-native/blob/ba14e032a2fb5e4ef4f3094baab48c03ec3e900b/samples/videoplayer/src/main/kotlin/DecoderWorker.kt#L328

[David]

Thanks, the DecoderWorker was a helpful example! I have made changes to my code with a @ThreadLocal variable for gyroSumComputer. However, I am getting kotlin.IllegalStateException: Illegal transfer state. The work in execute seems to be done and a result is computed, however the input to consume is null.
https://github.com/dtornqvist/kotlin-native-threading-debug/blob/kotlin-native-worker/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumWorker.kt

[olonho]

your program is not exactly thread safe, that’s why you seeing an exception. currentOutput field is accessible from both inside worker and passed outside. For example following diff would make it work as expected:

diff --git a/MySimpleProject/KotlinNativeFramework/build.gradle b/MySimpleProject/KotlinNativeFramework/build.gradle
index 4dd4e34..2aab1d4 100755
--- a/MySimpleProject/KotlinNativeFramework/build.gradle
+++ b/MySimpleProject/KotlinNativeFramework/build.gradle
@@ -13,23 +13,15 @@ buildscript {
     }
 }
 
-apply plugin: 'kotlin'
-
-repositories {
-    mavenCentral()
-}
-
-dependencies {
-    compile "org.jetbrains.kotlin:kotlin-stdlib"
-}
-
 apply plugin: 'konan'
 
 konan.targets = [
-    'ios_arm64', 'ios_x64'
+    //'ios_arm64', 'ios_x64'
+    'macos_x64'
 ]
         
 konanArtifacts {
-    framework('KotlinNativeFramework')
+    // framework('KotlinNativeFramework')
+    program('KotlinNativeTest')
 }
-        
\ No newline at end of file
+        
diff --git a/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumComputer.kt b/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumComputer.kt
index 63f52fa..37abeca 100755
--- a/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumComputer.kt
+++ b/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumComputer.kt
@@ -9,10 +9,10 @@ class GyroSumComputer {
 
     fun performOperation(input: InputData): OutputData {
         currentOutput = OutputData(currentOutput.xSum + input.x)
-        return currentOutput
+        return OutputData(currentOutput.xSum)
     }
 }
 
 data class OutputData(val xSum: Double)
 
-data class InputData(val x: Double)
\ No newline at end of file
+data class InputData(val x: Double)
diff --git a/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumWorker.kt b/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumWorker.kt
index 9499f72..62f8811 100644
--- a/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumWorker.kt
+++ b/MySimpleProject/KotlinNativeFramework/src/main/kotlin/GyroSumWorker.kt
@@ -10,14 +10,14 @@ class GyroSumWorker {
     private val worker = Worker.start()
 
     init {
-        worker.execute(TransferMode.SAFE, { GyroSumComputer() }) {
-            gyroSumComputer = it
+        worker.execute(TransferMode.SAFE, { null }) {
+            gyroSumComputer = GyroSumComputer()
         }
     }
 
     fun performOperation(input: InputData) {
         input.freeze()
-        worker.execute(TransferMode.SAFE, { input }) {
+        worker.execute(TransferMode.SAFE, { input }) { it ->
             val res = gyroSumComputer?.performOperation(it)
             println("res = " + res) // res is not null
             res
@@ -31,3 +31,8 @@ class GyroSumWorker {
 interface GyroSumWorkerListener {
     fun gyroSumUpdate(outputData: OutputData)
 }
+
+fun main() {
+  val w = GyroSumWorker()
+  println(w.performOperation(InputData(1.0)))
+}

[David]

Fixed the problem! gyroSumComputer stored it’s result internally and then returned it. As gyroSumComputer is @ThreadLocal this didn’t work…

[David]

Thanks! 🙂 So, now this approach is working. However, it requires that all sensor data is forwarded on the main thread. This might not be desirable on a mobile application. Is there a way around that?

[olonho]

you can communicate with this worker from any thread, not only main

[olonho]

Worker object is frozen, and can be liberally used from any execution context

[David]

Ok, sounds great! However, I tried to go back to having the sensor data come in on a separate thread. The app then crashes with SIGABRT. See my changes here:
https://github.com/dtornqvist/kotlin-native-threading-debug/commit/d7baa05e261f33ec6df84e90694c414e23695278