Is there any kotlin low level process library supporting all native platforms? (Same way ktor networ...

# multiplatform

[Nathan Fallet]

Is there any kotlin low level process library supporting all native platforms? (Same way ktor network supports low level sockets (not websockets) for all platforms). Asking to know if I should implement an expect/actual to spawn a process for each platform or use any existing official Kotlin Process library.

[Nathan Fallet]

This is what I made so far with custom expect/actual: https://pastebin.com/EeYLhr8n

[ephemient]

you must _exit() and not exit() on the fork child, and also I would not fork() from Kotlin/Native - it creates a new process with one thread, so it's missing everything else the Kotlin/Native runtime expects to exist, such as the GC thread, which could lead to some difficult-to-debug issues. consider using posix_spawn instead

[loke]

To handle errors correctly, it's a bit trickier on Linux, since it doesn't have a good implementation of posix_spawn. Here's my implementation, which is think is mostly correct. https://codeberg.org/loke/array/src/branch/master/array/src/linuxMain/kotlin/com/dhsdevelopments/kap/io.linux.kt#L361

[loke]

It's also not 100% tread-safe because Kotlin Multiplatform does not expose dup3() and pipe2()

[ephemient]

if you're going with this approach, I'd do all the allocation before fork, to reduce the chance of triggering or waiting for GC which will not work inside the child. even then, it's still not safe because you don't know which system and library calls the rest of the Kotlin runtime will make

[ephemient]

https://pubs.opengroup.org/onlinepubs/9799919799/functions/fork.html

A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, the application shall ensure that the child process only executes async-signal-safe operations until such time as one of the exec functions is successful.

async signal-safe operations being a key concept that is not guaranteed in Kotlin

[ephemient]

if posix_spawn isn't reliable, I'd write a similar wrapper in C and use cinterop to call it from Kotlin

[loke]

The problem is that you still need to do the mess with a pipe because otherwise there is no way to reliably detect how the exec call failed.

[loke]

But you're right about the GC stuff. That's an issue.

[loke]

One would think vfork would address this, since it blocks the parent process, but that also is not reliable since the standard says the child is not allowed to modify any data, which obviously a GC will. Is there a way to block GC calls?

[loke]

And cinterop isn't going to help either, will it? Another thread can still call gc.

[ephemient]

move the whole fork-pipe-exec into C, and it will be the only thing running in the child, which is fine. it won't touch GC or any other Kotlin runtime state

[loke]

And use vfork?

[ephemient]

doesn't matter if you use fork or vfork at that point, the limitations of both are easier to handle in C

[Nathan Fallet]

It seems to be a complex subject, maybe I should wait for kotlinx-io or a similar library to handle it so that I’m sure it was thought by experts on that subject (which I’m not)

[Oleg Yukhnevich]

Have you seen/tried https://klibs.io/project/05nelsonm/kmp-process?
Is this something you are looking for?

[loke]

@Oleg Yukhnevich seems like their implementation is very similar to mine (even down to the pipe tricks).

[loke]

Any arguments @ephemient raised with mine (and they are valid arguments) should be raised against this.

[loke]

It's probably as good as it gets without help from the Kotlin runtime.

[Nathan Fallet]

@Oleg Yukhnevich This one looks promising, I’ll have a look. As you know more or less Kotlin’s team plan, do you know if it will implemented in an official package like Kotlinx-io or if using third party project like the one you mentioned is the long term and production ready way?

[Oleg Yukhnevich]

do you know if it will implemented in an official package

I'm not aware of any plans at this moment. Feel free to check the roadmap; the focus on libraries is a bit different at this moment.
Also, I haven't really found an issue in kotlinx-io or in the kotlin YT about support for process functionality. So it's not even clear how popular it is 🙂

using third party project like the one you mentioned is the long term and production ready way?

It's up to you to decide. I haven't really used the library, so I can't judge it. I just recently discovered it.