https://kotlinlang.org logo
#feed
Title
# feed
o

Oleg Yukhnevich

03/02/2023, 9:22 AM
Hey! Last weeks I’ve been working on researching multiplatform possibilities to provide access to native libraries (https://youtrack.jetbrains.com/issue/KT-39144), mainly for future use in https://github.com/whyoleg/cryptography-kotlin (which BTW will have first release soon) https://twitter.com/why_oleg/status/1631223213110829058 And here are some results: • kotlinx.cinterop inspired common API • Works on Native, JVM(JNI/Panama), Android(JNI), WASM, JS • manual bindings to part of libcrypto API Next: automated bindings generation, API completness Code can be found here: https://github.com/whyoleg/ffi-kotlin Does someone is interested to contribute? :) P.S. in thread there are screenshots for comparison of user-facing APIs for `ffi-kotlin`(fully common code) and `kotlinx.cinterop`(native only)
🚀 2
👍🏻 1
👀 4
😮 2
👍 6
❤️ 6
kotlinx.cinterop
ffi-kotlin
s

svyatoslav.scherbina

03/08/2023, 3:11 PM
Hi!
kotlinx.cinterop inspired common API
While kotlinx.cinterop is a fine baseline in a sense (because it is known to be able to represent many of the required concepts, and the unsupported concepts are more or less known), it might make sense to start from scratch, just to avoid being affected by the current (arguable and opinionated) design and to properly take platform specifics into account.
Panama
Panama includes https://openjdk.org/jeps/424, which somewhat restricts operations with unsafe memory segments (see “Unsafe memory segments” and “Safety” sections). In particular, memory access operations on arbitrary
CPointer
can only be implemented using restricted FFM API. Do you use FFM API? How do you overcome those restrictions?
• requires compiler to build JNI library - how to set up it properly to support different OSs?
• is it possible to use K/N provided clang with cross-compile possibilities here?
(from https://github.com/whyoleg/ffi-kotlin#jvmandroidjni) You can start with
run_konan clang clang $target
from the Kotlin/Native distribution, e.g. this
Copy code
~/.konan/kotlin-native-prebuilt-macos-x86_64-1.8.10/bin/run_konan clang clang linux_x64 -c 1.c
compiles
1.c
to
1.o
(and prints the whole
clang
command as well). Doesn’t play nicely with a linker, but you can fix this by adding
-fuse-ld=lld
compiler flag:
Copy code
~/.konan/kotlin-native-prebuilt-macos-x86_64-1.8.10/bin/run_konan clang clang linux_x64 1.c -fuse-ld=lld
o

Oleg Yukhnevich

03/08/2023, 4:16 PM
@svyatoslav.scherbina 1. can you explain more about what’s wrong with cinterop API, and what would you change if something? Overall only working with strings and structs is somehow different per platforms, while everything else is super similar and fit well into cinterop api 2. Panama implementation uses https://openjdk.org/jeps/434 (jep for jdk 20 with some changes to API) and yes I use FFM API, but Im not sure of what restrictions specifically you are talking, can you give an example? 3. Yeah, I tried using run_conan, but had issues with linker - so I will try with what you propose, thx! 4. If using konan, is it possible to download it from gradle without adding native targets, f.e. if there is no konan on user PC, and ffi used only for jvm?
s

svyatoslav.scherbina

03/08/2023, 5:41 PM
can you explain more about what’s wrong with cinterop API, and what would you change if something? Overall only working with strings and structs is somehow different per platforms, while everything else is super similar and fit well into cinterop api
Overall, cinterop was designed 1. with language limitations in mind, for example, lack of value types, implicit conversions etc. 2. driven by really peculiar examples, like POSIX. 3. with attempts on making simple examples shorter, without real scalability here; this also made it inconsistent to some extent. 4. targeting people who know well both C and Kotlin 5. with attempts to be able to represent C concepts as correctly as possible (to the degree where it became unnatural to both C and Kotlin) cinterop introduces really convoluted concepts (often to workaround 1, like CValuesRef). As a result, cinterop is pretty hard to use. It is not intuitive: with a specific goal in mind (like use a C library in Kotlin, having C examples for it), one might have a hard time figuring out how to achieve it. It cinterop doesn’t provide a way to write some of the bindings manually, which could have made things simpler (by allowing auto-generator to be imprecise or non-universal in some cases). I believe some of these things can be avoided even in a third-party project.
I use FFM API, but Im not sure of what restrictions specifically you are talking, can you give an example?
Quote from the JEP:
The FFM API can also produce unsafe segments, that is, memory segments whose spatial and temporal bounds are user-provided and cannot be verified by the Java runtime (see
MemorySegment::ofAddress
).
The unsafe methods in the FFM API do not pose the same risks as JNI functions; they cannot, e.g., change the values of
final
fields in Java objects. On the other hand, the unsafe methods in the FFM API are easy to call from Java code. For this reason, the use of unsafe methods in the FFM API is restricted: Their use is permitted but, by default, every such use causes a warning to be issued at run time.
CPointer
is basically an unsafe segment (e.g. when it is returned from a C function).
if using konan, is it possible to download it from gradle without adding native targets, f.e. if there is no konan on user PC, and ffi used only for jvm?
I’m not aware of any stable easy approach for this.
o

Oleg Yukhnevich

03/08/2023, 7:20 PM
1. Can we chat/make c call sometime to discuss ideas on how it will be possible to improve ffi? Dart BTW has similar API 2. For now my idea was to be able to easy migrate native cinterop -> multiplatform. though even now it differs in some places, but improving overall API is always good 3. Manual bindings for multiplatform will be even harder, as there are much more things involved 4. When CPointer returned from function, it can be exposed via Unbounded(Unsafe) segment, and you can do with it what you want. I haven’t seen any warnings during testing. Jextract uses the same approach for such declarations, so Im not sure, that there is some problem here, or at least I don’t understand it yet :)
s

svyatoslav.scherbina

03/10/2023, 11:50 AM
Can we chat/make c call sometime to discuss ideas on how it will be possible to improve ffi? Dart BTW has similar API
Yes, sure. I’ve sent you a DM.
Manual bindings for multiplatform will be even harder, as there are much more things involved
A compiler plugin might help.
When CPointer returned from function, it can be exposed via Unbounded(Unsafe) segment,
Exactly. And the JEP says that unsafe segments are restricted and trigger a warning.
11 Views