https://kotlinlang.org logo
#kotlin-native
Title
# kotlin-native
o

Oleg Yukhnevich

02/07/2023, 2:40 PM
Hey, is there someone who can help with cinterop and pointers ? 🙂 there is external declaration:
fun EVP_MAC_CTX_set_params(ctx: CValuesRef<EVP_MAC_CTX>?, params: CValuesRef<OSSL_PARAM>?): <http://kotlin.Int|kotlin.Int>
(params is a reference to an array) example of usage in C is here https://www.openssl.org/docs/man3.0/man3/EVP_MAC_init.html (in the end of the page). I’ve tried to something like this, and it compiles and works without errors on runtime, but looks like it doesn’t work as expected, as those
params
are not visible by `openssl`…
Copy code
val mac = EVP_MAC_fetch(null, "HMAC", null)
val context = EVP_MAC_CTX_new(mac)

val params = allocArrayOf(
    OSSL_PARAM_construct_utf8_string("digest", "SHA256".cstr, 0).ptr,
    OSSL_PARAM_construct_end().ptr
)
check(EVP_MAC_CTX_set_params(context, params[0]) == 1) // works, returns 1

check(EVP_MAC_CTX_get_mac_size(context) > 0) // fails, but should not
The error is
error:0300009F:digital envelope routines::message digest is null
- so looks like it don’t see
digest
key in
params
array for some reason Where I could dig to understand what I do wrong? Thx in advance! 🙂
e

ephemient

02/07/2023, 2:50 PM
the
cstr
doesn't live long enough.
OSSL_PARAM_construct_utf8_string
doesn't copy it, it holds onto the pointer
o

Oleg Yukhnevich

02/07/2023, 3:10 PM
any idea on how to overcome this? I’ve tried pinning, but looks like it doesn’t help here
I’ve added such declaration to def file (the same should work for
EVP_MAC_CTX_set_params
):
Copy code
static inline int hmac_init(EVP_MAC_CTX *ctx, const unsigned char *key, size_t keylen, const char *digest_name) {
    OSSL_PARAM params[4], *p = params;
    *p++ = OSSL_PARAM_construct_utf8_string("digest", digest_name, 0);
    *p = OSSL_PARAM_construct_end();
    return EVP_MAC_init(ctx, key, keylen, params);
}
and using like:
Copy code
hmac_init(context, key.asUByteArray().refTo(0), key.size.convert(), hashAlgorithm)
it works as expected While I can leave with it, I literally don’t understand, why it doesn’t work with kotlin constructions…
n

napperley

02/07/2023, 10:54 PM
Every C library has a different way of dealing with "fake" strings. Have you run into design issues with the OpenSSL API where its string handling is all over the place?
How does the OpenSSL library handle memory management (incl with strings)?
o

Oleg Yukhnevich

02/09/2023, 10:57 AM
TBH, I don’t know 🙂 As Im not really a good C developer (or even not at all a C developer)
And I found one more thing, that is hard to understand the issue is that while this code works as expected:
Copy code
val params = allocArrayOf(
  OSSL_PARAM_construct_utf8_string("digest".cstr.ptr, hashAlgorithm.cstr.ptr, 0).ptr,
  OSSL_PARAM_construct_end().ptr
)
EVP_MAC_init(context, key.asUByteArray().refTo(0), key.size.convert(), params[0])
But in some conditions (looks like when GC works, or anything else) this code starts to fail with seg fault So I tried once again using C code in def file:
Copy code
static inline int EVP_MAC_init_HMAC(EVP_MAC_CTX *ctx, const unsigned char *key, size_t keylen, const char *digest_name) {
     OSSL_PARAM params[2], *p = params;
     *p++ = OSSL_PARAM_construct_utf8_string("digest", digest_name, 0);
     *p = OSSL_PARAM_construct_end();
     return EVP_MAC_init(ctx, key, keylen, params);
 }
and on call site:
Copy code
EVP_MAC_init_HMAC(context, key.asUByteArray().refTo(0), key.size.convert(), hashAlgorithm)
This way it works all time. (and look, here we pass
hashAlgorithm: String
as plain string and it works as expected for reference commit with this changes: https://github.com/whyoleg/kcwrapper/commit/93dec2fbbb4b74499684b1addce5b92519655c0d So I would think that is some issue in K/N interop mappings generation or I’m missing some obvious thing. @svyatoslav.scherbina may be you can somehow help with understanding: is it an issue on my side, or inside K/N interop and I should fill an issue?
s

svyatoslav.scherbina

02/09/2023, 12:43 PM
Copy code
return EVP_MAC_init(ctx, key, keylen, params);
So, in C version you pass the pointer to the
params
array, while in Kotlin
Copy code
EVP_MAC_init(context, key.asUByteArray().refTo(0), key.size.convert(), params[0])
params[0]
is not a pointer to the array, but the first value instead, i.e.
OSSL_PARAM_construct_utf8_string("digest".cstr.ptr, hashAlgorithm.cstr.ptr, 0).ptr
. Please try replacing
params[0]
with
params
to make it equivalent to C.
o

Oleg Yukhnevich

02/09/2023, 12:53 PM
I’ve tried passing
params.reinterpret()
and just casting (because types are different), but in this case it doesn’t work at all (init returns 0, means failure) I will point one time, that when passing
params[0]
code works, but under stress test (which create a lot of keys and then try to sign with them) it starts to FAIL with seg fault at random moment specifically in call to
EVP_MAC_init
. while using C code in def - everything works For context •
OSSL_PARAM_construct_utf8_string
returns
CValue<OSSL_PARAM>
(https://www.openssl.org/docs/man3.0/man3/OSSL_PARAM_construct_utf8_string.html) •
params
in kotlin is of type
CPointer<CPointerVarOf<CPointer<OSSL_PARAM>>>
• EVP_MAC_init accepts for params
CValuesRef<OSSL_PARAM>
(in openssl header it’s OSSL_PARAM[], https://www.openssl.org/docs/man3.0/man3/EVP_MAC_init.html)
s

svyatoslav.scherbina

02/09/2023, 2:50 PM
Your C code creates an array of two struct values, and passes the pointer to that array (i.e. to the first element) to
EVP_MAC_init
. Your Kotlin code creates an array of two pointers to struct values, and passes the first of them to
EVP_MAC_init
. So, as a result,
EVP_MAC_init
gets a pointer to
OSSL_PARAM_construct_utf8_string("digest", hashAlgorithm.cstr.ptr, 0)
, followed by arbitrary garbage instead of
OSSL_PARAM_construct_end()
. Sometimes it gets lucky, and that garbage works as a terminator, sometimes not. So, you need to start with
val params = allocArray<OSSL_PARAM>(2)
, initialize the elements properly and pass
params
to
EVP_MAC_init
. Or, alternatively, use
val params = createValues<OSSL_PARAM>(2) { ... }
.
o

Oleg Yukhnevich

02/09/2023, 3:17 PM
Thx for help! I’ve tried already to do something like this, but the problem is, that I wanted to use
OSSL_PARAM_construct_utf8_string
declaration from openssl, and not construct structs by my self (error prone) And so I haven’t found any information/documentation about how to allocate array of structs, and then put in it already created struct values… But! Looks like I found a way (after several tries on finding what I need):
Copy code
val params = allocArray<OSSL_PARAM>(2)
OSSL_PARAM_construct_utf8_string("digest".cstr.ptr, hashAlgorithm.cstr.ptr, 0).place(params[0].ptr)
OSSL_PARAM_construct_end().place(params[1].ptr)
EVP_MAC_init(context, key.asUByteArray().refTo(0), key.size.convert(), params)
And this works as expected! Big thanks for help! P.S. I think, that API or at least documentation around creating arrays of structs could be improved 🙂
s

svyatoslav.scherbina

02/09/2023, 3:40 PM
Yes, that should work.
P.S. I think, that API or at least documentation around creating arrays of structs could be improved 🙂
That is totally correct 🙂
o

Oleg Yukhnevich

02/09/2023, 3:43 PM
Should I fill an issue on youtrack for documentation or even minimal POC API for this?
s

svyatoslav.scherbina

02/09/2023, 3:51 PM
Yes, that would be helpful, thank you!
o

Oleg Yukhnevich

02/09/2023, 4:10 PM
BTW, while I understand, that may be it’s not strictly your area @svyatoslav.scherbina but, is there any plans to release cinterop for JVM/WASM ? I know, that JVM flavor exists to power cinterop clang bindings, but no doc for it of course :) Some time in a future I will want to port those openssl bindings to JVM (via jni or panama) And so I will be interested even on working to integrate cinterop with JVM similar to how we have it for K/N (I just like hard interesting tasks, hah)
s

svyatoslav.scherbina

02/10/2023, 11:04 AM
is there any plans to release cinterop for JVM/WASM ?
We have no particular plans for this. As you probably know, we have a YouTrack issue for this feature request: https://youtrack.jetbrains.com/issue/KT-39144/Generate-C-bindings-for-all-platforms-through-cinterop (just adding it here for visibility).
I know, that JVM flavor exists to power cinterop clang bindings, but no doc for it of course :)
Missing docs is not the main problem here. There are a lot of complicated integration tasks to be solved before this can become useful. Also, even the existing basic implementation for JVM is somewhat behind. And that implementation we have should probably be ported to panama.
o

Oleg Yukhnevich

02/10/2023, 11:21 AM
We have no particular plans for this.
okay.. yes, I know about an issue, still thought that ‘speculation’ talks are better to perform in slack to not pollute an issue discussion 🙂
And that implementation we have should probably be ported to panama.
just wanted to mention, in ideal case I would think that there should be a choice: use JNI or use Panama, as there is no Panama support in Android. So to be able to build 2 different JARs (or klibs?) or multi-release JAR will be useful
There are a lot of complicated integration tasks to be solved before this can become useful. Also, even the existing basic implementation for JVM is somewhat behind
Yeah, I understand, that it’s not easy… still, if implementation of something like this will be started some day, i’m willing to help!
s

svyatoslav.scherbina

02/10/2023, 1:55 PM
Thank you!
o

Oleg Yukhnevich

03/02/2023, 9:25 AM
Hey @svyatoslav.scherbina, regarding cinterop for JVM (and not only) I’ve done some research on multiplatform cinterop - results are here https://kotlinlang.slack.com/archives/C0BJ0GTE2/p1677748948814649
89 Views