https://kotlinlang.org logo
Join Slack
Powered by
Can someone please describe to me, or link to some...
# kotlin-native
a
Can someone please describe to me, or link to some detailed documentation, about how 
usePinned
works in terms of C code? I am wondering why it is necessary, or perhaps if I am misusing it. If I am misusing it, then I would like to know what situations it should be used in. The way I have used it is similar to this:
Copy code
// declare a 256 byte array to pass to a C function as an output parameter
UByteArray(SHA256_DIGEST_LENGTH).usePinned { signatureBuf ->
    c_func(signatureBuf.addressOf(0))
}
My understanding of the equivalent of this in C:
Copy code
unsigned char signatureBuf[SHA256_DIGEST_LENGTH];
c_func(signatureBuf);
c_func
computes some value and stores it in the address provided by its parameter. This is a simplistic example and obv not good code. Just want to show my understanding of it.
l
Modern Garbage Collectors use an idea called ‘generational garbage collection’ (https://wiki.c2.com/?GenerationalGarbageCollection). This means that the garbage collector can move an Object’s location in memory. In Kotlin, this is fine, since the Kotlin runtime can keep track of when the variable changes locations. C, however, is not aware of the concept of a Garbage Collector. It assumes that memory doesn’t move automatically.
a
Ah, so it is a safety mechanism then? That seems to mean I am using it correctly. I thought it guaranteed that the memory wouldn't change, but I did not understand how/why it could :) thank you!
l
To bridge this gap, we ‘pin’ the object, which tells Kotlin that the variable should never be moved by the Garbage Collector. When we hand 
singatureBuf.addressOf(0)
to 
c_func
, we know the variable won’t mysteriously move. When it’s no longer needed by C, we unpin the memory, so Kotlin can move it as needed.
a
Thank you, Landry! I will need to do some further digging. I have attempted to translate some basic C code to Kotlin native, and I am getting incorrect results, but I don't understand why. I thought this might be the culprit, but it seems not.
l
I believe it also stops the GC from collecting the object. Keep in mind also that c_func may save the pointer. If it does this, don’t use 
usePinned
, as this will unpin the variable when the lambda completes. If some C code uses the saved pointer, it’s possible the variable moved and the pointer points to invalid memory.
a
What would be the better way to do it then?
I don't think the C code is stateful, if that's what you mean.
e
pinning is a feature in every FFI from a language with generational GC to C; most of the same principles that you have to deal with in JNI apply here as well. the big difference is handling the pins on the Kotlin side rather than on the C side
l
If you know that the pointer is used by C after c_func completes, you should use 
pin()
and 
unpin()
. If you know for sure that C will never need the pointer after c_func completes, however, then 
usePinned
is fine.
e
making calls to C is unsafe in that you have to make assumptions about its behavior that you can't verify through the type system (you have to rely on documented behavior or perhaps the source)
a
If you want to know, it's OpenSSL's 
HMAC
function. I don't believe it carries any state, so once it returns I can do what I want with the variables.
l
I would assume that’s safe once it completes. What issues are you seeing?
e
that sounds like a reasonable assumption, the only memory it should mutate is its output
a
I'm still investigating, but I'm trying to compute the SHA256 HMAC of a string, and the signature computed by my Kotlin native code is different from the signature computed by my C code, which computes the correct signature. I thought it might be unsafe memory access, but it seems not. This code does take place inside a coroutine (ktor). Are there known issues with that?
l
Have you tried running this outside of a coroutine and seeing if the same code runs correctly? I’d write a unit test.
a
Yeah, that's what I was planning to try next. I'll let you know what happens.
l
Off the top of my head, I’d be careful about charsets. It’s possible your Kotlin code is converting the String using UTF-16 to be compatible with Java. If that’s the case, the String would be wrong, since C uses ASCII.
UTF8 should be compatible.
a
I believe it is UTF-8. The string is retrieved from an HTTP request, which appears to default to UTF-8 in ktor
e
K/N uses UTF-8 when interoperating with C strings
a
That's very helpful to know. I also found this code inside ktor:
Copy code
public suspend inline fun ApplicationCall.receiveText(): String {
    val charset = try {
        request.contentCharset() ?: Charsets.UTF_8
    } catch (cause: BadContentTypeFormatException) {
        throw BadRequestException("Illegal Content-Type format: ${request.headers[HttpHeaders.ContentType]}", cause)
    }
    return receiveChannel().readRemaining().readText(charset)
}
e
what I would check first is whether your C and Kotlin code are passing the same input - possibly the data length is including/excluding a trailing 
'\0'
differently
a
I will check that as well, good idea. 
HMAC
does take a length parameter, so I don't think so, but I'll double check.
Using the debugger inside a coroutine has not really worked, but with the unit test it will be easier.
Unit test shows that it's my code that's off. Investigating where inputs and outputs start to differ from my C code.
16 Views
Open in Slack
PreviousNext
Powered by