https://kotlinlang.org logo
Join Slack
Powered by
`ByteArray` on WasmJs not being able to be used a...
# webassembly
m
ByteArray
on WasmJs not being able to be used as a parameter in externally defined interfaces/classes is a huge blocker. Anyone know if there are plans to provide access to the backing 
storage
array that the 
ByteArray
implementation uses?
➕ 2
t
How it should work? With some automatic translation to 
Int8Array
?
m
I care not how it works under the hood, even if I have to call 
ByteArray.unwrap()
which returns a 
JsAny
and is only available from 
wasmJs
source sets. I'd simply like to mitigate the awful amount of array copying being done when bridging the gap between native Js code and Kotlin code.
r
Had similar issues when writing a compression api using JS's compression streams. Automatic translation to int8array would be absolutely amazing, but anything would be better than the current state
m
All I want to be able to do is 
Uint8Array(kotlinByteArray.asInt8Array().buffer)
laugh cry face palm
t
It's already here 😉
m
That copies the ByteArray to a new Uint8Array. It does not instantiate the new Uint8Array view with the ByteArray's backing 
Int8Array.buffer
ByteArray.asInt8Array()
<< copy no 
ByteArray.toInt8Array()
<< copy yes
t
copy no
Do you need shared memory?
m
Yes exactly. many native Js/WasmJs APIs require a 
Uint8Array
to utilize (e.g. Node.js 
stream.Writable.write
only added other 
TypedArray
in version 
18
, or the 
Crypto
fill functions). I do not want to create a new wholly separate instance. I want to create a new view for the underlying 
ArrayBuffer
so native, externally defined functions can manipulate the 
ArrayBuffer
, thus manipulating the 
ByteArray
indices. Currently, WasmJs implementation of 
ByteArray
is just a wrapper around an internal 
WasmJsByteArray
whereby all 
ByteArray
calls are delegated to 
ByteArray.storage
. It is inaccessible.
➕ 1
t
If it's possible to write 
asIntArray()
- we can add it in Kotlin Wrappers
m
Is there a way to obtain access to wasmJs's 
ByteArray.storage
value?
t
Where you see it in sources?
m
t
Looks like access is possible 😉
m
How's that?
And how to mitigate future breakage if the API changes?
t
How's that?
On JS and JVM you can access internal members with suppress
m
Yeah that's fine for an application to do because you control the version of Kotlin, but for a library?
t
We use latest all the time 😉
m
Oof, lol.
Can't put that in a library man, lol.
b
kotlinx-browser provides extensions to copy Typed Arrays back and forth. Unfortunately, Wasm GC arrays and JS TypedArrays are not the same, and are not interchangeable.
m
Is there any way to bridge that with compiler magic or something?
Copy code
internal external sealed interface SomeJsThing {
    @VeryWellNamedAnnotationLol
    fun update(data: ByteArray, offset: Int, len: Int)
}
t
@Matt Nelson could you please describe your use case? Do you send data from JS to Wasm only? Or you have big mutable data, which you want to share for faster computations?
m
My usecase is that I want to pass the 
ByteArray
to externally defined things on WasmJs so that it can do it's thing and mutate, or read the data. e.g. (source set 
jsWasmJsMain
which is 
jsMain
+ 
wasmJsMain
)
Copy code
internal external sealed interface NodeJsFs {
    fun writeSync(fd: Int, buffer: ByteArray, offset: Int, length: Int): Int
}
The above code will not work because 
ByteArray
on 
wasmJs
is not 
JsAny
It forces me to do the following
Copy code
internal external sealed interface NodeJsFs {
    fun writeSync(fd: Int, buffer: Uint8Array, offset: Int, length: Int): Int
}

fun write(buf: ByteArray, offset: Int, len: Int): Int {
    val jsBuf = Buffer.allocUnsafe(len) // <<<< Would like to ELIMINATE
    // copy buf to jsBuf
    return fs.writeSync(someFd, jsBuf, 0, len)
}
Same goes for reading, or virtually any other externally defined Js thing.
t
e.g. (source set 
jsWasmJsMain
which is 
jsMain
+ 
wasmJsMain
)
webMain
?
m
yeah
I name it differently
t
It looks like common type like 
JsByteArray
can help in your case
m
What is that, 
JsByteArray
? Also that's not available from 
commonMain
t
Why do you need it in 
commonMain
?
m
because my thing is multiplatform and supports all targets?
What is a 
JsByteArray
, is that something in the standard library?
t
Helper, which will do 
as
in JS and 
to
in WasmJS will solve your problem
We have such internal helpers
m
You literally cannot do 
as
in wasmJs
t
It's what I wrote
Do you have good naming for such 
as/to
extensions?
e
@Matt Nelson question: even if you had access to the internal 
storage
, what would you be able to do with it? Looks limited as far as I can see.
But I agree with you. Currently, copying a JS 
Uint8Array
to a WebAssembly 
ByteArray
is extremely inefficient.
I'm not sure if there is a way to work with memory pointers directly, but it seems messy in any case.
8 Views
Open in Slack
PreviousNext
Powered by