does kotlinx-io allow to manipulate byte buffer ou...

# io

[alexandre mommers]

does kotlinx-io allow to manipulate byte buffer outside JVM heap ? for example to send it to a C library ?

[Filipp Zhinkin]

No, currently kotlinx-io `Buffer`s are backed by byte arrays only.
As an option, you can access a backing array via unsafe API and then pin it inside the C function by means of JNI.

There were some efforts to support DirectByteBuffer-backed `Buffer`s, but we paused it as it didn't gave any immediate benefits for use cases that were considered.
If you have any scenarios where direct byte buffers are the only way to implement some IO operations efficiently, I'd appreciate it if you can share them (so we can reconsider DBB-support).

[alexandre mommers]

I'm working on multiplatform libraries that enable Kotlin to interact with the GPU API (check out my latest project: https://github.com/wgpu4k/wgpu4k).
Transferring data to the GPU can be achieved using ByteArray to Native Buffer, but the double allocation overhead becomes significant as data size increases.

Currently, many Kotlin rendering libraries implement their own buffer solutions. Perhaps there's an opportunity to collaborate and create a more unified approach.

It's important to note that Java ByteBuffer has a size limitation of 2GB, which is insufficient for large machine learning models.
To allocate native buffers, I use JNA on Android and the new Panama API introduced in JDK 22 on the JVM.