https://kotlinlang.org logo
n

napperley

04/08/2021, 2:37 AM
Does Kotlin Native have any facilities available that make it easier to manage buffers? Below is a code snippet that combines image files into a single file:
Copy code
internal actual fun combineImageFiles(dirPath: String) {
    val appendBinaryMode = "ab"
    val readBinaryMode = "rb"
    val arena = Arena()
    val entries = listDirectory(dirPath).sortedWith(ImageNameComparator())
    val videoFile = fopen("$dirPath/video.mjpg", appendBinaryMode)
    entries.forEach { e ->
        val dataSize = fileSize("$dirPath/$e")
        val tmpFile = fopen("$dirPath/$e", readBinaryMode)
        val data = arena.allocArray<ByteVar>(dataSize.toInt())
        fread(__n = 1, __stream = tmpFile, __ptr = data, __size = dataSize)
        fclose(tmpFile)
        fwrite(__n = 1, __s = videoFile, __ptr = data, __size = dataSize)
    }
    arena.clear()
    fclose(videoFile)
    Unit
}
This snippet works however it is using multiple buffers which isn't ideal. If there are a lot of image files being handled then the memory usage would become excessive.
d

Dominaezzz

04/08/2021, 8:11 AM
You don't need to
allocArray
here, you can just use a
ByteArray
and temporarily pin it.
You can create a fix sized array like
ByteArray(4096)
and have more frequent calls to
fread
and
fwrite
when the file is big. This is how JVM
InputStream.copyTo(OutputStream)
does it.
m

mbonnin

04/08/2021, 10:56 AM
Could okio help there? It is multiplatform and the latest SNAPSHOTs alphas include some filesystem APIs to read/write files
n

napperley

04/08/2021, 10:02 PM
Does Okio have Maven artifacts for the K/N linuxX64, and linuxArm32Hfp targets?
m

mbonnin

04/09/2021, 7:57 AM
They have linuxX64, not Arm32Hfp: https://repo1.maven.org/maven2/com/squareup/okio/
Certainly worth a pull request though
1
n

napperley

04/13/2021, 2:01 AM
Ended up going with a single CArrayPointer that is used as a reusable buffer, and have defined a function to clear a buffer:
Copy code
internal fun clearBuffer(buffer: CArrayPointer<ByteVar>, bufferSize: ULong) {
    memset(__s = buffer, __c = 0, bufferSize)
}

internal actual fun combineImageFiles(dirPath: String) = memScoped {
    val appendBinaryMode = "ab"
    val readBinaryMode = "rb"
    val entries = listDirectory(dirPath).sortedWith(ImageNameComparator())
    val videoFile = fopen("$dirPath/video.mjpg", appendBinaryMode)
    val bufferSize = 71680
    val buffer = allocArray<ByteVar>(bufferSize)
    entries.forEach { e ->
        val dataSize = fileSize("$dirPath/$e")
        val tmpFile = fopen("$dirPath/$e", readBinaryMode)
        clearBuffer(buffer, bufferSize.toULong())
        fread(__n = 1, __stream = tmpFile, __ptr = buffer, __size = dataSize)
        fclose(tmpFile)
        fwrite(__n = 1, __s = videoFile, __ptr = buffer, __size = dataSize)
    }
    fclose(videoFile)
    Unit
}
3 Views