Is there any way to more efficiently get an photo ...

# compose-ios

[Stefan Oltmann]

Is there any way to more efficiently get an photo from Apple Photos as a Compose ImageBitmap?

Below is my current approach, which has the drawback that converting NSData into ByteArray requires having both byte arrays in memory at a time. For huge a lot of images this will result very fast in an OutOfMemoryError.

Isn't there any way to improve the code below? 🤔

class SkiaAppleImageLoader() : AbstractImageLoader() {

    private val imageManager = PHImageManager.defaultManager()

    private val fullImageRequestOptions = PHImageRequestOptions().apply {
        deliveryMode = PHImageRequestOptionsDeliveryModeHighQualityFormat
        resizeMode = PHImageRequestOptionsResizeModeNone
        networkAccessAllowed = true
        synchronous = true
    }

    fun loadFromSystemPhotoLibraryAsData(uri: String): NSData? {

        val asset: PHAsset = PHAsset.fetchAssetsWithLocalIdentifiers(
            identifiers = listOf(uri),
            options = null
        ).firstObject as? PHAsset ?: return null

        var imageData: NSData? = null

        imageManager.requestImageDataAndOrientationForAsset(
            asset = asset,
            options = fullImageRequestOptions
        ) { result, _, _, _ ->

            imageData = result
        }

        return imageData
    }

    @OptIn(ExperimentalForeignApi::class)
    override suspend fun loadFullImageFromSystemPhotoLibrary(uri: String): ImageBitmap {

        val nsData = loadFromSystemPhotoLibraryAsData(uri)

        assertNotNull(nsData) { "Failed to load NSData for asset $uri" }

        // FIXME Converting NSData into ByteArray causes having both in memory at a time.
        val bytes = ByteArray(nsData.length.toInt()).apply {
            usePinned {
                memcpy(
                    __dst = it.addressOf(0),
                    __src = nsData.bytes,
                    __n = nsData.length
                )
            }
        }

        return Image.makeFromEncoded(bytes).toComposeImageBitmap()
    }
}

[Stefan Oltmann]

ChatGPT suggests streaming data reading from NSData, which will certainly reduce memory usage at any given moment. However, I'm primarily interested in finding a solution that allows Skia to natively read the NSData without the need for copying, possibly through the use of a wrapper.

ChatGPT code for streaming:

private suspend fun readNSDataIncrementally(nsData: NSData): ByteArray {
        val bufferSize = 4096 // Adjust the buffer size as needed
        val byteArrayOutputStream = ByteArrayOutputStream()
        val buffer = ByteArray(bufferSize)
        
        nsData.bytes.usePinned { pinned ->
            var bytesRead = 0
            while (bytesRead < nsData.length.toInt()) {
                val bytesToRead = min(bufferSize, nsData.length.toInt() - bytesRead)
                val srcOffset = bytesRead.toLong()
                memcpy(
                    __dst = buffer.refTo(0),
                    __src = (pinned.address + srcOffset).reinterpret(),
                    __n = bytesToRead.toLong()
                )
                byteArrayOutputStream.write(buffer, 0, bytesToRead)
                bytesRead += bytesToRead
            }
        }
        
        return byteArrayOutputStream.toByteArray()
    }

[Michael Paus]

I do not quite understand your problem. If any image is so large that two of them blow up your device then this whole code would not work at all. The bytes you are concerned about are the relatively small compressed bytes of a JPEG, PNG, … The real memory eater is in Image.makeFromEncoded which will uncompress the image bytes to their full size which is normally much bigger.

[Stefan Oltmann]

JPEG files can be 20 MB each with no problem, which makes for Compose for iOS 40 MB in heap instead of 20 MB. If you process a lot of them doubled space requirements is a problem.

I edited my problem statement above. I have both: Big images and a lot of them.

[Stefan Oltmann]

The fact that an uncompressed Bitmap is sizable is a given, and unfortunately, we can't alter that reality.

However, what compounds the issue is that every set of image bytes I load— and I do load a substantial number of them— ends up occupying twice the memory and subsequently requires garbage collection.

[Michael Paus]

I still don’t see how this could lead to an OOME. The garbage collector will reclaim the memory whenever it is needed because it is used only temporarily during a single conversion. The garbage collection effort should also be neglectable because we are talking about just two large objects per image. Of course I don’t know your exact requirements but don’t you think you are over-optimizing here? Just my two €ent.

[Stefan Oltmann]

Nope, I'm not over-optimizing here. Ashampoo Photos for iOS suffers from crashes due to high memory usage when scrolling the gallery very fast. I try to improve the situation. I now do more aggressive GC.collect() here and there, which reduces scrolling performance.

But sometimes it's not enough and memory demand is very high.

The Android & JVM Versions, which are already available, don't have this problem. The JVM GC works here very well without any explicit calls.

[Sebastian Aigner]

Heya! Just checked with our team folks, they are aware of this issue: https://youtrack.jetbrains.com/issue/KT-52658/Native-GC-scheduler-ignores-associated-native-memory