I want to use Coroutines Flow to generate Thumbnai...

# getting-started

[Stefan Oltmann]

I want to use Coroutines Flow to generate Thumbnails from images, but I'm not sure which Flow is the correct one to use.

So my idea was to send images that need to be processed to a Channel and read from that with channelFlow() ... but since Channels are hot they seem to be a bad idea (after watching the KotlinConf2019 talk).

I want to send a URL to a flow somehow and receive the ImageBitmap back. The first flow should read images from disk one after another, feed them to multiple coroutines that process thumbnails and send that back to the caller.

How to do that?

[CLOVIS]

Why do you want to use a flow?

It seems to me that you have an input of an image, and want its thumbnail, many times, which fits a normal suspending function:

suspend fun Image.createThumbnail() {
  // Your logic here
}

Then when you have that, you can use normal flows if you want to, but I don't see any benefits (or maybe I didn't understand your objective) :

val thumbnails = flowOf(image1, image2, ...)
  .map { it.createThumbnail() }

[Stefan Oltmann]

I want to use the flow to construct a pipeline where I feed URLs into it (for the images that become visible as the user scrolls trough a gallery grid) and just load them - one after another.

[Stefan Oltmann]

With flowOf() I would need to know the list of images in advance 😕
And a hot channel is bad as I understood @elizarov at his talk at KotlinConf19.
So it's not quite clear to me what a good channel alternative is to pipe something into a flow.

[Stefan Oltmann]

Without coroutines in pure java I would create Runnables and submit them to a

Executors.newSingleThreadExecutor()

That way they get executed one after another

[Stefan Oltmann]

I see that there is something like FlowChannels , but also callbackFlow...

[Stefan Oltmann]

It's a bit confusing what to use when

[Casey Brooks]

I don’t think your use-case is a good fit for a Flow, either hot or cold. Unlike traditional threads, Coroutines are designed to be able to spawn many hundreds, thousands, or even millions of concurrent jobs, and you could very easily launch a new job for each image in the grid.

I think some of your confusion is not fully understanding the differences and use-cases of hot vs cold flow. A cold Flow is not necessarily a pipeline of data, but more accurately a pipeline of transformations on a single set of data. You can’t post additional items to a cold Flow from outside of the Flow, and since the image-loading jobs come from the UI posting to the background and the images are revealed, that rules out the possibility of a cold Flow.

And a hot channels/flow is not necessarily good or bad, you just need to know how to properly manage it. It’s like a blocking queue, where items get inserted in one end from anywhere, and consumed sequentially in-order from the other end. This is a better choice for handling events that come from the UI, but it has the drwback that things are processed one-at-a-time, which would make the overall image-loading slower.

A better option than either of those is just to launch the image-loading job directly from each image in the grid and load all images fully in parallel. There’s really no reason why you should wait for one image to finish loading before starting the next, so don’t limit yourself to a Flow because it seems like a higher-level API on suspending functions. Suspending functions are already a high-level API and should be the first place to start. Channels, Flows, and other coroutine types exist to solve a very specific problem, and are not designed to be general, higher-level mechanisms like one might be used to with RxJava.

[Stefan Oltmann]

Thank you. I understand that now better. I achieved what I wanted by defining a single thread coroutine just for the file-reading part. Everything before and after runs in parallel.

[CLOVIS]

If it's an IO operation (like loading a file), it's probably better to load them all in parallel (if it's IO-bound anyway, the OS will make it sequential, and if it isn't you get a bit more speed).

I'm interested in what your architecture looks like. For example, using React, the easiest solution would be to have a useEffect in your component that launches to load that particular imagine, and let other components deal with their own things.

It's unlikely you use React though so I'm curious what your implementation looks like so we can help better

[Stefan Oltmann]

I use Jetpack Compose for Desktop

[Stefan Oltmann]

I wasn't aware that the OS will make it sequential. I just remember that for instance copying two files to an USB stick in parallel makes the whole process slower than just handle both files after another. That's why I was thinking it might help performance to reduce parallelism here.

[Casey Brooks]

Oh yeah, since you’re using Compose you really shouldn’t be bother with posting the image-loading to a background queue. That’s just making things more complicated than it needs to be, and what would be more useful is only loading the images that are actually visible on the screen. Using produceState with LazyVerticalGrid to load the image for each view will keep things declarative, but also cancel loading when the view leaves the screen if it hasn’t finished yet (so you’re not wasting system resources loading images that won’t be seen).

Also to that point, if the screen feels sluggish and that’s why you’re looking into this, it might be due to the lazy grid (not the image-loading); there seem to be a lot of performance issues with that right now. Image-loading should all be happening in a background thread and would not cause UI jank or low framerates; at worst, unoptimized image-loading would just make it take slightly longer for the image to become visible, but should not impact the responsiveness of the UI itself

[Stefan Oltmann]

How can I cancel if its leaving the stream? I only find the DisposedEffect which triggers later.

[Stefan Oltmann]

And yes, it's really sluggish in Compose for Desktop right now 😞

[Stefan Oltmann]

@Casey Brooks Currently I try to find out what makes it so sluggish. 😕

I got an interface like this:

interface ImageLoader {

    suspend fun loadFullImage(photo: Photo): ImageBitmap

    suspend fun loadThumbnailImage(photo: Photo): ImageBitmap

}

And this is my Composable:

@Composable
fun AsyncImage(
    photo: Photo,
    imageLoader: ImageLoader
) {

    val imageBitmap = remember { mutableStateOf<ImageBitmap?>(null) }

    LaunchedEffect(photo) {

        /* Load image if missing. */
        if (imageBitmap.value == null) {
            launch {
                withContext(Dispatchers.Default) {
                    imageBitmap.value = imageLoader.loadThumbnailImage(photo)
                }
            }
        }
    }

    imageBitmap.value?.let {

        Image(
            it,
            null,
            modifier = Modifier.padding(8.dp)
        )

    } ?: run {

        Placeholder(100.dp)
    }
}

The whole grid looks like this:

@Composable
fun App(
    darkMode: Boolean,
    store: PhotoStore,
    imageLoader: ImageLoader
) {

    AppTheme(
        darkMode = darkMode
    ) {

        val state = store.observeState().collectAsState()

        val photos = state.value.photos

        Column {

            LazyVerticalGrid(cells = GridCells.Fixed(4)) {
                items(photos.size) { index ->

                    val photo = photos[index]

                    AsyncImage(
                        photo,
                        imageLoader
                    )
                }
            }
        }
    }
}

[Stefan Oltmann]

Maybe my ImageBitmap Thumbnails are too big? (480x360px)

[Stefan Oltmann]

It really hangs if an Image is set and recomposition of AsyncImage happens

[Stefan Oltmann]

Please tell me that I do something really wrong here.

[Casey Brooks]

There are lots of types of effects in Compose for various purposes. This doc page does a good job of describing them all; some launch coroutines, some return values, but all are tied to the Composable’s lifecycle, which is what’s most important. For example, LaunchedEffect isn’t the only way to launch a coroutine; LaunchedEffect is more of a “fire-and-forget” coroutine, but you also have produceState which launches a coroutine and returns a result as a State. You can also rememberCoroutineScope to get a handle to a CoroutineScope and do more custom stuff with it, if needed.

To your example, you should be using produceState instead of LaunchedEffect (and you should’t need to do an additional launch { } inside LaunchedEffect either, it’s already running a coroutine). https://developer.android.com/jetpack/compose/side-effects#producestate

If your image-loader already exposes a suspend API, then it should clean itself up automatically when it gets cancelled. If it’s a callback-based API, you can clean that up with coroutines APIs when the coroutine is cancelled. It’s the job of Compose to decide when to cancel the coroutine, and the coroutine’s job to unsubscribe and/or cancel the background work in response to cancellation.

This snippet shows how one might tie a callback API to coroutines and cancel it properly (example uses OkHttp), entirely within the normal coroutines library:

https://github.com/gildor/kotlin-coroutines-okhttp/blob/master/src/main/kotlin/ru/gildor/coroutines/okhttp/CallAwait.kt#L36-L56

Or, produceState includes its own awaitDispose function for the same purpose:

https://developer.android.com/reference/kotlin/androidx/compose/runtime/ProduceStateScope#awaitDispose(kotlin.Function0)

And here’s how you could transform your snippet to using `produceState`:

kotlin
@Composable
fun AsyncImage(
    photo: Photo,
    imageLoader: ImageLoader
) {
    val imageBitmap = produceState<ImageBitmap?>(initialValue = null, photo) { 
        value = withContext(Dispatchers.Default) {
            imageBitmap.value = imageLoader.loadThumbnailImage(photo)
        } 
    }

    imageBitmap.value?.let {
        Image(
            it,
            null,
            modifier = Modifier.padding(8.dp)
        )
    } ?: run {
        Placeholder(100.dp)
    }
}

[Stefan Oltmann]

Thank you. That helps me a lot to make it a cleaner solution. 👍

[Stefan Oltmann]

The performance issues may be related to the early stage of Compose for Desktop. It's faster on Android.

[Stefan Oltmann]

I handle cancellation with calls to "isActive" in my suspend function. If I scroll really fast trough my library I see that some images actually get cancelled because the Composable was disposed before creating a thumbnail was ready. It's not really a "not visible" and more a "the lazy grid reuses this cell" thing, but it may be good enough.

What bothers me is that even if I cache the ImageBitmap and there is no IO anymore it's sluggish if I scroll trough the gallery. Did you mean that by problematic rendering of images?

What's the best practice? Using geometry reader and generate thumbnails that have 100% correct size so that no scaling is needed? I just investigate what's so slow. Profiler says 30% of time on main frame is just used for drawImage()

[Stefan Oltmann]

On Android the same code runs perfectly fluid and it just hangs when there is a GC pause - that's a hint where I can optimize memory.

But for Compose for Desktop I don't know what to do.

[Casey Brooks]

I’ve only used Compose on Desktop so I don’t really now how it compares to Android, but I think it’s just a general problem with lazy lists right now, where they perform worse than non-lazy lists, and isn’t related to images or any specific content in the list. And yes, the view getting reloaded is exactly the behavior I was thinking of, I don’t think produceState gets cancelled when something is non-visible (like scrolling off-screen), it’s cancelled because the view gets disposed when its scrolled off-screen in a lazy list

[Casey Brooks]

For 30% of time spent in drawImage, that might be something more related to how Swing renders bitmaps? I’m not sure, but you might ask in the #compose-desktop channel about that

[Stefan Oltmann]

Thanks. I will try that out.

I just discovered that part of the problem might be that you can't directly draw an Image but you need to wrap it into a ImageBitmap which has a IMHO expensive readPixels() method if it creates a new int/byte array on every image drawing.

[Stefan Oltmann]

I speeded it up a bit by further reducing my thumbnail size down to 320px, but it's still not as good as on Android. 🤷‍♂️

[Stefan Oltmann]

Maybe it's related to the current state and there will be performance improvements in the future. I hope so.

[Stefan Oltmann]

Oh wow... What really helped was switching from ImageIO directly to SKIJA for resizing the images. ImageIO.read() is 10x slower than doing it with SKIJA and takes half the memory. Seems like a lot of performance problems on the main Thread came from GC.

So I consider still that "stop the world" GC as a problem. Even if you can do the work on another thread, if that other thread produces to much garbage it will hit your main thread. 😕