Hey - Why can’t I access `ImageBitmap` pixels dire...

# compose-desktop

[spierce7]

Hey - Why can’t I access ImageBitmap pixels directly? If I wanted to create a filter for ImageBitmap it doesn’t seem very intuitive. Is ImageBitmap even the best abstraction for me to create a filtering system on?

[Kirill Grouchnikov]

It's probably not going to be the most performant for you to do this pixel by pixel. After some experimentation with it, I think the best way is to drop to the level of Skia and do image filtering with its shaders - if you're only targeting the desktop variant.

[spierce7]

I’ve never used shaders before

[spierce7]

Most of the filtering I’m doing is not to be displayed, but rather for pixel / object recognition in the image.

[Kirill Grouchnikov]

https://github.com/kirill-grouchnikov/aurora/blob/8885b6012f2959e81b64fa315ab57309[…]otlin/org/pushingpixels/aurora/bitmapfilter/BaseBitmapFilter.kt can get you started down the road of extracting pixels from the bitmap and then converting the filtered pixels back into a new bitmap

[Kirill Grouchnikov]

And https://www.pushing-pixels.org/2021/09/22/skia-shaders-in-compose-desktop.html for the Skia shaders where you can take advantage of things running on GPU where they can

[spierce7]

Can I use shaders to performantly create a color filter over an image to say… change an image to 8bit color?

[Kirill Grouchnikov]

That's exactly what shaders are great at

[spierce7]

How would I do something like that, and then programmatically get the pixel colors?

[Kirill Grouchnikov]

Probably something along the lines of

bitmap = ImageBitmap(400, 400)
canvas = Canvas(bitmap.asDesktopBitmap())
shader = ... // your shader
paint = Paint()
paint.setShader(shader)
canvas.drawPaint(paint)

and at that point you can get the pixels from the bitmap and inspect them.

[Kirill Grouchnikov]

Now, this is rather elaborate and rather useless - your shader will not run on GPU since this is an offscreen bitmap

[Kirill Grouchnikov]

So you might as well get the pixels directly from the source bitmap and manipulate them in software

[Kirill Grouchnikov]

The important question is what is "filtering system" in your use case. Where are the inputs coming from, and what are the outputs used for. If you can cut out the intermediate bitmaps and plug into the Skia rendering pipeline, you can have your shader / filters running on GPU and be quite performant.