Is there a good way to get a reference to a compose Image that I could pass to C (or Kotlin/Native) code to write to? Currently, I have an IntArray in JVM that I pass through JNI, and write to that from my K/N code. I then use the IntArray to create a compose Bitmap and draw that to a compose Canvas. Presumably, there’s a way to back an image with a Skia canvas, and pass the Skia canvas, like with the android Surfaces?

On Android or desktop?

Ideally cross-platform, but even if just Android, I’d be happy.

A

Canvas

isn’t a backing store, it’s an interface to write into a bitmap

I figured the canvas wouldn’t be the best route. Is there a way to do this with an Image?

Skiko would only be relevant if you use Jetbrains’s version of Compose on Android

What are you worried about exactly on the Android side? You can create a

Bitmap

from an

IntArray

just fine (or even

lock/unlockPixels

in JNI to write into the bitmap directly)

Mainly performance. Passing the IntArray through JNI for filling, then using copyPixelsFromBuffer seems to have a non-trivial performance hit. I’ll have to look into lock/unlockPixels.

Passing the int array through JNI isn’t costly no, but it depends on what you do with it then

Passing the array through JNI for filling wasn’t as much of a hit as I’d initially assumed it would be (less than a ms if I remember correctly). Would just be nice to avoid writing to the IntArray, then copying data to a Bitmap, then whatever runtime cost it is to draw the bitmap to the compose canvas (the last part is likely the least performance cost).

Passing an array is just passing a reference. It’s what you do with it in JNI that matters

Where would be the best place to learn more about the texture upload?

What do you want to learn?

I’d mainly like to learn how to optimize this rendering process. Our performance on Android is lagging behind our iOS performance (my Tab S8+ struggles to keep up with an older iPad). I’m looking to optimize it as much as possible. I reduced recompositions as much as possible last week. Looking to do more.

Also: do you do this every frame?

We needed a video player, but ExoPlayer couldn’t fill everything we needed. We’re currently using ffmpeg to decode frames, then send them for rendering. We used to use Surfaces, but looking to switch to full compose, especially for the iOS support. I already have a branch that uses this same method of rendering on iOS, with an expect/actual for drawing the IntArray to the DrawScope.

Oh yeah ouch, bitmaps on Android aren’t meant for this. You should use a

Surface

One requirement is the ability to ‘screen record’ parts of the screen, and surfaces didn’t show up when having a view draw to a bitmap. I had a method that would splice together the surface and the produced bitmap (also had to handle things above the surface).

SurfaceView

shows its content in a separate window

I remember TextureView having a major blocker for us at the time (2 years ago now) we looked at it. Can’t remember for the life of me what it was. I’ll have to look at PixelCopy for sure.

The only difference between

TextureView

and

SurfaceView

is that

TextureView

requires an extra GPU copy at draw time, but should otherwise be the same

There doesn’t happen to be a good TextureView equivalent for Compose, does there?

There isn’t, you’d use a

TextureView

FFmpeg 6.0 claims MediaCodec support, so hopefully that helps on the decoding side. The time we spend in ffmpeg calls right now is quite significant.

Use a

Surface

or

lockPixels

, it’s going to be your best solution if you are dead set on doing the decoding yourself

Should I unlock the pixels as soon as I finish writing the new data, or is it fine to lock it once, then unlock it when it’s no longer used? I’d imagine it’s not pinning the object, but using one of the Get*Critical JNI methods, which would make it unsafe to leave locked, is this correct?

You’ll need to unlock before you can draw

I see. So even for the same bitmap, I need to assume that addrPtr can change between draws.

It probably won’t (although it could, there’s no guarantee) but it’s not about that, it’s about synchronizing the data for the rendering pipeline

I’ll take a look at passing the bitmap through to JNI to avoid the IntArray copy. This will diverge the behavior a bit from iOS, so I’ll have to see what code I can share in this process. Hopefully it will be more when #compose-ios is ready to use.