I'm experimenting with custom (OpenGL) rendering i...

# compose

[Leon Linhart]

I'm experimenting with custom (OpenGL) rendering inside Compose UI composables and have a couple regarding to correctly schedule the rendering and embedding an offscreen framebuffer. (See 🧵 for more)

[Leon Linhart]

Basically, there are two ways to integrate custom rendering with Compose UI:
1. Compose UI as overlay over an custom OpenGL scene.
2. Custom OpenGL rendering inside of composables.
The first seems rather easy, but I wondered if 2. is feasible because it would allow even more stuff. In my current approach, I set up an OpenGL context and manually configure the surface (etc.) similar to how it's done in the LWJGL integration example. Next, I set up a routine that renders to an off-screen framebuffer and wrap it in a surface (just like the backbuffer for the containing Window). However, when I then try to draw the surface into a canvas (see below), the off-screen buffer does not appear to be correctly used.

Canvas(modifier) {
        drawIntoCanvas { canvas ->
            surface().draw(canvas.nativeCanvas, 0, 0, null)
        }
    }

RenderDoc shows that Skia creates an empty texture and uses this instead of the off-screen framebuffer's color attachment.

[Leon Linhart]

So, what I'm worried about is state. I noticed that the OpenGL state (e.g. bound buffers, current program) is seemingly not cleared/set between frames. If this is true, then I'm either running my off-screen rendering at the wrong time, or this idea is infeasible. Does anyone have experience/feedback regarding this before I fully dive into what Compose/Skia does under the hood here?

[Zach Klippenstein (he/him) [MOD]]

You might need AndroidExternalSurface I think?

[Leon Linhart]

Looks promising. I'll try that out. Thanks for the pointer! 🙂

[Leon Linhart]

Unfortunately, AndroidExternalSurface is only available on Android and the implementation is not of much use either because it heavily depends on Android views. Do you happen to know if there is something that's available on desktop too? (Apologies for the confusion, maybe I should've posted in compose-desktop.)

[Zach Klippenstein (he/him) [MOD]]

I don’t know, sorry