This should arguably go into compose, but I'm building a pure desktop compose app (and I've only really done android development), but I'm looking to generate "clouds" as the background of my desktop weather app. I don't really know where to start, but I want my clouds to be generated since desktop will need to support like infinite sizes and so shipping a large video of clouds in the sky won't really work. Do I try to implement this with unity? OpenGL? Something specific in compose? A particle generator? Any pushes in the right direction would be helpful. Image located in thread of the type of cloud I'm looking at generating.

It depends how fancy you want your clouds to be

@romainguy this is what my designer gave me. I'm scrubbing the timeline here which shows that the clouds aren't entirely static, but I'd be okay with static at first. Physically based volumetric clouds sounds interesting. The way that the video shows... it's almost like smoke with some of the ends curling, etc. I will play around with 2d noise functions tonight maybe. Is that like an open gl thing or a compose thing? (sorry the most custom stuff I've drawn in my life was a custom background on a button. 😂 so idk where to even start)

2D noise functions/etc. --> OpenGL

Thanks. Will try some openGL! If that fails I'll just try to get a bunch of pngs of clouds and animate them from left to right. 😄

And rotate/fade them to get those wispy bits

@romainguy one last question. If I did this in compose android would I also just try to run OpenGL in an AndroidView, or is there a successor to openGL in compose (could have sworn I read on twitter about openGL going away in compose?)

Compose has nothing to do with OpenGL

Thank you. This helps!

With GL you could just play the video in a surface texture and treat the color as an alpha channel

Really? It seems like just putting the video on top of a blue background for example doesn't have the right effect though. In final cut pro, if I take the video above and put it on a blue background, it comes out all grey looking. I need to add a blend mode (like screen) for it to look good. Unless you're telling me that GL can also do a blend mode?

So in GL you would do something like this:

I was originally trying to play it in my android app using exoplayer, but exoplayer doesn't support alpha. Didn't know that GL could do it: https://github.com/google/ExoPlayer/issues/7789

Then you can reinterpret that data however you want. For instance:

But cool. Okay. so you think I should be able to just play that video in GL in my android/compose app. Alright. I've got some learning to do. OpenGL is completely foreign to me. Thanks for pointing me in the right direction!

Or you could do fancier blendings like screen, overlay etc.

Yeah. Screen does work really nicely in final cut pro when I tried it there.

Note that if you render into a TextureView you could set the screen mode on that view directly

Oooh. Look at all these things that I'm learning.

Nope

Cool. Giving it a whirl now. We budgeted about a week of my time to do this, so if I can get this working in one night... then I owe ya... something? lol

😁

Okay. So I have my black and white video playing, but the how am I supposed to make the black background transparent?

You don't. Just set the blend mode to screen or overlay and see what happens

gotcha. alright. Gimme a sec... 🤞

You need to set the view to not opaque before the surface is created I think

Wasn't passing anything. So now I need to just learn what I should be passing into setLayerPaint.

Paint().apply { xfermode = PorterDuffXfermode(PorterDuff.Mode.SCREEN) }

OHHHHH SHIT

blendMode does the same but it's API 29+

The javadoc of PorterDuff.Mode shows examples of all the available modes btw

I only said kinda because I need to fix up a few things. mainly... get it looping infinitely.

Yeah you'll need an mp4

i think webm with vp9 can actually support transparency too (IIRC). What started this mess was that the design team hear that ios can support alpha videos with hevc. https://developer.apple.com/videos/play/wwdc2019/506

Yeah VP9 supports alpha but I don't know if it works through the entire pipeline

@romainguy thanks again so much. Could not have done it without you. I may try to write this up in a little blog post. This ended up being simpler than I thought. One last question. Should I be worried about killing battery life if this just would play as the background of my app?

The video should be decoded in hardware so probably not too bad. It's not ideal but not as bad as playing 3d animations or something I think. You'd have to measure :/

ColorMatrixColorFilter would work with textureView + mediaPlayer? Or is that a different technique in general?

It would work yes

thank you again. porter duff is wild. some of these example inputs and outputs are like magic https://medium.com/mobile-app-development-publication/practical-image-porterduff-mode-usage-in-android-3b4b5d2e8f5f

Yep they're super useful and the basis of all compositing techniques. Note that what we call Porter-Duff modes in our API contains more than actual Porter-Duff modes (that's why I split them into Porter-Duff and blend modes in the docs). Screen is NOT a Porter-Duff mode for instance :)