https://kotlinlang.org logo
Join Slack
Powered by
This should arguably go into compose, but I'm buil...
# compose-desktop
c
This should arguably go into compose, but I'm building a pure desktop compose app (and I've only really done android development), but I'm looking to generate "clouds" as the background of my desktop weather app. I don't really know where to start, but I want my clouds to be generated since desktop will need to support like infinite sizes and so shipping a large video of clouds in the sky won't really work. Do I try to implement this with unity? OpenGL? Something specific in compose? A particle generator? Any pushes in the right direction would be helpful. Image located in thread of the type of cloud I'm looking at generating.
These clouds are almost like smoke that I want scrolling across the screen.
r
It depends how fancy you want your clouds to be
You could for instance generate physically based volumetric clouds
Or fake the hell out of it with 2D noise functions
An easy solution would be to just use a various sprites and stack them on top of each other in a somewhat random fashion
It would be easy to implement and cheap
c
@romainguy this is what my designer gave me. I'm scrubbing the timeline here which shows that the clouds aren't entirely static, but I'd be okay with static at first. Physically based volumetric clouds sounds interesting. The way that the video shows... it's almost like smoke with some of the ends curling, etc. I will play around with 2d noise functions tonight maybe. Is that like an open gl thing or a compose thing? (sorry the most custom stuff I've drawn in my life was a custom background on a button. 😂 so idk where to even start)
Kapture 2021-05-10 at 16.01.45.mp4
r
2D noise functions/etc. --> OpenGL
But given this design you can probably do it with a few sprite sheets
c
Thanks. Will try some openGL! If that fails I'll just try to get a bunch of pngs of clouds and animate them from left to right. 😄
r
And rotate/fade them to get those wispy bits
➕ 1
c
@romainguy one last question. If I did this in compose android would I also just try to run OpenGL in an AndroidView, or is there a successor to openGL in compose (could have sworn I read on twitter about openGL going away in compose?)
r
Compose has nothing to do with OpenGL
Here's a sample app I wrote that uses GL in Compose
It's based on alpha builds so outdated etc.
c
Thank you. This helps!
Hey @romainguy, I'm just getting back to this now. My designer basically gave me this to work off of and asked me to recreate it at the bottom of the screen (I tried to get out of it and just use the video that we purchased here: https://elements.envato.com/fog-8SCKJ6V) but it seems difficult to use video with alpha in android. Do you still recommend that I can create this rolling fog effect by just using openGL in my compose app?
r
With GL you could just play the video in a surface texture and treat the color as an alpha channel
c
Really? It seems like just putting the video on top of a blue background for example doesn't have the right effect though. In final cut pro, if I take the video above and put it on a blue background, it comes out all grey looking. I need to add a blend mode (like screen) for it to look good. Unless you're telling me that GL can also do a blend mode?
If so, then all I'm really learning is that I need to learn OpenGL lol. I'm a long time android developer (~5 years) and still haven't really ever had to touch it and so this requirement from design team is killing me.
r
So in GL you would do something like this:
vec4 color = texture(my video, UV);
And you have the RGBA color for the video
c
I was originally trying to play it in my android app using exoplayer, but exoplayer doesn't support alpha. Didn't know that GL could do it: https://github.com/google/ExoPlayer/issues/7789
r
Then you can reinterpret that data however you want. For instance:
myOutputColor.a = color.r;
Now you've just used the red channel of the video as the alpha channel for the output. So now you can blend the video properly :)
c
But cool. Okay. so you think I should be able to just play that video in GL in my android/compose app. Alright. I've got some learning to do. OpenGL is completely foreign to me. Thanks for pointing me in the right direction!
r
Or you could do fancier blendings like screen, overlay etc.
c
Yeah. Screen does work really nicely in final cut pro when I tried it there.
screen blending
no blending
lol
thats final cut pro showing the fog on top of a blue-ish background with screen blending (looks great) vs no blending. And so I'm trying to recreate in android
r
Note that if you render into a TextureView you could set the screen mode on that view directly
Maybe you don't need OpenGL at all
Yeah try that
Render the video in a texture view
But lemme check something first
Oooh yeah that'll work
Just call setLayerPaint() on TextureView
And set the screen xfermode on the paint you pass
Done
c
Oooh. Look at all these things that I'm learning.
Does the movie have to be in any specific format/encoding?
r
Nope
Just black and white like your example
And then any video format supported by Android's media codecs will work
When you create the MediaPlayer give it the surface TextureView gives you
And that's all
c
Cool. Giving it a whirl now. We budgeted about a week of my time to do this, so if I can get this working in one night... then I owe ya... something? lol
r
😁
Oh and you may have to setOpaque(false) on TextureView to disable overdraw optimizations I think
👍 1
c
Okay. So I have my black and white video playing, but the how am I supposed to make the black background transparent?
r
You don't. Just set the blend mode to screen or overlay and see what happens
Or add
If none of them work the way you want (but at least one should), I have another idea
c
gotcha. alright. Gimme a sec... 🤞
no dice.
Copy code
textureView.surfaceTextureListener = this
        textureView.isOpaque = false
        textureView.backgroundTintBlendMode = BlendMode.SCREEN
        mediaPlayer = MediaPlayer()
        try {
            fileDescriptor = assets.openFd("smoke.mp4")
        }
        catch (e: IOException) {
            e.printStackTrace()
        }
message has been deleted
top app bar showing, below is a blue background with the video playing over top, where the black is still visible.
r
You need to set the view to not opaque before the surface is created I think
👍 1
And don't set the background tint mode that won't do anything
👍 1
You need to set the xfermode of the Paint you pass to setLayerPaint
❓ 1
c
Wasn't passing anything. So now I need to just learn what I should be passing into setLayerPaint.
Don't see anything about xferMode.
but again. not my forte, so I'm probably missing something basic here.
r
Paint().apply { xfermode = PorterDuffXfermode(PorterDuff.Mode.SCREEN) }
Something like that
c
OHHHHH SHIT
IT'S LIKE KINDA WORKING
r
blendMode does the same but it's API 29+
c
message has been deleted
r
The javadoc of PorterDuff.Mode shows examples of all the available modes btw
c
I only said kinda because I need to fix up a few things. mainly... get it looping infinitely.
Had no idea textureView could do all this. lol
Also, the video I got from envato was a 
.mov
but that crashes, so it looks like I gotta try to convert the video to a .mp4
r
Yeah you'll need an mp4
Or mkv
H265 as mkv will probably be good quality/compression and it works on API 21+
WebM is also a good choice
With VP9 encoding
c
i think webm with vp9 can actually support transparency too (IIRC). What started this mess was that the design team hear that ios can support alpha videos with hevc. https://developer.apple.com/videos/play/wwdc2019/506
r
Yeah VP9 supports alpha but I don't know if it works through the entire pipeline
c
@romainguy thanks again so much. Could not have done it without you. I may try to write this up in a little blog post. This ended up being simpler than I thought. One last question. Should I be worried about killing battery life if this just would play as the background of my app?
r
The video should be decoded in hardware so probably not too bad. It's not ideal but not as bad as playing 3d animations or something I think. You'd have to measure :/
Btw looks like we could also use a ColorMatrixColorFilter to copy a color channel to the alpha channel and create transparency for a video this way
Basically you could green screen it :)
c
ColorMatrixColorFilter would work with textureView + mediaPlayer? Or is that a different technique in general?
r
It would work yes
But it has limitations and would require a carefully crafted video
👍 1
c
thank you again. porter duff is wild. some of these example inputs and outputs are like magic https://medium.com/mobile-app-development-publication/practical-image-porterduff-mode-usage-in-android-3b4b5d2e8f5f
r
Yep they're super useful and the basis of all compositing techniques. Note that what we call Porter-Duff modes in our API contains more than actual Porter-Duff modes (that's why I split them into Porter-Duff and blend modes in the docs). Screen is NOT a Porter-Duff mode for instance :)
👍 1
Also why there's now Paint.setBlendMode() :)
👍 1
Open in Slack
PreviousNext
Powered by