Is there a way to create a transparent video (e.g. with Apple ProRes 4444 format) from a Composable?...

# compose-desktop

[louiscad]

Is there a way to create a transparent video (e.g. with Apple ProRes 4444 format) from a Composable?

I know one can export a Composable to an image (including transparency) using ImageComposeScene, and I'd like to do just that, but with multiple renders following states changes, until my code says "done", and get a video outputted to an OutputStream/Sink/File with a format that supports transparency.

[Kirill Grouchnikov]

Did you try Xuggler? It has IMediaWriter for creating video streams

[Kirill Grouchnikov]

Or FrameRecorder in JavaCV

[louiscad]

Absolutely not, these names are new to me, I will search for those

[louiscad]

Xuggler has been deprecated for a while. The recommended replacement, Humble-video, doesn't support Apple Silicon 😞 https://github.com/artclarke/humble-video/issues/157

[louiscad]

JavaCV looks interesting, I'll see if it suits my needs 🙂

[louiscad]

Looks like it's using OpenCV (+ffmpeg), and OpenCV seems to use AVFoundation on macOS, so I guess it'll work alright on my laptop

[louiscad]

Me trying to record a red circle for 2 seconds…

small_test.mp4

[Skaldebane]

Not sure how I stumbled across this, but I've done this before with ffmpeg, by generating all frames as pictures from ImageComposeScene (PNG, with transparency). Encoding them into a video with the right settings preserves the transparency in the video itself.

[Skaldebane]

Here's the video result, from an animated Composable.

output.mp4

[Skaldebane]

This is a .mp4 , but you can encode the images into a VP9 video which supports transparency:

ffmpeg -framerate 30 -i input%d.png -vf "scale=1920:1080" -c:v libvpx-vp9 -pix_fmt yuva420p output.webm

[Skaldebane]

Change the framerate, resolution and other parameters to your needs, but what matters is -c:v libvpx-vp9 to use VP9, and -pix_fmt yuva420p which is a pixel format that supports an alpha channel.

[Skaldebane]

For ProRes: -c:v prores_ks alongside yuva444p10le (10-bit depth) or yuva444p (8-bit depth) pixel formats (idk what's the practical difference, but either one will do).

[louiscad]

That's what I ended up doing, but I used WEBP (100% quality) instead of PNG because it's faster to encode. I also leveraged coroutines cleverly to maximize the throughput. Fun fact: the Apple ProRes 4444 video is larger than the sum of the WEBP images, and even the sum of the PNG version.

[Skaldebane]

Yeah, ProRes is quite the heavy format, I guess that's because there's barely any compression going on, to preserve all detail, and video pixel formats are a whole lot larger and less optimized when they're not compressed or shared between frames afaik.
If size matters to you then I guess VP9 would be a better fit, and it's more widely supported (it's the one used by YouTube, for example).

[louiscad]

Yup, I'm using VP9 for the web, but used Apple ProRes 4444 so I could include the videos into Final Cut Pro (though I probably won't use it much long term)