I am searching for a solution to play mp4 video in CFD. Solutions like vlcj in the sample or using J...

# compose-desktop

[Timo Drick]

I am searching for a solution to play mp4 video in CFD. Solutions like vlcj in the sample or using JavaFX with a JFXPanel do work but i can not draw on top of the video compose components. So it only works when there is no UI on top of the video. Do anyone know a solution to draw compose components ontop of a video player?

[darkmoon_uk]

I've not tried to use it, but pretty sure I saw an official video player component being worked on in the main compose-jb repo recently...

[Timo Drick]

As far as i know there is just this sample code: https://github.com/JetBrains/compose-jb/tree/master/experimental/components/VideoPlayer
Which uses vlcj. And as i already mentioned the problem is that i can not draw other compose components on top of the video

[Timo Drick]

I got a working prototype using vlcj with RenderCallback that renders the image into a Skia Bitmap and than draw the image inside of a Compose Canvas.
Than it is a normal compose component without using Swing compatibility layer.
I will do a little bit of cleanup and than post a solution in the next days.

[Landry Norris]

I can't provide any code, but at work, I built a fully compose based video player using libav (ffmpeg) with Kotlin/Native and JNI. You can see an example video player in the kotlin compiler github that uses libav. They use SDL, but I can verify that using compose works too, since we use it.

[Landry Norris]

The frames get drawn directly into a compose canvas, so there's no restrictions on what can be shown above.

[Timo Drick]

Interesting i didn't know that compose works in Kotlin/Native. But i assume you use the jni interface for compose right?

[Landry Norris]

Compose doesn't work in Kotlin/Native right now. I created an IntArray and passed it through JNI and had libav write pixel data to it. I then used the nativeCanvas to write the pixel array.

[Landry Norris]

I did this on Android (and a quick prototype for iOS compose that hasn't been released yet). I'd assume it will work with desktop compose as well.

[Timo Drick]

Cool 😎

[Landry Norris]

Will you need any complex features for the player (seeking, reverse, etc)? The libav libraries can be hard to work with, but they're low level, so they can be used to implement nearly anything with enough skill. If all you need is forward playback, I'd highly recommend largely copying from the video-player sample in the kotlin github repo and modifying it to use jni+compose instead of SDL

[Timo Drick]

With vlcj i use a Skia Bitmap to copy the IntArray (ByteArray) into it and than draw it inside of a Canvas too:

//initializing a buffer image
val bitmap = Bitmap().also {
    it.allocN32Pixels(sourceWidth, sourceHeight, true)
}
val composeImage = bitmap.asComposeImageBitmap()

// Than in the Canvas drawing code..
// byteBuffer comes from the vlcj callback 
val array = ByteArray(buffer.remaining())
buffer.get(array)
buffer.rewind()
bitmap.installPixels(array)
drawImage(composeImage)

[Timo Drick]

I am not pleased with the code yet because it creates a new ByteArray on each frame that is drawn. But when i keep just one ByteArray which gets redrawn i get crashes and wired visuals.

[Timo Drick]

I need just playback and seeking. Nothing else. Maybe for one of my projects i need to create a thumbnail.

[Timo Drick]

I remember this kotlin native sample. But i can not find it any more. Looks like they have removed it.

[Landry Norris]

It's somewhere in the compiler backend tests now. I'd just use find file to look for video-player.

[Landry Norris]

I had to look it up the other day to check something. Took me forever to find that they moved the samples.