Hi everyone I want to use a media resource in my desktop app

Question

Hi everyone I want to use a media resource in my desktop application However I have noticed that there seems to be no MediaView component in Compose at the moment As a result I am forced to search for a component similar to Android s SurfaceView or JavaFx s PixelBuffer in order to use FFmpeg in C for decoding and render the decoded data into my Compose application Unfortunately it appears that Compose does not currently provide such a component May I inquire as to when a similar component might be added

Stylianos Gakis · Answer

A lot of words I am not familiar with in this message but reminded me of this thread here <https kotlinlang slack com archives CJLTWPH7S p1677599380551119> Just linking it in case it s useful to you

Michael Paus · Answer

Yes it would be great to have such an API in Compose I once wrote the code to connect the VLC player to JavaFX via such a PixelBuffer

Anivie · Answer

Well thanks < Stylianos Gakis> Indeed this is a similar issue However he is working on an Android device so he can use the API provided by Android to resolve the issue In my case I am working on a desktop device and do not wish to use Swing that provide the smae API

Anivie · Answer

And I share the same requirement with him Compose should provide us a native rendering interface to render data that we decoded by other way

Landry Norris · Answer

I unfortunately can attest that it s 100 possible to use ffmpeg to decode the frames and render to compose This was at my last job so I can t share any code

Landry Norris · Answer

I was able to create a Compose Image on Android and iOS then render that to an image I d imagine desktop will be similar to iOS

Landry Norris · Answer

FFmpeg is a very difficult library to learn with a lot of codec specific caveats I d highly recommend using Swing interop unless you have a very specific reason to write your own video player like we did

Anivie · Answer

Could you please inform me which component you are using

Landry Norris · Answer

I don t remember what method I used to create the image It was definitely specific to skiko

Anivie · Answer

If you are using a canvas or a similar component it is possible but unlike the component I mentioned in the example others can render images directly from the decoded data whereas canvas requires manual drawing in the JVM It s worth noting that if you only need to display a single image then canvas is acceptable but if I want to display consecutive frames for playing a video then the performance of canvas is clearly inadequate

Anivie · Answer

Are you utilizing an API present in Skia If so If so I may not know how it was implemented because I haven t studied Skia before

Landry Norris · Answer

Have you tested that The performance was great on iOS when I tested it On Android the canvas drawing was quite fast the main limiter was ffmpeg I could render at about 60fps using the canvas

Anivie · Answer

Did you mean `androidx compose ui graphics Canvas`

Landry Norris · Answer

Yes

Landry Norris · Answer

I remember having to make some optimizations to get that performance mainly regarding param stability and remembering but it s been a minute

Anivie · Answer

Ok thanks for the answer I ll try using Canvas

Anivie · Answer

But I don t think using Canvas for external drawing is the best for either performance or portability so I d still like compose to provide an API that allows us to do this more efficiently

Landry Norris · Answer

I agree It would be great to be able to have a SKCanvas as a composable That s what they should be backed with in the end anyways

Michael Paus · Answer

You know that you can get access to the native canvas via Canvas nativeCanvas Have you checked that API It has a lot of fancy methods which you don t find in the Compose Canvas

Anivie · Answer

OK that s a good idea Thank you for your answer

Landry Norris · Answer

I remember trying nativeCanvas but it didn t have what I needed at the time It may be different now

Anivie · Answer

OK thanks