https://kotlinlang.org logo
Join Slack
Powered by
anyone has experience with AVPlayer in iOS? Trying...
# compose-ios
j
anyone has experience with AVPlayer in iOS? Trying to play a video which works fine if using URL from web source of a mp4 file, but as soon as I am using compose jetbrains resources for local file it doesnt play at all, just black screen. Tried Google a lot about it and also tried get error messages but seems to be none. Works perfect with same code in Android with same mp4 file, so not that video itself I think if not encoding issues on iOS. Is there a smart way of knowing why getting black screens of video not playing? I also verified path, is correct.
e
Is it this issue? https://github.com/JetBrains/compose-multiplatform/issues/3698 Then it should be fixed in coming beta.
j
Not sure, this is my implementation:
Copy code
@Composable
actual fun MyVideoPlayer(
    modifier: Modifier,
    resource: MyResource,
    settings: MyVideoSettings
) {
    val resourcePath = resource.absolutePath()
    val url = NSURL.fileURLWithPath(resourcePath)
    val playerItem = AVPlayerItem.playerItemWithURL(url)

    val player = remember { AVPlayer.playerWithPlayerItem(playerItem) }
    val playerLayer = remember { AVPlayerLayer() }
    val avPlayerViewController = remember { AVPlayerViewController() }
    avPlayerViewController.player = player
    avPlayerViewController.showsPlaybackControls = settings.showController
    avPlayerViewController.videoGravity = AVLayerVideoGravityResizeAspectFill

    playerLayer.player = player
    playerLayer.videoGravity = AVLayerVideoGravityResizeAspectFill
    UIKitView(
        factory = {
            val playerContainer = UIView()
            playerContainer.addSubview(avPlayerViewController.view)
            playerContainer
        },
        onResize = { view: UIView, rect: CValue<CGRect> ->
            CATransaction.begin()
            CATransaction.setValue(true, kCATransactionDisableActions)
            view.layer.setFrame(rect)
            playerLayer.setFrame(rect)
            avPlayerViewController.view.layer.frame = rect
            CATransaction.commit()
        },
        update = {
            player.play()
            avPlayerViewController.player?.play()
        },
        modifier = modifier
    )
}
@Elijah Semyonov Its quite strange, because if I use NsUrl from web url mp4 file exact same code working. Whats happening it looks like it inits the video and everything, find the file but not resize it. Thats my impression, as it do cover my other content behind it. So it gets a black background color fullscreen.
e
I’ll have a look at it
thank you color 1
j
I use it like this:
Copy code
Box(Modifier.fillMaxSize()) {
            MyVideoPlayer(
                modifier = Modifier.fillMaxSize(),
                resource = myResource("raw/video.mp4")
                //url = "<https://test.com/video.mp4>
            )
            Scaffold( ...
        }
As resource path I do ti like this:
Copy code
private fun resourcePath() = NSBundle.mainBundle.resourcePath + "/compose-resources/" + path
Also I couldve missed something with iOS specific things, but not sure 😛 The same mp4 file plays fine with Androidx media3 exoplayer on Android actual if that makes any difference 🙂
@Elijah Semyonov Hi, just curious, did you notice anything strange about it? Like something in my code that is missing or some odd how to load local resources? When using NSUrl with web/server urls it works fine, but local files from mainBundle seems not. Not sure if I need to enable something for video files in Xcode project settings or such? I verified the path itself works and can play exact same file on my computer used by simulator, so doesnt seem to be file system.
Also can I use alpha or beta releases of compose multiplatform somewhere? Cant find a release changelog for them?
e
I still didn’t have a look at it, but you can help me if you create a repo with an example reproducing the issue, so I can check out it 🙂
j
@Elijah Semyonov Yeah sure, should I use template from kmp.jetbrains.com maybe? Or which repo easiest to try with?
Also I cant share my video file, do you know a video file can try with?
e
This one should suffice
Just download arbitrary one from the internet 🙂
j
Yeah thanks, will create one soon. Can try refer to it within the issue in github maybe, but dont know if same issue or not.
e
You can post the issue here, then you will be able to observe its resolution 🙂
j
Hmm created sample project and ofc should it work in that one 😄 Start wondering if some shit with DRM protection or such. My original video is MOV file which I converted with ffmpeg to mp4. Could that be the issue maybe?
Its a bought video from iStockPhoto my company delivered.
Will double check with same sample video if that works in my actual code as well, to be 100% sure.
Also quite interesting video only play once, and not repeating as I would expect.
@Elijah Semyonov If having any experience in this topic I am all ears, but from the iOS compose multiplatform perspective everything works as it should and my code as well, so thats nice to know at least 😄 Like if you know if AVPlayer has some limitations in video encoding of mp4 or such.
e
Sorry, I’m not really well informed about the video encoding and compatibility shenanigans. What I want to do there is to ensure, that it’s a Compose problem, or not 😄
j
From my testing now it doesnt seem to be a compose problem 🙂 So thats promising 🙂
e
We had an issue with full screen modal of AVVideoPlayer calling 
viewDidDisappear
on Compose ViewController, triggering eager ComposeScene disposal together with interop views, that were the source of modal expansion, it should be fixed now and is coming in beta
j
I managed to re-convert from MOV to mp4 finally with another tool and then worked. Seems to be wrong config params in ffmpeg somehow AVPlayer not like on iOS 😛
e
Well, it’s nice to hear 🙂
j
Only thing left is looping the video doesnt seem to work now. Glad to know I actually coded the iOS shared jetbrains resources correct and with Android cloning from resources to res folder with raw identifier 😄
Because thats what I started to evaluate, if I did process source wrong on iOS how to look it up or not. But also this was nice, as solved another bug with loading custom fonts. Hopefully Jetbrains component resources will resolve this later on better.
e
You mean checking that the codecs are supported by iOS?
j
No, I mean how to fetch the bundle files in iOS vs resource identifier in Android for other file types than the ones Jetbrains resources library supporting 🙂
Like video files or raw files not really supported, only as byte arrays, which not fits my use case.
e
@Konstantin Tskhovrebov, it will be possible to retrieve the resource file bundle URL in a new resource API, right?
k
only as byte arrays resources in that meaning is a part of UI an app. like icons and texts. to show videos or something like that you have to provide it as a file on a file system and read it via files API (e.g. kotlinx-io)
j
In my case I only need reference with Uri vs NSURL to absolute path. Then each framework like Avplayer, exoplayer, Coil, Kamel, Lottie etc responsible chunk/stream bytes in their optimized way. Reading bytes already possible but not feasable. Would you mind add possibility get path to file as well? In my case created my own resource impl similar to one in jetbrains resources so I can re-use. So having a hybride of your solution and my own right now. Can be needed for other file types as well, as images in CMP not supporting all types yet.
k
what do you mean by the path on a jvm when a resource is inside a jar archive?
e
Perhaps we need sourceSet specific APIs for cases like this one?
j
@Konstantin Tskhovrebov As of example for Android I did use:
Copy code
@Composable
    override fun absolutePath(): String {
        return LocalContext.current.packageResourcePath + "/" + path
        //val classLoader = Thread.currentThread().contextClassLoader ?: CoodyPlatformResource::class.java.classLoader
        //val resourceUrl: URL = requireNotNull(classLoader.getResource(path))
        //return resourceUrl.path
    }
The commented lines is how you do in Jetbrains versions, at least the stable version of it. Not sure about alpha or dev snapshots.
For iOS I have counter part of actual implementation, how to fetch same file but in iOS shared bundle.
I am using this in iOS:
Copy code
actual class PlatformResource(private val path: String): PlatformResource {

    private fun resourcePath() = NSBundle.mainBundle.resourcePath + "/compose-resources/" + path

    @Composable
    override fun absolutePath(): String {
        return resourcePath()
    }

    fun readBytes(): ByteArray {
        val absolutePath = resourcePath()
        val contentsAtPath: NSData? = NSFileManager.defaultManager().contentsAtPath(absolutePath)
        if (contentsAtPath != null) {
            val byteArray = ByteArray(contentsAtPath.length.toInt())
            byteArray.usePinned {
                memcpy(it.addressOf(0), contentsAtPath.bytes, contentsAtPath.length)
            }
            return byteArray
        } else {
            throw IllegalStateException(path)
        }
    }
}
Would be nice if you had something similar in Jetbrains resources library 🙂
This is not only for videos, could be like PDF, audio files or whatever, as I cant have identical qualifiers for Android in other platforms like iOS. Have to find a common abdominator like a string relative path and then look it up what I need to have it both for the file path, but also supporting all framework APIs avaiable per platform. For AvPlayer case I need a NSUrl as of example, but not in Android that using Uri in androidx media3. I would prefer not loading all bytes and copy to another file with new path, cloning my existing file to another place.
k
We try to avoid to have an access to resources by path and make it as static as possible to have compile checks for it. for your case it is right way to have platform builders for file paths
j
Yeah but then need to add support like audioResource, videoResource methods and such point to assets/raw on Android and something else on iOS I guess to make it compile safe. Same as using painterResource("drawable/myImage.png")?
k
I saved your idea to the backlog
👍 1
thank you color 1
j
Anyway I solved it myself in my app, but I think everyone would benefit from this. To get some agnostic mimetype based variant or custom injections to support more than drawables, strings etc 🙂
Also gives you the power of using the file path and do some nice streams yourself with kotlinx io or such for other scenarios 🙂
In my case hardcoding: NSBundle.mainBundle.resourcePath + "/compose-resources/" + path If you change 
compose-resources
my code will break 😄
k
we will change it
it will be a kinda random to achieve a module isolation
j
Yeah I hope it will work multi module in future. Would like to split up my strings, images between my feature modules.
I dont mind 🙂
k
where do you use URIs in common code? what libraries support it now?
j
In common code I dont use Uri, but I create a PlatformResource class with relative string, and then from that I create Uri on Android and NsUrl on iOS in my compose code with help from the platform resource. In my particular case using in common a Composable videoplayer using actual in Android using ExoPlayer that takes PlatformResource and get the Uri, for iOS using AVPlayer with NSUrl.
Coil is btw using their own Coil3.Uri I think 🙂 So that I am using when having Coil, but I do not rely on this mechanism on Coil variant, in those cases I loading the data differnetly with their ImageRequest builder instead. Depending on resource variant I delegate different. Having MyImageResource, MyPlatformResource and MyTextResource. And with MyImage, MyText and MyVideo, where last one using MyPlatformResource. Depending which direction all libs taking I can delegate different variants 🙂
Doing it in commonMain like this now for images:
Copy code
sealed interface MyImageSource {
    data class Painter(val relativePath: String) : MyImageSource {
        val painter: Painter
            @Composable
            get() {
                if (LocalInspectionMode.current) {
                    return rememberVectorPainter(Icons.Default.Add)
                }
                return painterResource(relativePath)
            }
    }

    data class Vector(val vector: ImageVector) : MyImageSource

    data class Bitmap(val bitmap: ImageBitmap) : MyImageSource

    data class Coil(
        val coilRequest: ImageRequest,
        val transform: (AsyncImagePainter.State) -> AsyncImagePainter.State = DefaultTransform,
        val onState: ((AsyncImagePainter.State) -> Unit)? = null,
        val clipToBounds: Boolean = true,
        val filterQuality: FilterQuality = DrawScope.DefaultFilterQuality
    ) : MyImageSource
}
The idea I use for now is that I am supporting what compose foundation support for images and text. WHich is also what CMP supports. And then for cases outside of this for libraries or custom actual/expect variants like COil or Videos I do using relative String instead in commonMain and look it up when I need it in my composables which then will get platform targets properly. So it works both for libraries supporting targets I support or for those that do not, so I can cherry pick whatever state I want to be in 🙂 For now realized it works very well in KMP scope.
@Konstantin Tskhovrebov Here is my Android variant:
Copy code
@Composable
actual fun MyVideoPlayer(modifier: Modifier, resource: MyResource, settings: MyVideoSettings) {
    val context = LocalContext.current
    val path = File(resource.absolutePath())
    val resourcePackageName = context.packageName
    val resourceName = path.nameWithoutExtension
    val resourceId = context.resources.getIdentifier(resourceName, "raw", context.packageName)
    val uri = Uri.parse("android.resource://$resourcePackageName/$resourceId")

    val exoPlayer = remember {
        ExoPlayer.Builder(context)
            .build()
            .apply {
                val defaultDataSourceFactory = DefaultDataSource.Factory(context)
                val dataSourceFactory: DataSource.Factory = DefaultDataSource.Factory(
                    context,
                    defaultDataSourceFactory
                )
                val source = ProgressiveMediaSource.Factory(dataSourceFactory)
                    .createMediaSource(MediaItem.fromUri(uri))

                setMediaSource(source)
                prepare()
            }
    }

    exoPlayer.playWhenReady = true
    exoPlayer.videoScalingMode = C.VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING
    exoPlayer.repeatMode = Player.REPEAT_MODE_ONE

    DisposableEffect(
        AndroidView(factory = {
            PlayerView(context).apply {
                if (!settings.showController) {
                    hideController()
                    useController = false
                }
                resizeMode = AspectRatioFrameLayout.RESIZE_MODE_ZOOM

                player = exoPlayer
                layoutParams = FrameLayout.LayoutParams(MATCH_PARENT, MATCH_PARENT)
            }
        })
    ) {
        onDispose { exoPlayer.release() }
    }
}
For now I enforce all videos have to be in raw folder, which I copy to android res folder. But can easy change qualifiers/resource paths however I want. At moment its fine for me 🙂
k
got it, thx. I guess an API to convert a resource path to an uri would be enough
j
No problem 🙂 I think path would be working for most use cases yes 🙂
365 Views
Open in Slack
PreviousNext
Powered by