Thread
#compose
    Pedro Gomez

    Pedro Gomez

    1 year ago
    Hi all! I'm writing an app and adding screenshot tests for my components and I'm facing some issues with the RoundedCornerShape. When using a shape for the component background the rendering result is not the changes a little bit and this makes the screenshot test tool to report there is an error in the rendering. The shape I'm using is this
    RoundedCornerShape(size = 16.dp)
    The error reported by the screenshot testing shows a bunch of small different pixels making the tests fail. Red dots in the screenshot are the errors reported by the testing tool. Do you know if the rendering can vary depending on the build or any other param?
    Andrey Kulikov

    Andrey Kulikov

    1 year ago
    could you show the code change which caused such diff in screenshots? or what you are saying is there were no changes?
    jim

    jim

    1 year ago
    Specifically, I'm not sure how to parse this sentence "When using a shape for the component background the rendering result is not the changes a little bit..."
    Pedro Gomez

    Pedro Gomez

    1 year ago
    the code is the same, it's just a box with a rounded corner Shape
    @Composable
    fun Skeleton(color: Color = MaterialTheme.colors.surface) {
        Box(Modifier.background(color = color, shape = RoundedCornerShape(size = 18.dp)).fillMaxSize())
    }
    The only difference is where the tests are being executed. In my laptop or the GitHub actions. Sorry @jim, part of the sentence is missing. What I mean is there are no code changes between the execution of this test in local and the CI environment. However, as you can see in the screenshot, the rendering result is not the same. The shape changes a little bit.
    @ppvi this is the screenshot I mentioned by DM
    ppvi

    ppvi

    1 year ago
    GitHub actions use a very small screen for tests, how are you checking the emulator params?
    Pedro Gomez

    Pedro Gomez

    1 year ago
    the emulator is the same in CI and my local environment. I use a script like this to create the emulator: https://github.com/Karumi/Shot/blob/master/shot-consumer/scripts/create_emulator.sh and one like this to start it https://github.com/Karumi/Shot/blob/master/shot-consumer/scripts/start_emulator.sh
    I think I'm going to create a branch reproducing the error. it's going to be easier xD
    ppvi

    ppvi

    1 year ago
    does Shot compare screenshots even if they have different sizes? or would that throw?
    Pedro Gomez

    Pedro Gomez

    1 year ago
    nope. if the screenshot has a different size Shot will fail with a different message. It sais the screenshots size should match before comparing them
    ppvi

    ppvi

    1 year ago
    does this happen with every single test you run on CI?
    Pedro Gomez

    Pedro Gomez

    1 year ago
    yes. I think I can get a working example soon
    @cb this is the interesting issue I've found 🙂
    another example
    ppvi

    ppvi

    1 year ago
    Can you downgrade to whatever compose alpha you used to generate the golden images and check?
    although the fact that it passes locally is strange
    can you link the PR?
    Pedro Gomez

    Pedro Gomez

    1 year ago
    ready @ppvi I was working on it. I've created a PR with a simple component you can find here: https://github.com/Karumi/Shot/blob/compose-reproducible-rendering-issue/shot-consumer-compose/app/src/main/java/com/karumi/shotconsumercompose/MainActivity.kt#L52. Once the component was ready, I've created a screenshot test for this component https://github.com/Karumi/Shot/blob/compose-reproducible-rendering-issue/shot-consumer-compose/app/src/androidTest/java/com/karumi/shotconsumercompose/RoundedCornersBoxScreenshotTest.kt and pushed the code so we can see CI failing. You can find the CI build failure here: https://github.com/Karumi/Shot/runs/1252791041?check_suite_focus=true If you download the artifact named "Reports" from the github action you'll find the HTML report with the images inside. You'll need to use a tool to diff images because the difference is so small you will not notice it without the tool. I'm going to attach the diff here anyway:
    jim

    jim

    1 year ago
    Seems like a graphics-level difference. Maybe @Nader Jawad or @romainguy would have an idea? Also cc @Filip Pavlis in case he has seen something similar with golden files.
    Specifically, I'm speculating that maybe there is some hardware-accelerated graphics involved and so platform differences might result in subtle pixel differences? But I know nothing about how that works, so maybe I'm just talking nonsense.
    ppvi

    ppvi

    1 year ago
    that's my speculation as well but these are emulators! Do they delegate rendering to the underlying OS somehow?
    Pedro Gomez

    Pedro Gomez

    1 year ago
    🤔 let's see what they say
    jim

    jim

    1 year ago
    @Pedro Gomez Perhaps try playing with the
    -gpu <mode>
    as described here: https://developer.android.com/studio/run/emulator-acceleration#command-gpu
    Specifically, I'd speculate that running in software-mode would be more consistent than
    auto
    or a hardware mode.
    Pedro Gomez

    Pedro Gomez

    1 year ago
    @jim what's the mode you want me to try? the choices are : auto, host, swiftshader_indirect, angle_indirect, and guest
    ppvi

    ppvi

    1 year ago
    I'd start with
    guest
    Andrey Kulikov

    Andrey Kulikov

    1 year ago
    what we basically do is just a regular Android's
    canvas.drawRoundRect
    with antialised
    Paint
    . So the issue should be not even Compose specific. I guess different emulators you use do antialising differently
    Pedro Gomez

    Pedro Gomez

    1 year ago
    🤔 @Andrey Kulikov the emulators configured are the same, the only thing changing is the environment. In local I use a mac book pro and CI uses github actions with a macos image
    gpu mode = guest it does not work 😞 @Andrey Kulikov is there any chance the team who develop the method captureToBitmap in the compose testing library is able to provide a version w/o antialising?
    ppvi

    ppvi

    1 year ago
    does
    compose-reproducible-rendering-issue
    need a repofolder with the snapshot artifact?
    Pedro Gomez

    Pedro Gomez

    1 year ago
    if you run ./gradlew uploadArchives from the root folder it will generate the repo folder for you.
    @ppvi look at this build! https://github.com/Karumi/Shot/runs/1253306441?check_suite_focus=true looks like the gpu mode configured as guest is working
    ppvi

    ppvi

    1 year ago
    after updating it to AGP alpha13 and compose alpha04 the test failed on Linux
    so it might be a Linux/mac thing. We're seeing the same behavior in the Rally sample cc @Manuel Vivo
    spoke too soon, the error is
    <http://java.io|java.io>.IOException: No such file or directory
    Pedro Gomez

    Pedro Gomez

    1 year ago
    @ppvi do you know if it is expected to see a black screen emulator when you start it in guest mode?
    ppvi

    ppvi

    1 year ago
    no idea
    r

    romainguy

    1 year ago
    Different GPUs and different CPUs might introduce small differences in rendering. You never want to do exact pixel to pixel comparisons, but use some kind of difference threshold (our cts tests are great examples of that)
    And it's not antialiasing specific. It will happen with gradients, etc.
    ppvi

    ppvi

    1 year ago
    so emulators in mac vs linux can render different things?
    r

    romainguy

    1 year ago
    Technically even on Linux vs Linux
    And technically even on a single emulator you may get different results depending on interpreted vs JIT vs AOT (yay floats)
    ppvi

    ppvi

    1 year ago
    so what do you think about starting the emulator in gpu mode = guest?
    r

    romainguy

    1 year ago
    In CI it’s probably what you want anyway because host requires a physical GPU
    I believe
    swiftshader_indirect
    is a better solution than
    guest
    though
    But no matter what you do, don’t do exact pixel comparisons
    It can fail for sooo many reasons (dithering for instance)
    nickbutcher

    nickbutcher

    1 year ago
    Are the emulators running on the same API level? As Compose uses skia to render and the ska version varies on API levels you can get small rendering issues like this. We have the same issue in VectorDrawable tests and have to add new golden images on different API levels.
    ppvi

    ppvi

    1 year ago
    yeah Pedro confirmed it's exactly the same emulator because it's launched with a script
    Pedro Gomez

    Pedro Gomez

    1 year ago
    As a temporal workaroudn we've implemented tolerance in our images diffing algorithm so we consider both screenshots are correct if 99% of the pixels are equals. Thank you all for your time!!!
    r

    romainguy

    1 year ago
    We do this in all graphics CTS tests of the platform
    Pedro Gomez

    Pedro Gomez

    1 year ago
    Just to let you know, implemeting tolerance in our diffing algorithm seems to be the best solution. The rendering diff shows that around 0.4% of the pixels can be different depending on where the tests are executed and the UI implemented. I used an app I'm working on as a reference.