Hi all! I'm writing an app and adding screenshot tests for my components and I'm facing some issues with the RoundedCornerShape. When using a shape for the component background the rendering result is not the changes a little bit and this makes the screenshot test tool to report there is an error in the rendering. The shape I'm using is this

RoundedCornerShape(size = 16.dp)

The error reported by the screenshot testing shows a bunch of small different pixels making the tests fail. Red dots in the screenshot are the errors reported by the testing tool. Do you know if the rendering can vary depending on the build or any other param?

could you show the code change which caused such diff in screenshots? or what you are saying is there were no changes?

Specifically, I'm not sure how to parse this sentence "When using a shape for the component background the rendering result is not the changes a little bit..."

the code is the same, it's just a box with a rounded corner Shape

@Composable
fun Skeleton(color: Color = MaterialTheme.colors.surface) {
    Box(Modifier.background(color = color, shape = RoundedCornerShape(size = 18.dp)).fillMaxSize())
}

The only difference is where the tests are being executed. In my laptop or the GitHub actions. Sorry @jim, part of the sentence is missing. What I mean is there are no code changes between the execution of this test in local and the CI environment. However, as you can see in the screenshot, the rendering result is not the same. The shape changes a little bit.

GitHub actions use a very small screen for tests, how are you checking the emulator params?

the emulator is the same in CI and my local environment. I use a script like this to create the emulator: https://github.com/Karumi/Shot/blob/master/shot-consumer/scripts/create_emulator.sh and one like this to start it https://github.com/Karumi/Shot/blob/master/shot-consumer/scripts/start_emulator.sh

does Shot compare screenshots even if they have different sizes? or would that throw?

nope. if the screenshot has a different size Shot will fail with a different message. It sais the screenshots size should match before comparing them

does this happen with every single test you run on CI?

yes. I think I can get a working example soon

Can you downgrade to whatever compose alpha you used to generate the golden images and check?

ready @ppvi I was working on it. I've created a PR with a simple component you can find here: https://github.com/Karumi/Shot/blob/compose-reproducible-rendering-issue/shot-consumer-compose/app/src/main/java/com/karumi/shotconsumercompose/MainActivity.kt#L52. Once the component was ready, I've created a screenshot test for this component https://github.com/Karumi/Shot/blob/compose-reproducible-rendering-issue/shot-consumer-compose/app/src/androidTest/java/com/karumi/shotconsumercompose/RoundedCornersBoxScreenshotTest.kt and pushed the code so we can see CI failing. You can find the CI build failure here: https://github.com/Karumi/Shot/runs/1252791041?check_suite_focus=true If you download the artifact named "Reports" from the github action you'll find the HTML report with the images inside. You'll need to use a tool to diff images because the difference is so small you will not notice it without the tool. I'm going to attach the diff here anyway:

Seems like a graphics-level difference. Maybe @Nader Jawad or @romainguy would have an idea? Also cc @Filip Pavlis in case he has seen something similar with golden files.

that's my speculation as well but these are emulators! Do they delegate rendering to the underlying OS somehow?

🤔 let's see what they say

@Pedro Gomez Perhaps try playing with the

-gpu <mode>

as described here: https://developer.android.com/studio/run/emulator-acceleration#command-gpu

@jim what's the mode you want me to try? the choices are : auto, host, swiftshader_indirect, angle_indirect, and guest

I'd start with

guest

what we basically do is just a regular Android's

canvas.drawRoundRect

with antialised

Paint

. So the issue should be not even Compose specific. I guess different emulators you use do antialising differently

🤔 @Andrey Kulikov the emulators configured are the same, the only thing changing is the environment. In local I use a mac book pro and CI uses github actions with a macos image

does

compose-reproducible-rendering-issue

need a `repo`folder with the snapshot artifact?

if you run ./gradlew uploadArchives from the root folder it will generate the repo folder for you.

after updating it to AGP alpha13 and compose alpha04 the test failed on Linux

@ppvi do you know if it is expected to see a black screen emulator when you start it in guest mode?

no idea

Different GPUs and different CPUs might introduce small differences in rendering. You never want to do exact pixel to pixel comparisons, but use some kind of difference threshold (our cts tests are great examples of that)

so emulators in mac vs linux can render different things?

Technically even on Linux vs Linux

so what do you think about starting the emulator in gpu mode = guest?

In CI it’s probably what you want anyway because host requires a physical GPU

Are the emulators running on the same API level? As Compose uses skia to render and the ska version varies on API levels you can get small rendering issues like this. We have the same issue in VectorDrawable tests and have to add new golden images on different API levels.

yeah Pedro confirmed it's exactly the same emulator because it's launched with a script

As a temporal workaroudn we've implemented tolerance in our images diffing algorithm so we consider both screenshots are correct if 99% of the pixels are equals. Thank you all for your time!!!

We do something similar in VectorDrawable tests: https://cs.android.com/androidx/platform/frameworks/support/+/androidx-master-dev:vectordrawable/vectordrawable/src/androidTest/java/androidx/vectordrawable/graphics/drawable/tests/VectorDrawableTest.java;l=402

We do this in all graphics CTS tests of the platform

Just to let you know, implemeting tolerance in our diffing algorithm seems to be the best solution. The rendering diff shows that around 0.4% of the pixels can be different depending on where the tests are executed and the UI implemented. I used an app I'm working on as a reference.