https://kotlinlang.org logo
Join Slack
Powered by
Hi all! I'm writing an app and adding screenshot t...
# compose
p
Hi all! I'm writing an app and adding screenshot tests for my components and I'm facing some issues with the RoundedCornerShape. When using a shape for the component background the rendering result is not the changes a little bit and this makes the screenshot test tool to report there is an error in the rendering. The shape I'm using is this
Copy code
RoundedCornerShape(size = 16.dp)
The error reported by the screenshot testing shows a bunch of small different pixels making the tests fail. Red dots in the screenshot are the errors reported by the testing tool. Do you know if the rendering can vary depending on the build or any other param?
a
could you show the code change which caused such diff in screenshots? or what you are saying is there were no changes?
j
Specifically, I'm not sure how to parse this sentence "When using a shape for the component background the rendering result is not the changes a little bit..."
p
the code is the same, it's just a box with a rounded corner Shape
Copy code
@Composable
fun Skeleton(color: Color = MaterialTheme.colors.surface) {
    Box(Modifier.background(color = color, shape = RoundedCornerShape(size = 18.dp)).fillMaxSize())
}
The only difference is where the tests are being executed. In my laptop or the GitHub actions. Sorry @jim, part of the sentence is missing. What I mean is there are no code changes between the execution of this test in local and the CI environment. However, as you can see in the screenshot, the rendering result is not the same. The shape changes a little bit.
@ppvi this is the screenshot I mentioned by DM
p
GitHub actions use a very small screen for tests, how are you checking the emulator params?
p
the emulator is the same in CI and my local environment. I use a script like this to create the emulator: https://github.com/Karumi/Shot/blob/master/shot-consumer/scripts/create_emulator.sh and one like this to start it https://github.com/Karumi/Shot/blob/master/shot-consumer/scripts/start_emulator.sh
I think I'm going to create a branch reproducing the error. it's going to be easier xD
p
does Shot compare screenshots even if they have different sizes? or would that throw?
p
nope. if the screenshot has a different size Shot will fail with a different message. It sais the screenshots size should match before comparing them
p
does this happen with every single test you run on CI?
p
yes. I think I can get a working example soon
@cb this is the interesting issue I've found 🙂
another example
p
Can you downgrade to whatever compose alpha you used to generate the golden images and check?
although the fact that it passes locally is strange
can you link the PR?
p
ready @ppvi I was working on it. I've created a PR with a simple component you can find here: https://github.com/Karumi/Shot/blob/compose-reproducible-rendering-issue/shot-consumer-compose/app/src/main/java/com/karumi/shotconsumercompose/MainActivity.kt#L52. Once the component was ready, I've created a screenshot test for this component https://github.com/Karumi/Shot/blob/compose-reproducible-rendering-issue/shot-consumer-compose/app/src/androidTest/java/com/karumi/shotconsumercompose/RoundedCornersBoxScreenshotTest.kt and pushed the code so we can see CI failing. You can find the CI build failure here: https://github.com/Karumi/Shot/runs/1252791041?check_suite_focus=true If you download the artifact named "Reports" from the github action you'll find the HTML report with the images inside. You'll need to use a tool to diff images because the difference is so small you will not notice it without the tool. I'm going to attach the diff here anyway:
j
Seems like a graphics-level difference. Maybe @Nader Jawad or @romainguy would have an idea? Also cc @Filip Pavlis in case he has seen something similar with golden files.
Specifically, I'm speculating that maybe there is some hardware-accelerated graphics involved and so platform differences might result in subtle pixel differences? But I know nothing about how that works, so maybe I'm just talking nonsense.
p
that's my speculation as well but these are emulators! Do they delegate rendering to the underlying OS somehow?
p
🤔 let's see what they say
j
@Pedro Gomez Perhaps try playing with the 
-gpu <mode>
as described here: https://developer.android.com/studio/run/emulator-acceleration#command-gpu
Specifically, I'd speculate that running in software-mode would be more consistent than 
auto
or a hardware mode.
p
@jim what's the mode you want me to try? the choices are : auto, host, swiftshader_indirect, angle_indirect, and guest
p
I'd start with 
guest
a
what we basically do is just a regular Android's 
canvas.drawRoundRect
with antialised 
Paint
. So the issue should be not even Compose specific. I guess different emulators you use do antialising differently
p
🤔 @Andrey Kulikov the emulators configured are the same, the only thing changing is the environment. In local I use a mac book pro and CI uses github actions with a macos image
gpu mode = guest it does not work 😞 @Andrey Kulikov is there any chance the team who develop the method ``captureToBitmap`` in the compose testing library is able to provide a version w/o antialising?
p
does 
compose-reproducible-rendering-issue
need a `repo`folder with the snapshot artifact?
p
if you run ./gradlew uploadArchives from the root folder it will generate the repo folder for you.
@ppvi look at this build! https://github.com/Karumi/Shot/runs/1253306441?check_suite_focus=true looks like the gpu mode configured as guest is working
p
after updating it to AGP alpha13 and compose alpha04 the test failed on Linux
so it might be a Linux/mac thing. We're seeing the same behavior in the Rally sample cc @Manuel Vivo
spoke too soon, the error is 
<http://java.io|java.io>.IOException: No such file or directory
p
@ppvi do you know if it is expected to see a black screen emulator when you start it in guest mode?
p
no idea
r
Different GPUs and different CPUs might introduce small differences in rendering. You never want to do exact pixel to pixel comparisons, but use some kind of difference threshold (our cts tests are great examples of that)
And it's not antialiasing specific. It will happen with gradients, etc.
p
so emulators in mac vs linux can render different things?
r
Technically even on Linux vs Linux
And technically even on a single emulator you may get different results depending on interpreted vs JIT vs AOT (yay floats)
p
so what do you think about starting the emulator in gpu mode = guest?
r
In CI it’s probably what you want anyway because host requires a physical GPU
I believe 
swiftshader_indirect
is a better solution than 
guest
though
But no matter what you do, don’t do exact pixel comparisons
It can fail for sooo many reasons (dithering for instance)
n
Are the emulators running on the same API level? As Compose uses skia to render and the ska version varies on API levels you can get small rendering issues like this. We have the same issue in VectorDrawable tests and have to add new golden images on different API levels.
p
yeah Pedro confirmed it's exactly the same emulator because it's launched with a script
p
As a temporal workaroudn we've implemented tolerance in our images diffing algorithm so we consider both screenshots are correct if 99% of the pixels are equals. Thank you all for your time!!!
n
r
We do this in all graphics CTS tests of the platform
p
Just to let you know, implemeting tolerance in our diffing algorithm seems to be the best solution. The rendering diff shows that around 0.4% of the pixels can be different depending on where the tests are executed and the UI implemented. I used an app I'm working on as a reference.
7 Views
Open in Slack
PreviousNext
Powered by