I’m implementing Snapshot Testing with Compose, however, when I compare the bitmap from different AP...

# compose

[Jordan Silva]

I’m implementing Snapshot Testing with Compose, however, when I compare the bitmap from different APIs (28 and 30 for example), I get an error saying they don’t match.
So, I checked what was different, and looks like it’s the background color. The first pixel (0,0) gives me a different color using depending on the emulator API version.
Looks like it renders a grey gradient at the very top (API 28 #E0E0E0 and API 30 #E2E2E2).
It is reproducible with the following code.

Box(Modifier.background(Color.White)) {
    Button(onClick = {}) {
        Text(text = "Label")
    }
}

I’m using the Snapshot code example from @ppvi.

[theapache64]

this is expected, and this is one of the major limitations of snapshot testing. The test should be run on the same device/apiLevel the snapshot created.

[Jordan Silva]

I was expecting to test 1 single component isolated. So I could ignore the background behind that component or something like that.
I tried to set a different background and Surface to Color.Cyan for example. But I still can see this gradient and the color changes from E6E6E6 to E5E5E5 (depending on the emulator).
So to workaround this I guess I’ll have to calculate the delta and have a tolerance like @ppvi mentioned.

[josefdolezal]

offtopic: what do you use for snapshot tests?

[Jordan Silva]

I’m building UI components. So I want to check size, shape, color, etc.
So I’m rendering only like a Button component and checking if everything matches.

[Jordan Silva]

I’m still trying to find the best way to do this with Compose. Would be nice if we could navigate through the nodes and check the modifiers for that Node, or something else. (I know it’s for semantics check, but would be nice to have standard modifiers there too).

[Zach Klippenstein (he/him) [MOD]]

Do you mean interactively? Or for test diagnostics? If the latter, you could try https://github.com/square/radiography

[Jordan Silva]

@Zach Klippenstein (he/him) [MOD] I’ll take a look at it. I’m kinda implementing different sets of UI tests to validate custom components. So, it’s not a diagnostic per se, but maybe I can use this library on my tests. 🙂
I’m checking colors, shape, size, font, animation, accessibility (contrast, semantics, navigation states - like if it’s focused) and their behavior (label wrap, RTL, event effect/animation).
And I was trying to avoid using Espresso or UI Automator to do this (no reason). 😄 So, I’m playing with ComposeRule & semanticNodes, and taking components screenshots to check colors etc.

[Zach Klippenstein (he/him) [MOD]]

Hm yea i don’t think the compose test apis were designed to assert at that level of granularity. They can handle semantics and focus, but not sure about color and shape.

[Zach Klippenstein (he/him) [MOD]]

Snapshot testing seems appropriate but stick to a single api level

[Zach Klippenstein (he/him) [MOD]]

There might be a way to write a test comparing radiography strings for this sort of thing, but I am not very certain it can guarantee it won’t include object hashcodes in some cases which would not work for testing.

[Jordan Silva]

Yeah, I know. I managed to validate the size with compose test api. Eventually I’ll need to implement Espresso tests to validate something.
The component itself should have the same look across different API levels. I’m just bumped into a strange gradient at the top of the screenshot (it guess it’s from the Activity created with compose rule). So, this gradient color differs per platform.
I’m implementing Delta E to check the difference between the colors and ignore if it’s not noticeable.

[Jordan Silva]

Does Radiography return colors or other modifiers for Compose?

[Zach Klippenstein (he/him) [MOD]]

I can’t actually remember if it spits out all modifiers by default - I think it does. But you can also customize it to display things however you want so you could probably make it work with a little extra code