Any opinions on snapshot testing for a greenfield ...

# compose-android

[eygraber]

Any opinions on snapshot testing for a greenfield project?

1️⃣ Paparazzi
2️⃣ Roborazzi
3️⃣ Compose Preview Screenshot Testing

[agrosner]

4, all options have their own tradeoffs 😫 😂

[Sergey Y.]

Roborazzi supports hardware-accelerated rendering, which is especially useful for layouts with elevated shadows or when you need to avoid clipping to bounds.

[Albert Chang]

My thoughts:
• Roborazzi is faster.
• Paparazzi supports snapshots of scrollable content (similar to the scrolling screenshot feature of Android).
• If you want to take snapshot after some UI action, Roborazzi is the only choice.
• Parapazzi actually supports hardware rendering better than Roborazzi. Roborazzi (or we should say Robolectric) introduced hardware mode recently but there are still some issues with shadows, blur effect, etc.

[agrosner]

Did a test on my company project with thousands of snapshot tests and found roborazzi much slower than paparazzi. Like 30-40% 😅

[kenkyee]

That's probably because it spins up Robolectric...that's much slower than unit tests in general. We've seen 10x slowness 😞

[agrosner]

yeah exactly. it has to do a lot more heavy lifting to even get off the ground…itll be much slower than unit tests that only spin up whats needed

[agrosner]

Here are the following pros and cons of each ive found with 2 years on paparazzi specifically, so take with grain of salt on other options.
(1). Pros: fast, cheap, easy to do parameterized tests. Easy to configure and get going. Cons: no way to say “only regenerate snaps that would fail”,. if any drift under the threshold, they get included in a record call. The test reports leave a lot to be desired when viewing the XML output, there should be the delta images included but none. also memory leak bugs (been since fixed), and other bugs popping up. Specifically some screenshots render 1px wide with a very tall layout (looks like a line), due to some discrepencies between composables and the real device rendering. Also views that require launched effects or delayed rendering do not work, you need to pass parameters to change the functionality to go in a single render. Otherwise relatively happy with it.
(2). Roborazzi: same pipeline as robolectric, can access a bunch of things you shouldnt in a unit test (android depedencies). Better test configuration options - can use default test task to run them vs paparazzi requires separate task. Has a way to verify and record changes, this is nice plz paparazzi include it 😇 . More options in general it seems. Cons: slow, and requires annotations for configurations which might provide less flexibility, though its standard robolectric APIs.
(3): Compose preview screenshot testing. cons: requires previews to exist in a separate source set, decoupling them from their source files, this makes previews a lot less useful. We combine our test data for component states through demos in a debug screen, snapshot tests of those same ones, robolectric interactive tests (click verification, etc) and previews for developers during development to reduce overall non-production footprint. making them in separate sources is not great.

[Albert Chang]

Normal unit tests are fast, but Paparazzi snapshot tests are not, and are slower than Roborazzi. At least that's what I see in my company project with the latest versions of both libraries. I'm not 100% sure but I feel that the latest version of Paparazzi (with the latest layoutlib) became slower.

[eygraber]

I'm kind of leaning towards Compose Preview Screenshot Testing because my currently plan is to only test with static screens (i.e. no clicking or scrolling), and I'm fine with having test previews in a separate source set since I'd still be keeping a "regular" preview together with the actual composable.

[eygraber]

4, all options have their own tradeoffs 😫 😂

I tried them all out today.

You weren't kidding 😅

[takahirom]

For those who feel Robolectric is slow, it’s worth trying this. The tests are twice as fast as before when using this in my project. However, the fix in Robolectric will be available in the next version, so you can wait for it.
https://github.com/robolectric/robolectric/issues/9043#issuecomment-2217319164