For some reason the same code (with same density set and Font loaded from the same TFF) there is a h...

# compose-desktop

[Stefan Oltmann]

For some reason the same code (with same density set and Font loaded from the same TFF) there is a huge difference in Font weight between my local dev machine and CI (GitHub runner, ubuntu-latest)... What causes this? How can I control it?

ci_linux.webplocal_macos.webp

[Stefan Oltmann]

Maybe it's just platform specific. If I run the tests on macos-latest the font don't differ, but the icons still do for some reason.

I guess what I'm looking for is something like the stuff that's needed for Android Emulators (for example swiftshader_indirect).

[Igor Demin]

Differences in rendering on different platforms is a huge pain now when we try to test Compose.

I know multiple techniques, that we can apply now and one solution that we can develop in the future.

The techniques are:
• Prepare screenshots for each platform separately. On Windows/macOs screenshots are usually stable. Can’t tell what is the status on Linux platforms.
• Maybe we can create some Docker container, which we will use everywhere, even for local tests.
• Try to make screenshots without text. Maybe you can add some Boolean that enables/disables all text.
• Don’t compare screenshots pixel-by-pixel. Instead compare screenshots by some similarity index (example).
These are huge trade-offs, but that is the current state unfortunately.

In the future we can provide out of the box solution with rendering via Freetype for text (it will help to draw text the same on all platforms), disabled antialiasing (which can behave differently too), fixed settings, etc. But we haven’t investigated this path yet, and it is hard to implement it on the user side. Maybe you can build a custom build of Skia/Skiko with Freetype and override it for tests, but that is only a guess that it is possible.

[Stefan Oltmann]

Thank you for this exhaustive answer! 🙏

In addition to your points I try to set "SKIKO_RENDER_API" to "SOFTWARE_COMPAT" as I hope this may eliminate nuances between METAL and OPENGL for instance.

[Igor Demin]

SKIKO_RENDER_API doesn’t affect ImageComposeScene, that is always rendered with software renderer.

[Stefan Oltmann]

Oh, okay. Thank you. Then that's not the source of the differences.

[Stefan Oltmann]

Execute the tests in Docker also sounds reasonable. I need to look for an example how to do it.

[Stefan Oltmann]

It's not only text. This seems to be stable if I choose a macOS 12 based CI runner and compare it with my local dev machine running macOS 12.

But for some reason SVG based icons also differ a bit.

I load them this way:

private val imageVectorCache = mutableMapOf<VectorRes, ImageVector>()

@Composable
actual fun painter(res: VectorRes): Painter {

    val imageVector = imageVectorCache.getOrPut(res) {

        val inputSource = object : InputSource() {
            override fun getByteStream() = openResource(res.path)
        }

        loadXmlImageVector(inputSource, LocalDensity.current)
    }

    return rememberVectorPainter(imageVector)
}

private fun openResource(resourcePath: String): InputStream {

    val classLoader = Thread.currentThread().contextClassLoader!!

    return requireNotNull(classLoader.getResourceAsStream(resourcePath)) {
        "Resource $resourcePath not found"
    }
}

[Igor Demin]

I am not sure, but it can be related to the fact that different macOS versions/installations have different color profiles in System preferences -> Display

Also, Skia can use some platform API’s for rasterisation. For example, for antialiasing. But I am not sure about it too.

Also, different platforms/versions can do math differently, and we can have some round error for colors (+-1)

[Stefan Oltmann]

Thank you for the hint.

[saket]

IIRC text rendering will also depend on the jdk. Are you using the same distribution on both ci and your machine?

[Stefan Oltmann]

I got behind what makes the difference.

I use Eclipse Adoptium 17.0.4+8 on both machines, but the macOS GitHub runner is an Intel machine and therefore has the x64 version installed. My dev machine is a MacBook M1 and therefore has the aarch64 installed.

After locally installing the x64 JVM on my dev machine the screenshots fail in the same manner: Subtle differences in the antialiasing of vector images.

[Stefan Oltmann]

I guess my plan is now to use the macOS GitHub runner until we get a platform-independent rendering with Freetype to avoid the relatively huge differences in font rendering. And this subtle color changes between x64 and aarch64 can be compensated by changing my pixel-by-pixel comparison to allow a difference of 10 in the color of a pixel.

[Stefan Oltmann]

It's just what Igor said:

Also, different platforms/versions can do math differently, and we can have some round error for colors (+-1)

[Stefan Oltmann]

Thank you both a lot for figuring this out 🙏

[Stefan Oltmann]

In the end I gave it up doing this on GitHub.
I wrote a comparison logic that allows colors to be rounded a bit.

That crashes the VM by trying to call skia.Bitmap.getColor() on CI.

I take that as the final sign that I'm just not supposed to compare screenshots on CI. 🙈

# A fatal error has been detected by the Java Runtime Environment: 

# 
# SIGSEGV (0xb) at pc=0x000000012f6949dd, pid=23658, tid=6147 
# 

# JRE version: OpenJDK Runtime Environment Temurin-17.0.4+8 (17.0.4+8) (build 17.0.4+8) 

# Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.4+8 (17.0.4+8, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64) 

# Problematic frame: 
# C [libskiko-macos-x64.dylib+0xf69dd] _ZNK8SkPixmap8getColorEii+0x2cd 

Compiled method (n/a) 19811 2768 n 0 org.jetbrains.skia.BitmapKt::Bitmap_nGetColor (native)

[Stefan Oltmann]

I share my code with you, so I didn't write it for nothing.
Hopefully someone has a use for this.

val expectedBitmap = Bitmap.makeFromImage(Image.makeFromEncoded(expectedImageBytes))
val actualBitmap = Bitmap.makeFromImage(Image.makeFromEncoded(actualImageBytes))

if (!expectedBitmap.isEquals(actualBitmap, tolerance = 3)) {

    fail("Image $fileName is not equal to reference image.")
}

private fun Bitmap.isEquals(other: Bitmap, tolerance: Int = 0): Boolean {

    if (this.width != other.width || this.height != other.height)
        return false

    for (x in 0..width) {
        for (y in 0..height) {

            val thisColor = this.getColor(x, y)
            val otherColor = other.getColor(x, y)

            if (thisColor != otherColor) {

                val redDelta = abs(getRed(thisColor) - getRed(otherColor))
                val greenDelta = abs(getGreen(thisColor) - getGreen(otherColor))
                val blueDelta = abs(getBlue(thisColor) - getBlue(otherColor))

                if (redDelta > tolerance || greenDelta > tolerance || blueDelta > tolerance)
                    return false
            }
        }
    }

    return true
}

private fun getRed(pixelValue: Int) =
    pixelValue and (255 shl 16) shr 16

private fun getGreen(pixelValue: Int) =
    pixelValue and (255 shl 8) shr 8

private fun getBlue(pixelValue: Int) =
    pixelValue and 255