I got a problem using SKIA / skiko: I want to crea...

# multiplatform

[Stefan Oltmann]

I got a problem using SKIA / skiko:
I want to create a gray scale image that's the same as created with OpenCV, but it differs.

This way I do it in OpenCV:

fun main() {

    OpenCV.loadLocally()

    val image = Imgcodecs.imread("/Users/sol/Pictures/Testfotos/photo_1.jpg")

    val targetImage = Mat(image.height(), image.width(), image.type())

    Imgproc.cvtColor(image, targetImage, Imgproc.COLOR_RGB2GRAY)

    Imgcodecs.imwrite("photo_1_opencv.png", targetImage)
}

And this is the way I do it in SKIA:

fun main() {

    val file = "/Users/sol/Pictures/Testfotos/photo_1.jpg"

    val image = Image.makeFromEncoded(File(file).readBytes())

    val surface = Surface.makeRasterN32Premul(image.width, image.height)

    surface.canvas.drawImageRect(
        image,
        Rect(0f, 0f, image.width.toFloat(), image.height.toFloat()),
        Paint().apply {
            colorFilter = ColorFilter.makeMatrix(
                ColorMatrix(
                    0.299000F, 0.587000F, 0.114000F, 0F, 0F,
                    0.299000F, 0.587000F, 0.114000F, 0F, 0F,
                    0.299000F, 0.587000F, 0.114000F, 0F, 0F,
                    0F, 0F, 0F, 1F, 0F
                )
            )
        }
    )

    val newImage = surface.makeImageSnapshot()

    File("photo_1_skia.png").writeBytes(
        newImage.encodeToData()!!.bytes
    )
}

According to https://docs.opencv.org/4.x/de/d25/imgproc_color_conversions.html this should be the right color matrix.

But the result differs. The SKIA version is a bit brighter. I want it to have the same pixel values.

What is my mistake here?

[Stefan Oltmann]

I consider this being a bug for now.
https://github.com/JetBrains/skiko/issues/625

[czuckie]

out of interest, do the coloured versions of the image match perfectly? (e.g. read with CV, output with CV, read with skia output with skia)

[Stefan Oltmann]

Yes, if I don't apply color filter or the cvtColor method the result is identical.

[czuckie]

A complete pixel for pixel match?

[czuckie]

(I'm sure identical means exactly the same, but I just want to be 100% sure 😅 )

[Stefan Oltmann]

Yes, complete pixel match according to Beyond Compare

[czuckie]

That rules out something I suppose. I guess if I were to try and narrow it down, I'd work with bitmaps first, maybe even just block R, G and B images to compare the outputs. I think if you could see the difference between the outputs then, it'd be easier to understand what was going wrong perhaps. I hope someone smarter than me comments on this!

[Stefan Oltmann]

I hope so, too. Thanks. 🙂

[Michael Paus]

Just a wild shot into the dark. May there be a problem with the pre-multiplication in the Skia version?

[Stefan Oltmann]

@Michael Paus Thank your for that idea.

I changed my code to the version below, but the resulting image is the same.

fun main() {

    val file = "/Users/sol/Pictures/Testfotos/photo_1.jpg"

    val image = Image.makeFromEncoded(File(file).readBytes())

    val colorInfo = ColorInfo(
        colorType = ColorType.RGBA_8888,
        alphaType = ColorAlphaType.UNPREMUL, // <-- has no effect
        colorSpace = ColorSpace.sRGB
    )

    val imageInfo = ImageInfo(
        colorInfo = colorInfo,
        width = image.width,
        height = image.height
    )

    val surface = Surface.makeRaster(imageInfo)

    surface.canvas.drawImageRect(
        image,
        Rect(0f, 0f, image.width.toFloat(), image.height.toFloat()),
        Paint().apply {
            colorFilter = ColorFilter.makeMatrix(
                ColorMatrix(
                    0.299000F, 0.587000F, 0.114000F, 0F, 0F,
                    0.299000F, 0.587000F, 0.114000F, 0F, 0F,
                    0.299000F, 0.587000F, 0.114000F, 0F, 0F,
                    0F, 0F, 0F, 1F, 0F
                )
            )
        }
    )

    val newImage = surface.makeImageSnapshot()

    File("photo_1_skia.png").writeBytes(
        newImage.encodeToData()!!.bytes
    )
}

[Stefan Oltmann]

Setting ColorType.GRAY_8 does not change the result. If I change the color type and remove the colorFilter, the result is different, but still not the same as OpenCV.

[Stefan Oltmann]

I reported to https://bugs.chromium.org/p/skia/issues/detail?id=13956 . Hope they have an answer if it's not a problem with the binding or just my user error.

[Stefan Oltmann]

Update: User error, SKIA is fine. 🙈

Imgproc.COLOR_RGB2GRAY must be changed to Imgproc.COLOR_BGR2GRAY because OpenCV does load images BGR instead of RGB. That's wild.