What's the right way of applying a 3D transformati...

# compose

[Pierre Schrodinger]

What's the right way of applying a 3D transformation to an image?

[Kirill Grouchnikov]

Affine transformations can’t do this

[Michael Paus]

But who says that Skia can only handle affine transforms? As far as I know you can also use a kind of 3D transforms but I haven’t yet used them myself. See, e.g., https://learn.microsoft.com/en-us/xamarin/xamarin-forms/user-interface/graphics/skiasharp/transforms/3d-rotation

[Michael Paus]

Maybe this is a better link for compose: https://github.com/JetBrains/compose-multiplatform-core/pull/598

[ephemient]

that's still affine rotationX and rotationY affect "Z"

[ephemient]

you can set https://developer.android.com/reference/kotlin/androidx/compose/ui/graphics/Matrix directly

[Pierre Schrodinger]

you can set https://developer.android.com/reference/kotlin/androidx/compose/ui/graphics/Matrix directly

Thanks, but how do I apply it? Can't find any documentation on it whatsoever.

[Pierre Schrodinger]

(for example in my case I have an Image I want to transform)

[Michael Paus]

Have a look at https://developer.android.com/reference/kotlin/androidx/compose/ui/graphics/drawscope/DrawTransform#transform(androidx.compose.ui.graphics.Matrix)

[Pierre Schrodinger]

I tried it out, but it doesn't seem to be able to do 3D transformations. E.g. if I have a rectangle, it will remain at least a parallelogram no matter what I do with it, since you can only translate, rotate and scale. Am I missing something?

[Pierre Schrodinger]

Also it seems like you can only apply those transformations to whatever you can draw on a canvas, which is limited to primitive shapes and images. Is there a way to transform a Button for example?

[Timo Drick]

You could use the Modifier.graphicsLayer. But it also do not directly support affine transformation i think.
Solution could be writing a custom shader.
Of course if you just want to do a transormation of an image you could use it as a image texture inside of a shader and do the transformations in shader code.

[Timo Drick]

You could use AGSL for this but than it will only run on >= Android 13
Or you use opengl

[Michael Paus]

Here is a simple Matrix example which shows that you can do 3D transforms.

import androidx.compose.foundation.layout.*
import androidx.compose.material.MaterialTheme
import androidx.compose.material.Slider
import androidx.compose.runtime.*
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.drawBehind
import androidx.compose.ui.geometry.Size
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.Matrix
import androidx.compose.ui.graphics.drawscope.withTransform
import androidx.compose.ui.unit.dp
import androidx.compose.ui.window.Window
import androidx.compose.ui.window.application

@Composable
fun App() {
    MaterialTheme {
        fun doMyTrafo(rotX: Float): Matrix {
            return Matrix().apply {
                rotateX(rotX)
            }
        }
        var rotX by remember { mutableStateOf(-45f) }
        var matrix by remember { mutableStateOf(doMyTrafo(rotX)) }
        Column(modifier = Modifier.fillMaxSize()) {
            Box(modifier = Modifier.weight(1f).fillMaxWidth().drawBehind {
                withTransform({
                    this.transform(matrix)
                }) {

                    drawRect(
                        color = Color.Gray,
                        size = Size(this.size.width, this.size.height/2f)
                    )
                }
            })
            Slider(
                rotX,
                valueRange = -90f..0f,
                onValueChange = {
                    rotX = it
                    matrix = doMyTrafo(rotX)
                },
                modifier = Modifier.height(40.dp).fillMaxWidth())
        }
    }
}

fun main() = application {
    Window(onCloseRequest = ::exitApplication) {
        App()
    }
}

[Michael Paus]

The result of the above code looks like this:

[Pierre Schrodinger]

@Michael Paus thanks for the code snippet, I tried it, but I'm only getting a rectangle that gets thinner the more you rotate it. I don't see how you got that trapezoid, when the only transformation applied to the rectangle is rotateX. It feels like you're somehow using a perspective camera while mine is orthographic .-.

[Michael Paus]

@Pierre Schrodinger My images above where created with the exact code snipped I posted and nothing else. But your are right - there is a difference depending on the Compose version you use. My results above where created with version 1.4.0 but I just realised myself that the results change if you use version 1.4.1 or the latest 1.5.0-dev1080 instead. This is probably related to this fix https://github.com/JetBrains/compose-multiplatform-core/pull/598 but I have to admit that I think that this change is not correct. I should have a closer look at it. The change also seems to contradict the statements in https://learn.microsoft.com/en-us/xamarin/xamarin-forms/user-interface/graphics/skiasharp/transforms/3d-rotation which has also been referred to by Jetbrains in the past due to lack of proper Skiko documentation.

[Michael Paus]

For what you want to achieve, this seems to be a good tutorial: https://medium.com/mobile-app-development-publication/have-fun-with-jetpack-compose-graphicslayer-modifier-e39c12a4791f I just verified that this code works, with some slight modifications to get rid of the android specifics, for desktop too. With Compose multiplatform 1.4.0 and 1.4.1.