https://kotlinlang.org logo
Join Slack
Powered by
I am trying to get the pixels of a `nativeCanvas` ...
# compose-desktop
a
I am trying to get the pixels of a 
nativeCanvas
into a Bitmap during a 
drawWithContent
modifier, but 
readPixels
always returns false, and I don't know why:
Copy code
drawContent()

val nativeCanvas = drawContext.canvas.nativeCanvas
val imageInfo = ImageInfo(
    width = 1280,
    height = 800,
    colorInfo = ColorInfo(
        colorType = ColorType.RGBA_8888,
        alphaType = ColorAlphaType.PREMUL,
        colorSpace = ColorSpace.sRGB
    ),
)

val bitmap = Bitmap()
if (!bitmap.setImageInfo(imageInfo)) {
    println("Couldn't set imageInfo")
}

if (!bitmap.allocPixels()) {
    println("Couldn't allocate pixels")
}

if (!nativeCanvas.readPixels(bitmap, 0, 0)) {
    println("Couldn't read pixels") <-- this one returns false
}
k
Does not copy, and returns false if: • Source and destination rectangles do not intersect. • SkCanvas pixels could not be converted to pixmap.colorType() or pixmap.alphaType(). • SkCanvas pixels are not readable; for instance, SkCanvas is document-based. • SkPixmap pixels could not be allocated. • pixmap.rowBytes() is too small to contain one row of pixels.
a
Yes I read that as well. But it is very difficult for me to check which case failed
k
This one seems to be useful to get the image info from the canvas
Looks like https://github.com/JetBrains/skiko/blob/master/skiko/src/jvmMain/cpp/common/Canvas.cc and https://github.com/JetBrains/skiko/blob/master/skiko/src/commonMain/kotlin/org/jetbrains/skia/Canvas.kt do not have bindings for that function. You can make a local copy of Skiko, add the bindings, build it, deploy to local Maven, and then use the local dependency like this:
Copy code
resolutionStrategy.eachDependency {
            if (requested.group == "org.jetbrains.skiko") {
                useVersion("0.0.0-SNAPSHOT")
                because("Replacing for local development")
            }
        }
It might be that you won’t be able to do this at all. Looking at the source of SkiaLayer.macos.kt at least, it appears that the canvas is indeed created by a 
PictureHolder
image.png
image.png
And very similar in 
SkiaLayer.awt.kt
a
Ah okay 😞, so I was trying to do this for creating a 'screenshot' of a running composable, without using ImageComposeScene. Thanks for your help!
k
Ah, I wasn’t dreaming that I saw that question popping up when I woke up.
a
Haha, no. I deleted it because I thought I had the solution... but I hadn't
k
I would confirm this with @Igor Demin to be sure, but my overall understanding is that a Compose window on desktop is backed by a single Skia canvas, and each composable that puts pixels on the screen is seeing a clipped “part” into that single window-level canvas.
a
I guess that makes sense. The question then is.... can I access the single Skia canvas (which is the 
nativeCanvas
maybe?) and get these contents. However, in your Aurora repository I also saw some sort of screenshot functionality if I am not mistaken?
k
a
Yes, that's the one. If I am understanding it correctly it is recreating the whole composable tree of your app inside the ImageComposeScene
k
I wouldn’t say recreating. The way I see it conceptually that for taking screenshots, I place an intermediate layer that wraps the content of the whole window, and that layer has support for rendering itself into an offscreen bitmap.
Maybe if your point is that using 
ImageComposeScene
needs that intermediate layer somewhere in your real app, see if 
renderComposeScene
might fit your needs better,
image.png
a
I'll need to investigate your code better I think. What I am searching for is to have a running compose application and have a way to capture the content of (a part of) the running application into an Image.
Eventually I solved it with a simple 
ScreenshotContainer
which takes a special state object to trigger screenshots and a content lambda. When the screenshot is taken the content lambda is drawn inside an 
ImageComposeScrene
, which results in:
Screen Recording 2023-02-24 at 09.16.32.mov
Copy code
class ScreenshotState {

    private val _trigger = MutableSharedFlow<Unit>(
        replay = 1,
        extraBufferCapacity = 0,
        onBufferOverflow = BufferOverflow.DROP_OLDEST
    )

    val trigger = _trigger.asSharedFlow()

    private val _latestScreenShot = MutableStateFlow<ImageBitmap?>(null)
    val latestScreenShot = _latestScreenShot.asStateFlow()

    fun takeScreenshot() {
        _trigger.tryEmit(Unit)
    }

    fun renderContent(
        width: Int,
        height: Int,
        density: Density,
        content: @Composable () -> Unit
    ) {
        val scene = ImageComposeScene(
            width = width,
            height = height,
            density = density,
            content = content
        )

        _latestScreenShot.update {
            scene.render().toComposeImageBitmap()
        }
    }
}

@Composable
fun ScreenshotContainer(
    screenshotState: ScreenshotState,
    targetDensity: Density = Density(1f),
    content: @Composable () -> Unit
) {
    val density: Density = LocalDensity.current
    var layoutCoordinates: LayoutCoordinates? by remember { mutableStateOf(null) }

    LaunchedEffect(screenshotState) {
        screenshotState.trigger.collect {
            if (layoutCoordinates != null) {
                val scale = targetDensity.density / density.density
                screenshotState.renderContent(
                    width = (layoutCoordinates!!.size.width * scale).toInt(),
                    height = (layoutCoordinates!!.size.height * scale).toInt(),
                    density = targetDensity,
                    content = content
                )
            }
        }
    }

    Box(
        modifier = Modifier
            .onGloballyPositioned { actualLayoutCoordinates ->
                layoutCoordinates = actualLayoutCoordinates
            }
    ) {
        content()
    }
}
42 Views
Open in Slack
PreviousNext
Powered by