https://kotlinlang.org
Join Slack
Hi, There's a significant difference between Compose Desktop and Swing in term of latency, is there ...
# compose-desktop
m
Hi, There's a significant difference between Compose Desktop and Swing in term of latency, is there any advices how I can improve the Compose version more? I will provide a sample code soon. But nothing special, I'm just drawing a path. I'm planning to try a skiko implementation and see the difference.
Screen Recording 2024-04-18 at 4.29.50 PM.mov
Screenshot 2024-04-18 at 4.30.43 PM.png,Screenshot 2024-04-18 at 4.30.55 PM.png
m
Which method do you use to get the mouse pointer location?
m
I did some investigations to check whether the delay in from the input or the rendering, and I'm pretty sure that there's a delay from the rendering. I tried both 
pointerInput
and 
onPointerEvent
but I got the same result. I even tried using the swing mouse listener directly but I keep getting delay on rendering.
m
Do you observe a time delay or a more or less constant offset? I observed the later when I use pointerInput-detectTransformGestures in contrast to pointerInput-awaitEachGesture-drag, for example.
m
I think that it's not a problem of input delay, I tried a skiko awt sample and there's the same delay.
a
Did you figure it out?
c
Have you also played around with touchSlop? I notice I have to set it to almost zero to have minimal delay between the drag and the outcome of the drag
Copy code
// Decrease the touch slop. The default value of too high for desktop
val vc = LocalViewConfiguration.current.withTouchSlop(
    with(LocalDensity.current) { 0.125.dp.toPx() },
)

CompositionLocalProvider(LocalViewConfiguration provides vc) {
    // content
}
a
IIRC touchSlop only affects when the drag gesture begins. After it begins, it doesn’t play a role.
🤔 1
m
Not yet, I will try skija as well. For now I will see if it's possible to render the drawing with Swing while the user is drawing and then render it with Compose when it's done. This would require being able to have a transparent JPanel on top of Compose, it should be possible I guess.
m
The touch slop is what I meant with a constant offset. It is indeed too high for desktop. I did not know that it can be modified and will give that a try. Actually I wonder why you need a touch slop at all for desktop.
a
Without it, every click becomes a drag-and-drop, unless your hand is perfectly stable.
But it shouldn’t cause a constant offset, unless there’s a bug.
m
I see a constant offset like in the pictures above. I can see that it starts with 0 after the mouse press and than increases to some maximum of a few pixels when the mouse is dragged and after that stays constant. I’ll investigate this in more detail today.
a
It’s not constant, I presume. If you stop moving the mouse, the line catches up. It’s just delay.
m
As I said before, I think that it's not just a problem with input. I tried a skiko awt project without Compose and I got a better result but still having some latency.
Copy code
val path = Path()
val skiaLayer = SkiaLayer()
skiaLayer.renderDelegate = SkiaLayerRenderDelegate(skiaLayer, object : SkikoRenderDelegate {
    val paint = Paint().apply {
        color = Color.BLACK
        strokeWidth = 5f
        strokeCap = PaintStrokeCap.ROUND
        strokeJoin = PaintStrokeJoin.ROUND
        pathEffect = PathEffect.makeCorner(5f)
        mode = PaintMode.STROKE
    }
    override fun onRender(canvas: Canvas, width: Int, height: Int, nanoTime: Long) {
        canvas.drawPath(
            path = path,
            paint = paint
        )
    }
})

// Mouse events
override fun mousePressed(e: MouseEvent) {
    path.reset()
    path.moveTo(e.x.toFloat(), e.y.toFloat())
}

override fun mouseReleased(e: MouseEvent) {
    path.reset()
}

override fun mouseDragged(e: MouseEvent) {
    path.lineTo(e.x.toFloat(), e.y.toFloat())
}
Screenshot 2024-04-19 at 8.15.31 AM.png
m
@Alexander Maryanovsky No, not in my case. The gap does not close again. That’s why I tried to distinguish a delay from an offset. But as I said, I’ll investigate that further.
a
Looks like we have a 2-frame lag with vsync on. This is with vsync:
Screen Recording 2024-04-19 at 21.49.59.mov
👀 2
This is with vsync disabled (1-frame lag):
Screen Recording 2024-04-19 at 21.49.16.mov
👀 2
@Igor Demin FYI
Also @Elijah Semyonov
Copy code
fun main() {
//    System.setProperty("skiko.vsync.enabled", "false")

    val path = Path()
    val skiaLayer = SkiaLayer()
    skiaLayer.renderDelegate = SkiaLayerRenderDelegate(skiaLayer, object : SkikoRenderDelegate {
        val paint = Paint().apply {
            color = org.jetbrains.skia.Color.BLACK
            strokeWidth = 5f
            strokeCap = PaintStrokeCap.ROUND
            strokeJoin = PaintStrokeJoin.ROUND
            mode = PaintMode.STROKE
        }
        override fun onRender(canvas: Canvas, width: Int, height: Int, nanoTime: Long) {
            canvas.drawPath(
                path = path,
                paint = paint
            )
            val lastPt = path.lastPt
            if (lastPt.x != 0f) {
                canvas.drawCircle(x = lastPt.x, y = lastPt.y, radius = 6f, paint = paint)
            }
        }
    })

    SwingUtilities.invokeLater {
        val window = JFrame("Skiko example").apply {
            defaultCloseOperation = WindowConstants.EXIT_ON_CLOSE
            preferredSize = Dimension(800, 600)
        }
        skiaLayer.attachTo(window.contentPane)
        skiaLayer.needRedraw()
        skiaLayer.addMouseListener(object: MouseAdapter() {
            override fun mousePressed(e: MouseEvent) {
                path.reset()
                path.moveTo(e.x.toFloat(), e.y.toFloat())
            }

            override fun mouseReleased(e: MouseEvent) {
                path.reset()
            }
        })
        skiaLayer.addMouseMotionListener(object: MouseAdapter() {
            override fun mouseDragged(e: MouseEvent) {
                path.lineTo(e.x.toFloat(), e.y.toFloat())
            }
        })
        window.pack()
        window.isVisible = true
    }

}
m
Btw it's hard to notice when you have 120 fps, it's more noticable on 60fps or lower. I unplug the Mac charger to enable power save mode and get 60fps.
I noticed a big improvement after disabling vsync,
a
Yes. Hopefully Elijah will know how to fix this.
kodee happy 2
m
That will be a big improvement for our product. Thanks!
m
I also investigated this issue a little more. 1. The constant offset was due to a specific problem in my code, so can be ignored here. 2. The time delay has already been described in detail by you. 3. The touch slop is definitely too big on desktop. On my Mac it’s 18.dp (or 36 pixels). It has the noticeable effect that the user believes that objects must first be torn off before they can be moved. This can be very disturbing and makes, e.g., a precise working in a graphics editor very difficult. It is good to know now (I did not know that before) that this can be reduced locally, e.g., just for a canvas or something similar. On my Mac I could even set it to 0 without any negative effect. As I don’t know whether this is the case on all devices I have finally settled on 0.125.dp like propose above by @Chris Sinco [G].
👍 1
m
Is it possible to write some tests to detect this lag and to get the exact amount of latency?
e
We can query CAMetalDrawable actual presentationTime but it will require work to wire specific events causing redraw with the actual screen update.
👍 1
m
I can't confirm yet but I think that the issue is not just in Metal but in other engines as well. This issue is really critical for us, I can help with the investigation so if you have any thoughts about what could be causing it let me know.
Any updates on this?
e
Not yet, needs some internal discussion
a
I’d recommend setting vsync off, at least on macOS. We limit the frame rate to the display’s refresh rate regardless of the vsync setting.
If you experience tearing, let us know.
m
I tried that, it gives better results but specially for low-end devices there’s a significant increase in cpu usage and something there’s some weird behaviour in the ui so we decided to keep vsync enabled.
Yes we experience tearing on some devices.
a
CPU usage when idle, or when redrawing?
m
After some investigations, updating the Canvas after the frame duration, makes the latency so much lower.
But the interesting thing is if I add a loop with 
withFrameMillis
the latency is back 🤔
a
I’m not sure what case we’re talking about anymore. Is this with vsync or without?
m
This is with vsync enabled. Also, this is Compose not Skiko. The main two changes as mentioned above: • Not updating the canvas more than one time in a frame • Not using withFrameMillis inside a loop
You can check the code here: https://github.com/MohamedRejeb/skiko-swing
Screen Recording 2024-05-14 at 6.52.07 AM.mov
Screen Recording 2024-05-14 at 6.52.07 AM.mov
a
That’s very strange
kodee lost 1
66 Views
Open in Slack
PreviousNext
Powered by

kotlinlang

A modern programming language that makes developers happier.

Powered by