https://kotlinlang.org logo
Join Slack
Powered by
Hi everyone! Is there a way to reuse the content o...
# compose
d
Hi everyone! Is there a way to reuse the content of a Canvas component ? Canvas redraws fully at each re-composition. How should I proceed if I wanted to draw above currently drawn content and clear the canvas on-demand ? I am trying to make an oscilloscope-like gui from a stream of audio data. I want to draw the new content acquired between each animation frames (web lingo, sorry) and maybe clip/move the old content. Should I use an off-screen canvas ?
t
You can just use a Canvas. Compose is optimized to redraw components when the input data changes. So it only makes sense to redraw anything when it is changed. So you do not need to redraw every frame. In you case the input would be the audio data. So just use a mutableState variable that get updated every time you have new data.
j
I'm sort of doing that in https://github.com/joreilly/chip-8 ....the "emulator" (in shared KMP code) is generating "display data" which it's passing back to UI and then using Compose Canvas API to render
d
@Timo Drick I don't get it. My audio signals are real-time. Only the last 16ms (considering the display refresh rate is 60Hz) of my audio data might be new. I don't want to redraw anything drawn before those last 16ms. I just want to move what I've drawn before. If I had to redraw all the data of all my channels, my application would be really slow.
t
Ok i see. No i don't think this is possible. You have to redraw every thing. But it is very fast. So you could just save the data in a variable and redraw anything.
Of course you could also draw into a bitmap and just show the bitmap. Than you can use the Android Canvas to draw onto a bitmap
d
Yeah I think this is what I have to resort to. But I can't find what class from the low-level Compose I have to use.
t
Drawing into a bitmap is independent of Compose.
d
Android supports all of AWT ?
t
Some thing like this:
Copy code
val bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
val canvas = Canvas(bitmap)
canvas.drawLines....
But drawing lines with compose is very fast. So i would recommend to first try to redraw everything and use just the Compose Canvas. Maybe it is fast enough.
d
But 
androidx.compose.ui.graphics.Canvas
doesn't seem to have a way to move pixels or be cleared...
t
Yea you have to redraw everything. Every time it is rendered
d
I am quite sure that redrawing a 5 seconds window of 16 channels of 44kHz audio every 16 ms will not work... 🙂
t
it depends on how many lines you have to draw 😄
But yes than you can draw it into a bitmap. That should work. But you need to increase a counter or s.th. every time draw s.th. into the bitmap to tell compose that s.th. is changed
d
Well, if I resample my audio to the number of pixels on a 4K screen, this means at least 3840 * 16 channels = 61440
t
Maybe you could first do some performace tests with random data. To see if Android is able to handle this.
d
OK, I think I could draw the last 16ms in a bitmap and remember the last X bitmaps (that I need to cover the screen) and then every redraw all those bitmaps in my canvas.
But that is a lot to do when I could use the canvas's own memory.
t
No you could just use one bitmap.
d
How so ?
t
You want that is moves in time?
d
Oh, OK double-buffering.
1. draw bitmap1 to bitmap2 2. draw the new 16ms 3. swap bitmaps
t
But performance wise it should not make a different how many bitmaps you have
Drawing bitmaps is very fast
Just try your idea. It sounds cool to me.
d
OK thanks.
t
Maybe you can recycle the bitmaps. So reuse the bitmaps you do not need anymore
d
No but I will only use 2 bitmaps to do double-buffering.
t
Just create e.g. 10 bitmaps using remember { }
i don't think you have to care about double buffering. Compose/Android do this under the hood for you
d
But I can't draw my canvas above itself, right ?
t
Just make sure that you draw on the Main thread
Hmm. I think i still not really get what you want to do 😄
You can use the Compose Canvas to draw bitmaps. And you can draw multiple bitmaps on top of each other.
d
In an ideal world I would do:
Copy code
Canvas {
  drawImage(thisCanvasUnderlyingBitmap, -pixelsFor16ms, 0)
  // now draw the new 16ms of signal
}
j
@Didier Villevalois have you looked at the repo I mentioned above....I still think based on what you describe that it does something quite similar
d
@John O'Reilly I did look but did not find the corresponding code
j
it renders 
screenData
which is generated by the emulator every 16ms
d
Yeah OK I have seen that code. But it re-renders everything every time.
j
you mentioned above "In an ideal world I would do:"
Copy code
Canvas {
  drawImage(thisCanvasUnderlyingBitmap, -pixelsFor16ms, 0)
  // now draw the new 16ms of signal
}
would that not also re-render every 16ms?
t
Yes but it do not draw the lines. So it depends on how many lines you have to draw. But as i mentioned above. Just try what happens if you draw 61440 lines every frame. Maybe it works.
d
Sure but I would not have to redraw all the previous signal (which means traversing all the previous data and do the computations), apart for when the layout size changes.
t
Maybe you could also optimize the code to reduce the number of lines a little bit
You do not have to do the computations again and a again. You can just save the data.
d
You mean store the results of the computations of the lines I have to draw ?
t
Just store the points for the lines an an array
d
Yeah...
I could do that. I just still thinks it is a lot of hassle when I could rely on what the graphics card has in its memory.
t
Skia uses opengl to draw lines. It is very efficient.
d
Yeah, but my problem is doing the extra-work of storing my computations.
t
Just make sure that you do not create too many new objects every frame
Storing you computations is the same like drawing the lines. What is your problem?
d
In other frameworks, when you redraw, you still have what has been drawn before, you can clip part of it and move it and then draw what you need more.
t
Yes if you really want to do this. Than you have to do the bitmap approach.
d
I think I will try that approach. This is way less code for me :)
Thank you guys.
r
Android uses double or triple buffering, we do have the previous frames available but it is not necessarily efficient to do an old school blit copy like you mention
Have you tried drawing your lines every frames? Is it indeed to slow?
And have you tried using drawLines() instead of drawLine()?
This will become a single GPU draw call
d
@romainguy No, I did not try yet. I am still assessing feasability and designing.
I have no doubt that drawing lines is efficient given any hardware-acceleration. My problem is the cost of the computations to determine the line to draw and/or the added effort of having to store the results of these computations.
@romainguy Is there a reason you do not provide access to the previous frames ? API complexity ?
r
They are not in a format that's usable for an app
Plus the complexity (there might not be a previous frame, it might be in a different orientation etc.)
Note that Compose and the View toolkit only redraw the part of the screen that changed though
I'd be surprised if keeping the previous lines would be more expensive/complicated than juggling bitmaps, esp. if you need to handle layout resizes
t
I did a small poc for this:
Copy code
const val historySize = 500

fun fract(x: Float): Float = x - floor(x)
fun noise(c: Long): Float = fract(sin(c.toFloat()*100f) * 5647f)

class SignalMemory {
    private val data = Array(historySize) { 0f }
    private var pointer = 0
    val ts = mutableStateOf(0L)

    fun addData(value: Float) {
        data[pointer] = value
        pointer = (pointer+1) % data.size
        ts.value += 1
    }
    fun size() = data.size
    fun get(i: Int): Float = data[(pointer + i) % data.size]

    private var running = false
    fun start() {
        GlobalScope.launch {
            running = true
            while (running) {
                addData(noise(ts.value))
                delay(33)
            }
        }
    }
    fun stop() {
        running = false
    }
}

@Composable
fun lineTest() {
    val mem = remember { SignalMemory() }
    onActive {
        mem.start()
        onDispose {
            mem.stop()
        }
    }
    Canvas(Modifier.fillMaxSize()) {
        val size = drawContext.size
        val stepSize = size.width / mem.size().toFloat()
        val ts = mem.ts
        val currentData = ts.value // just to track data changes
        drawIntoCanvas {
            it.scale(stepSize, -(size.height / 2f))
            it.translate(0f, -1f)
            for (i in 0..mem.size()) {
                val x = i.toFloat()
                val v = mem.get(i)
                drawLine(Color.Red, Offset(x, -v), Offset(x, v), strokeWidth = 1f)
            }
        }
    }
}
With 500 lines on a Pixel 3a it is ok but at the limit. Maybe some one could help to optimize the code.
device-2021-01-18-003230.mp4
r
Don't issue individual drawLine calls, instead do a drawLines() with a single array of lines.
t
Did not found this call. In which API is it?
r
It's on the native Canvas API
Use .nativeCanvas.drawLines()
t
ok thx. I will try this
r
(which may require a cast to android.graphics.Canvas)
t
It is a little bit better:
Copy code
Canvas(Modifier.fillMaxSize()) {
        val size = drawContext.size
        val stepSize = size.width / mem.size().toFloat()
        val ts = mem.ts
        val currentData = ts.value // just to track data changes
        val paint = Paint().apply {
            strokeWidth = 1f
            isAntiAlias = true
        }
        drawIntoCanvas {
            it.scale(stepSize, -(size.height / 2f))
            it.translate(0f, -1f)
            val lineArray = FloatArray(mem.size() * 4)
            for (i in 0 until mem.size()) {
                val x = i.toFloat()
                val v = mem.get(i)
                val offset = i * 4
                lineArray[offset] = x
                lineArray[offset + 1] = -v
                lineArray[offset + 2] = x
                lineArray[offset + 3] = v
            }
            it.nativeCanvas.drawLines(lineArray, paint)
        }
    }
Maybe i should also move the FloatArray to the other class so it not get allocated in each frame
r
You could also keep the array preallocated instead of allocating it every time
😄 1
Ah what you just said yes :)
t
ok yes that helps
thx @romainguy
l
Is there a plan to have 
drawLines
be part of the Compose UI Canvas API?
r
Feature request for @Nader Jawad 🙂
n
It exists, it's just labeled as 
drawPoints
and pass in 
PointMode.Lines
for the corresponding 
PointMode
l
Why not also have a specialized 
drawLines
?
r
@Nader Jawad TIL!
👍 1
n
@louiscad, it does end up calling the framework's 
drawLines
API behind the scenes and helps simplify the usage of the 
stepBy
parameter to be more explicit of the intent which could be used either to sets of lines between pairs of points using PointMode.Lines or draw each line connected to create a polygon using PointModes.Polygon
👍 1
r
@Nader Jawad Which is closer to my ideal Canvas API which would basically be more like what we do in 3D: 
render(Renderable, Mask, Shader, RasterState)
or something like this
Where 
Renderable
would be a set of vertices + how you interpret them (points/lines/line strips/triangles, etc.) or parametric shapes (Path, circle, etc.)
l
I see, thank you @Nader Jawad. Wouldn't it make sense to introduce an error level deprecated 
drawLines
function, akin to deprecated operators in kotlinx.coroutines for people that know the RxJava APIs?
n
Which 
drawLines
method are you referring to in this case? We don't expose a drawLines method in compose. I'm not sure it makes sense to deprecate the one in the framework either
l
kotlinx.coroutines isn't deprecating the RxJava operators either. What it's doing is using the deprecated symbols for its own types to hint the developers towards the one with a different naming. For Compose UI, it'd mean that there's a 
drawLines
function that is deprecated at error level (instead of none currently), with a 
ReplaceWith
clause that uses 
drawPoints
, and a deprecation message that explains why 
drawLines
from Compose UI is deprecated. I didn't meant to deprecate the one from Android, we're talking about the API of Compose UI here. Since it'd be deprecated at error level, it wouldn't need any implementation.
n
There isn't a drawLines method to deprecate so it would be odd to introduce it only to deprecate it immediately
👆 1
l
I don't find it odd, I find it useful for people used to the 
drawLines
function on Android or other platforms that have it.
16 Views
Open in Slack
PreviousNext
Powered by