https://kotlinlang.org logo
#compose
Title
# compose
j

Jez

03/13/2023, 7:51 PM
Has anyone tried using runtime shaders on a Jetpack Compose view element? I've been getting really poor performance trying to animate over time probably because there's limitations that causes per-frame recompositions and shader recreations and I'm not sure what the correct pattern for this should be.
r

romainguy

03/13/2023, 8:38 PM
Could you show your code? If you are only updating the shader uniforms it shouldn't be expensive (beyond the cost of the shader itself; maybe that's the issue?)
j

Jez

03/13/2023, 9:26 PM
Hi Romain, it might be the shader. I've adapted this from a shadertoy "sphere of dots" example:
Copy code
const val SphereShaderSource = """
uniform shader composable;
uniform float iTime;
uniform float2 iResolution;

half4 main(float2 fragCoord) {
    vec2 p,c,u=fragCoord*2.-iResolution;
    half4 o = half4(0.,0.,0.,1.);
    for(float i=0;i<4e2;i++) {
//    for(float i=100;i<101;i++) {
        float a=i/2e2-1.;
        p = cos(i*2.4 + iTime + vec2(0.,11.))*sqrt(1.-a*a);
        c = u/iResolution.y+vec2(p.x,a)/(p.y+2.);
        o +=(cos(i+half4(0.,2.,4.,0.))+1.)/dot(c,c)*(1.-p.y)/3e4;
    }
    return o;
}
"""
I'm executing it in a simple activity like this:
Copy code
class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        val shader = RuntimeShader(SphereShaderSource)
        setContent {
            EmptyComposeTheme {
                // A surface container using the 'background' color from the theme
                Surface(
                    color = MaterialTheme.colors.background
                ) {
                    BoxWithConstraints(modifier = Modifier.fillMaxSize()) {
                        ShaderSurface(shader)
                    }
                }
            }
        }
    }

    @Composable
    fun ShaderSurface(shader: RuntimeShader) {
        val time by produceState(0f) {
            while (true) {
                withInfiniteAnimationFrameMillis {
                    value = it / 1000f
                }
            }
        }

        Surface(
            modifier = Modifier
                .fillMaxSize()
                .onSizeChanged { size ->
                    shader.setFloatUniform(
                        "iResolution",
                        size.width.toFloat(),
                        size.height.toFloat()
                    )
                    shader.setLocalMatrix(
                        Matrix().apply {
                            postScale(1f, -1f)
                            postTranslate(0f, size.height.toFloat())
                        }
                    )
                }
                .graphicsLayer {
                    shader.setFloatUniform("iTime", time)
                    renderEffect = RenderEffect.createRuntimeShaderEffect(
                        shader,
                        "composable"
                    ).asComposeRenderEffect()
                }
        ) {}
    }
}
With this setup I get about 1 frame rendered per second. I tried adjusting the loop in the shader to only draw one point, see commented-out line, and it still only ran at about 5 fps, and I can see some jitter in the movement.
(when I first asked this question I was also declaring the runtime shader inside the ShaderSurface composable, and that really didn't work XD )
r

romainguy

03/13/2023, 9:31 PM
I believe the problem is where you are setting the float uniform, you are causing recompositions there
Of a graphics layer + the render effect
j

Jez

03/13/2023, 9:42 PM
hmm, I was concerned about that myself, but the docs on Modifer.graphics layer say:
Copy code
graphicsLayer can be used to apply effects to content, such as scaling, rotation, opacity, shadow, and clipping. Prefer this version when you have layer properties backed by a androidx.compose.runtime.State or an animated value as reading a state inside block will only cause the layer properties update without triggering recomposition and relayout.
what's more this is the method shown in Rebecca Franks' jellyfish example (ctrl-f for "createRuntimeShaderEffect")
unless I've messed something up with the
produceState()
in some way?
r

romainguy

03/13/2023, 9:52 PM
@Jez oh yeah nvm you are using the lambda version, I missed that
I just profiled your app
and on my M1 emulator (after making sure it’s using hardware acceleration and not software rendering which may happen in “automatic” mode on M1), all the time is spent rendering
So the issue isn’t Compose here
387 microseconds spend recomposing, 90ms spent rendering on the GPU…
Drawing 1 sphere is perfectly smooth
So check your emulator settings, and draw fewer spheres
j

Jez

03/15/2023, 9:43 AM
Thank you for going to those lengths. I've been testing on a physical pixel 5 (Intel processor in my computer precluding emulator usage), so the performance issues I've been seeing are realistic. Guess phone GPUs aren't as powerful as desktop ones, who'd have thought? ;) Really helpful to understand that I'm not making a compose framework mistake though, appreciate that. I've been finding it a challenge to port shaders from GLSL to Android - there seems to be all sorts of weird edge cases, like in one shader I can't access an array by an index during a loop because the index isn't constant. This appears to be a limitation of a certain version of GLSL but also kinda feels like a pretty fundamental thing to be able to do XD Why are shaders never straightforward?!
r

romainguy

03/15/2023, 7:37 PM
It's because Skia needs to target OpenGL ES 2.0
It comes with many limitations
299 Views