No doubt it’s my fault, but I find Compose impossi...

# compose

[min]

No doubt it’s my fault, but I find Compose impossible to understand. I’d be enormously grateful if anybody could help me out with the following questions:
1. What is CompositionContext? The docs say that it ‘links compositions’, but what does it mean for compositions to be linked?
a. What’s the composition: Composition? field for? What does it mean for a CompositionContext to reference a Composition? Does it make sense for a Composition to be referenced by multiple CompositionContext objects? Or is it expected to be a 1 to 1 relationship? Why? What does it signify? Can a CompositionContext outlive its Composition? Should they be invalidated at the same time?
b. I see that in ComposeView, a Recomposer: CompositionContext is set up to observe global snapshot applies. It ‘parents’ the composition, even though the recomposer doesn’t reference it (?). What’s CompositionContextImpl for?
2. What’s the difference between a CompositionContext and a Composer? I see that CompositionImpl.setContent calls Recomposer.composeInitial, and every time we recompose, Recomposer.composing is called. I know that a Composer is injected into every @Composable, but what division of work does that achieve? A CompositionContext is for what, and a Composer is for what?
3. What is a Composition? A CompositionImpl has a invalidationDelegate: CompositionImpl? field. I thought compositions were ‘linked’ (whatever that means) via contexts? What does it mean for a composition to own another?
I don’t expect to be spoonfed the answers, and trust me, I’ve been reading the compose source code for a while now. Attached are my notes that I’ve been able to put together myself. But as you might know if you’re familiar with the codebase, almost everything references each other at all times, and everything entails passing calls back and forth between everything. Nothing is isolated into discrete logical units, and there’s no way to understand anything without understanding everything. I’m stuck and unable to make much progress, and would appreciate any insights.

compose.html

[Mark Murphy]

I find Compose impossible to understand

What you are seeking are details of Compose internals. Please bear in mind that the vast majority of Compose developers do not need this.

That said, there are existing resources that cover this material, such as Jorge Castillo's book.

[min]

@Mark Murphy I am referencing the book but I’m still unable to figure out the answers to my questions. I’d be grateful if you could provide any insights

[Mark Murphy]

Personally, I can't. I decided a long time ago that I was happy treating Compose as largely a magic black box and focused on how to use it rather than how the magic worked.

[min]

I agree it’s not necessary to know how a fridge works to keep food in it, but there are levels to it. I am able to cobble together simple reactive and responsive UIs using Compose, but my understanding of the underlying machinery is unclear to the point where it feels like I’m adding salt to my dish because the recipe tells me to without knowing what it even tastes like. I don’t mean to get philosophical about the nature of saltiness, but I’d at least like to know that salt is salty. So my questions remain: what are CompositionContext, Composer, and Composition? It’s okay if you don’t know the answers, but I’d appreciate it if somebody who does could tell me

[min]

image.png,image.png

[min]

Is a CompositionContext not a registry of Composition and Composer objects? Also, if a Composer is meant to be ephemeral, why does CompositionImpl have a nonreassignable composer: ComposerImpl field?

[yschimke]

The APIs you are asking about are really more for the implementators of a Compose based system on top of the general Compose Runtime. I'd suggest if you want to learn this level you might start with a simpler smaller compose system like Jake's Mosaic.

[Kirill Grouchnikov]

Depending on what the end goal is here (write your own book about Compose internals? write some sort of a separate system that is in the same rough vein as Mosaic? gain a deeper understanding how the internals work for your own knowledge of how a reactive toolkit works? something else?) - it might be that the best you can get is to track closely the commit history of https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:compose/runtime/runtime/src/;l=1;bpv=1 and, over time, build the mapping from CL descriptions to the specific places in the runtime module code that are touched.

Some of your questions are very deep in the internal implementation details that may - and frequently do - change over time. Without speaking for the core Compose team, some of the answers might just be "it was the most expedient thing to do at the moment".

[shikasd]

SlotTable holds the data about groups and slots for composition
Composer is an abstraction that modifies and reads SlotTable. It is responsible for handling startGroup / endGroup calls, so the slot table remains in the correct shape.
Composition glues together Composer , Applier, state observations and other things that composition does.
CompositionContext is an abstraction over parent-child relationship between compositions. Recomposer is a root driver that keeps the clock running, and most methods in the context just delegate there, but some things like CompositionLocalMap needs to be passed through when creating subcompositions.

[shikasd]

As a person who has spent a few months to understand the system from scratch for fun (before joining the team), I'd recommend to avoid going deep in every topic and ensure there's an understanding of how the whole thing fits together. It only gets worse as you go up the stack to LayoutNode etc.

You don't need to understand every single detail about composition, states and others to efficiently use those APIs. It is useful no doubt, but it simply takes too much time to understand every little edge case that was covered when implementing some of those elements.

In many cases, reading commit messages should give you some history for the decisions if you are interested. Some more tricky lines (cough.. derived state.... cough..) have layers of those decisions that are hard to explain without looking at the tests.

[min]

@shikasd Thanks for the detailed response, but I have a couple followups if you don’t mind
> CompositionContext is an abstraction over parent-child relationship between compositions.
What does this mean? In a setup with parents and children, one would expect to see something along the lines of
• struct Parent { children: Vec<Child>, /* ... */ }
• Or struct Child { parent: &Parent, /* ... */ }
What is this CompositionContext abstraction, how does it work, and why does it exist? Also would it be fair to say that a CompositionContext tracks the validity of every registered Composition?
> creating subcompositions
Also, I’ve been led to believe that there can be a tree of compositions that’s arbitrarily deep. But when I read the code, there seems to only be a flat tree rooted by a Recomposer with which Composition children are registered. This is another thing that confuses me. I’m never able to find code that matches what’s explained to me in English prose.

composition = setContent(resolveParentCompositionContext()) { Content() }

I know that this in AbstractComposeView.ensureCompositionCreated creates a Recomposer for the view tree. The call graph reaches View.createLifecycleAwareWindowRecomposer,

val recomposer =
    Recomposer(contextWithClockAndMotionScale).also { it.pauseCompositionFrameClock() }

Where a Recomposer is constructed,

viewTreeLifecycle.addObserver(
    object : LifecycleEventObserver {
        override fun onStateChanged(source: LifecycleOwner, event: Lifecycle.Event) {
            val self = this
            when (event) {
                Lifecycle.Event.ON_CREATE -> {
                    runRecomposeScope.launch(start = CoroutineStart.UNDISPATCHED) {
                        var durationScaleJob: Job? = null
                        try {
                            // ...
                            recomposer.runRecomposeAndApplyChanges()

And once the view tree has been created, .runRecomposeAndApplyChanges() is called on it in the same CoroutineContext passed to the Recomposer constructor.

recompositionRunner { parentFrameClock ->
    /* ... */

    while (shouldKeepRecomposing) {
        awaitWorkAvailable()

        /* ... */
        parentFrameClock.withFrameNanos { frameTime -> /* ... */}
        discardUnusedMovableContentState()
        nextFrameEndCallbackQueue.markFrameComplete()
    }
}

The method calls recompositionRunner and passes it a lambda. The call to awaitWorkAvailable either enters or stores the continuation of the coroutine.

val unregisterApplyObserver =
    Snapshot.registerApplyObserver { changed, _ ->
        synchronized(stateLock) {
                if (_state.value >= State.Idle) {
                    /* ... */
                    deriveStateLocked()
                } else null
            }
            ?.resume(Unit)
    }

The function installs a global apply observer that resumes the stored continuation if eligible. So we have a coroutine that runs an infinite loop, which might hang at the start of each iteration. When the global snapshot has been advanced because an event listener for example has modified a state, the observer above takes the stored pending continuation and resumes it. So the division of labour between Composition, CompositionContext, and Composer must be:
• A CompositionContext tracks the validity of every registered Composition
• A Composition materialises a @Composable call graph, into which a Composer output parameter is injected
Is this fair, and what’s all the parenting business about?

[shikasd]

Check how SubcomposeLayout works, it should give you a better idea about parent-child relationship. The recomposer contains a flat list that just maintains a list of active compositions to what to recompose, that's why most of the methods go back to it.

[shikasd]

CompositionContext is an abstraction over recomposer + parent composition local context, to simplify this a bit

[min]

@shikasd I diligently study every major framework that my apps depend on so that I could handroll a basic (naive and inefficient) toy version of each. I know how my web framework works, how my database driver works, and so on. After months of digging, Compose remains a complete mystery because it has the most confusing organisation of things I’ve ever seen. I could not even begin to author my toy version of CompositionContext because I cannot wrap my head around how the work is divided between the types.

interface CompositionContext {
    // which decls here?
}
interface Composition {
    // which decls here?
}
interface Composer {
    // decls that support the use of the type as
    // an output parameter injected into `@Composable` functions
}

Also, the docs give the impression that there’s a composition per app or window, but the actual code reveals that there can be multiple objects of type Composition (why?) at a given point in time. They can be parents and children, or invalidation delegates (are they the same thing?), where the hierarchy is abstracted over by a CompositionContext (why and how?)

The recomposer contains a flat list

So it’s not a tree that can be arbitrarily deep, is it? It’s just a list?

active compositions to what to recompose

What do you mean? Recomposer.runRecomposeAndApplyChanges enters a concurrent loop that advances on global snapshot apply where it reports the updated state objects to the registered Composition elements.

knownCompositionsLocked().fastForEach { value ->
    if (
        value !in alreadyComposed &&
            value.observesAnyOf(modifiedValuesSet)
    ) {
        toRecompose += value
    }
}

Each composition knows which recompose scopes within it read which state objects. It passes the invalidations to the Recomposer so new output can be recorded by rerunning composable functions.

guardInvalidationsLocked { invalidations ->
    composer.recompose(invalidations, shouldPause).also { shouldDrain ->
        // Apply would normally do this for us; do it now if apply shouldn't happen.
        if (!shouldDrain) drainPendingModificationsLocked()
    }
}

Am I misunderstanding anything?
• A Composer is written to by @Composable functions, so that a Composition (or multiple? why? when?) can be materialised?
• A CompositionContext despatches updated state objects to every registered Composition on global snapshot apply, so that recompositions can be scheduled
• A Composition is an element in a CompositionContext list
◦ A child Composition of a Composition is stored in the CompositionContext associated with the parent Composition
Is this it?

[shikasd]

I recommend reading https://intelligiblebabble.com/compose-from-first-principles/ for a surface level understanding

For nested subcompositions, I recommend looking through SubcomposeLayout (which is also a bit of a mess admittedly). The idea is that some compositions might happen out of phase e.g. during measure and layout phases. The subcompositions keep composition local context of the parent group and some other things making it a tree

Recomposer does not care about compositions as a tree and only keeps them in a list right now. We want to make sure we recompose every active composition, it does not have to be in a tree structure for that.

[min]

@shikasd Thanks for the pointer. I saw the talk and read the entire article, but unfortunately it didn’t shed much light on what I didn’t already know. As for CompositionContext trees, I know that a CompositionImpl: Composition is constructed with a val parent: CompositionContext (typically a Recomposer: CompositionContext) which it uses to initialise its internal val composer: ComposerImpl.

internal val composer: ComposerImpl =
    ComposerImpl(
            applier = applier,
            parentContext = parent,
            slotTable = slotTable,
            abandonSet = abandonSet,
            changes = changes,
            lateChanges = lateChanges,
            composition = this,
            observerHolder = observerHolder,
        )
        .also { parent.registerComposer(it) }

The ComposerImpl class has an internal inner class CompositionContextImpl. Each instance is tied to the ComposerImpl through which it was constructed, and thus has access to its private val parentContext: CompositionContext. This is how CompositionContext implementors are organised into a tree: a CompositionImpl has a CompositionContext and a ComposerImpl, and the composer can construct children of the context.

override fun buildContext(): CompositionContext {
    startGroup(referenceKey, reference)
    if (inserting) writer.markGroup()

    var observerHolder = nextSlot() as? RememberObserverHolder
    if (observerHolder == null) {
        observerHolder =
            ReusableRememberObserverHolder(
                CompositionContextHolder(
                    CompositionContextImpl(
                        [e-mail],
                        forceRecomposeScopes,
                        sourceMarkersEnabled,
                        composition.observerHolder,
                    )
                ),
                afterGroupIndex = -1,
            )
        updateValue(observerHolder)
    }
    val holder = observerHolder.wrapped as CompositionContextHolder
    holder.ref.updateCompositionLocalScope(currentCompositionLocalScope())
    endGroup()

    return holder.ref
}

And the implementation by CompositionImpl of the buildContext method seems to be the only place CompositionContextImpl is instantiated. I don’t know what this is for, and how it achieves what this is for.

override fun composeInitial(
    composition: ControlledComposition,
    content: @Composable () -> Unit,
) {
    parentContext.composeInitial(composition, content)
}

Most of its methods just delegate to the parent. So I don’t understand the need for composition contexts to be a tree.
> A CompositionContext is a tree of runtime services active Composition objects are registered with and driven by
Would this be a fair summary?

[shikasd]

didn't shed much light on what I didn't already know

tbh it is enough to hand roll a small version of Compose imo :)

For other questions, see the comment above

For nested subcompositions, I recommend looking through SubcomposeLayout (which is also a bit of a mess admittedly). The idea is that some compositions might happen out of phase e.g. during measure and layout phases. The subcompositions keep composition local context of the parent group and some other things making it a tree

[min]

@shikasd Does every child composition of a composition have to be in either the same context or a descendent context? Or is a child composition allowed, with valid semantics, to belong to a context that’s neither the same nor a descendent context?