I have a race condition between an `Initialize` in...

# ballast

[rocketraman]

I have a race condition between an Initialize input in a repository and a subsequent input. The Initialize input handler does observeFlows but the next input executes before the observeFlows side job starts. Doing the Initialize earlier in an init block or via an interceptor works, but feels a bit messy. Is there an idiomatic way to handle this?

[Casey Brooks]

My first thought would be to explicitly define the ordering you need. For example, sending a single Initialize Input which starts a sideJob, and in taht sideJob send that next Input before starting to observe the flow. observeFlows is just a basic wrapper around a sideJob

[rocketraman]

I don't want to couple these Inputs together in that way. I just want to make sure observe starts before anything else happens. For example, if we need to observe auth state from another repository, there should be a way to guarantee that we are seeing the latest auth state.

[Casey Brooks]

In this case, I’d say that you should try to structure your state such that the ordering on each Input does not matter. For example, if you need the result of both the Flow and the initial Input to continue, set a property in your State class’s body that combines the two. Until both values are supplied, that 3rd value is not valid, so you end up just looking at that 3rd valid to know when to continue.

For example, something like this:

object MyRepository {
    data class State(
        val repositoryValue: String? = null,
        val initialValueFromUi: String? = null,
    ) {
        val derivedValue: String = if(repositoryValue != null && initialValueFromUi != null) {
            computeDerivedValue(repositoryValue, initialValueFromUi)
        } else {
            null
        }
    }

    sealed class Inputs {
        object InitializeFlow : Inputs()
        data class RepositoryValueUpdated(val repositoryValue: String) : Inputs()
        data class InitializeFromUi(val initialValueFromUi: String) : Inputs()
    }
}

[rocketraman]

Hmm... one of the state values comes from a repository (higher up in the tree) that the model observes e.g. user auth state, and the input comes from lower in the tree e.g. a request to display something.

I think this approach could work but requires introducing complexity / boiler-plate without any corresponding business value.

I'd love to see a built-in way for models lower in a hierarchy to see a consistent view of state/models above, without workarounds like this.

[Casey Brooks]

For one, I don’t quite know how I would ensure that kind of behavior. But also, thinking of a pure MVI model, if you need certain Inputs to be processed in a specific order, you’re introducing an extra implicit form of state: time.

I’d expect that if something requires 2 inputs sent from independent sources, it wouldn’t matter what order they are sent, because they both ultimately result in some values being set in the state. But if they do depend on each other, then that should also be modeled in the contract, so that you process them in such a way that the one value is processed as a result of the other. Having 2 independent values need to be processed in a specific order is the kind of problem that MVI attempts to avoid, because your app is primarily concerned about the State itself rather than the Inputs.

IMO, placing the derived property in the State class body isn’t much extra work at all, and I do this pattern all the time. But I may just be mis-understanding your intended use-case, could you share an example of what you’re trying to do, and show why the above doesn’t work in your case?

[rocketraman]

Here is a simple example, among others. I have a repository which deals with users (my app has both anonymous, or session-based users, and authenticated users), and another repository which deals with some entities related to both types of users.

Conceptually this is what happens when a page of my app loads (its a compose-html app):

1) Ballast Repositories are initialized via DI
2) An input is triggered on the page View Model to fetch entities for display on the current page
3) That triggers an input on the EntityRepository to
a) trigger an Initialize input on UserRepository to start observing a flow of user states from the auth system
b) trigger a fetch input to fetch entities

Now the problem I have is that the (3b) fetch input started before the observeFlows side job that was kicked off by (3a), which meant no entities were retrieved.

Yes, my app only cares about the state of the user and the entities. But the ordering requirement here is simply that the fetch of the entities must occur after a user is available, so there is an ordering requirement between input (3a) and (3b). I can't have (3a) start (3b) as you initially suggested because that violates my UDF requirement (user repo doesn't know anything about entity repo, it sits above it).

I think one option that leverages state would be: (3a) checks the user state, and if no user is available, it sets some state like entityFetchRequested=true and then triggers the user initialize. If a user is available in the state, it triggers the fetch directly. Once a user becomes available, we check if entityFetchRequested=true, and if it is, trigger the fetch entities. Is that right? It feels messy to me to have some state that is only useful for this temporary ordering purpose, and of course in reality there are many of these entities so we would need 10s or 100s of these temporary state variables.

How would you do this?

[rocketraman]

@Casey Brooks I'd still be interested in your thoughts about my previous message, if you have some time to reply.

[Casey Brooks]

Yes, I am sorry about that. I’ve been meaning to get back here, but kept getting pulled into other things. I’ll try to share my thoughts in another day or two

[rocketraman]

No rush, thank you!

[Casey Brooks]

Ok, so after coming back to this scenario and thinking about this for a while, I think the solution may be as simple as not having the screen directly fetch the entities, but rather let the AuthState flow trigger the fetch once it’s ready. The UI ViewModel, then, would either observe the combined Flow of AuthState and Entities, or else suspend until the first valid value is ready (for a once-shot request).

So modeling it as a contract, we’d get something like this:

kotlin
object EntityRepositoryContract {
    data class State(
        val authState: AuthState = AuthState(false, null),
        val entities: List<Entity>? = null, // null when not fetched yet
    )

    sealed class Inputs {
        object Initialize : Inputs()
        data class AuthStateLoaded(val authState: AuthState) : Inputs()
        data class EntitiesLoaded(val entities: List<Entity>) : Inputs()
    }
    sealed class Events
}

which is pretty basic, and doesn’t really give much info about the required loading order between the AuthState and the Entities. And that’s actually a good thing: the UI really shouldn’t know much about how these values are fetches, just that they exist as a form of state. So the UI then just observes the State, or uses Flow operators to select the first valid combination of state values, and it needs to make sure that the data-loading gets kicked off at the right time.

kotlin
class EntityRepository : BallastViewModel<EntityRepositoryContract.Inputs, EntityRepositoryContract.Events, EntityRepositoryContract.State> {
    // use this to observe all changes in the UI
    fun observeEntities(): Flow<List<Entity>> = observeState()
        // when the UI starts requesting this data, make sure we try to fetch it again
        .onStart { trySend(EntityRepositoryContract.Initialize) }
        // wait for the repository state to stabilize 
        .filter { it.authState.loggedIn && it.entities != null }
        // select the specific value you need from the state
        .mapNotNull { it.entities }
    
    // or use this for a single-shot value
    suspend fun observeEntities(): List<Entity> { 
        return observeEntities().first()
    }
}

The complexity around the caching of the authState and entities should be handled entirely within the implementation (the InputHandler), so that the UI doesn’t have to worry about it. And internally, we’ll set up the fetches such that anytime the AuthState changes, we’ll re-fetch the entities.

kotlin
class EntityRepositoryInputHandler(
    private val userRepository: UserRepository, 
    private val entityDatabase: EntityDatabase,
) : InputHandler<
        EntityRepositoryContract.Inputs,
        EntityRepositoryContract.Events,
        EntityRepositoryContract.State> {
    override suspend fun InputHandlerScope<
            EntityRepositoryContract.Inputs,
            EntityRepositoryContract.Events,
            EntityRepositoryContract.State>.handleInput(
        input: EntityRepositoryContract.Inputs
    ) = when (input) {
        is EntityRepositoryContract.Inputs.Initialize -> {
            observeFlows(
                "authState",
                userRepository
                    .getAuthState()
                    .map { EntityRepositoryContract.Inputs.AuthStateLoaded(it) }
            )
        }

        is EntityRepositoryContract.Inputs.AuthStateLoaded -> {
            val updatedState = updateStateAndGet { it.copy(authState = input.authState) }
            if(updatedState.authState.loggedIn) {
                observeFlows(
                    "entities",
                    entityDatabase
                        .getEntitiesForUser(username = updatedState.authState.username)
                        .map { EntityRepositoryContract.Inputs.EntitiesLoaded(it) }
                )
            } else {
                noOp()
            }
        }
        is EntityRepositoryContract.Inputs.EntitiesLoaded -> {
            updateState { it.copy(entities = input.entities) }
        }
    }
}

Furthermore, with this setup, we can assume a similar situation is happening with the UserRepository. When the EntityRepository subscribes to the authState, it will internally fetch the authState if needed, or cache the value for faster access from the downstream repos. The EntityRepository really only cares about the current state of the user, and doesn’t really care about how the value is actually fetched or managed.

In this way, we are able to structure the repositories as separate things, where each “layer” is able to cache its own data and fetch it in the order it needs, without caring how its parent layers cache their data, and without leaking implementation details like input ordering to the parent/child layers. You can think of each VM as a “checkpoint”, where there is a lot of complexity, but ultimately it will eventually stabilize around some “good” value, and that “good” value is the only thing that the downstream layers really need to care about.

[rocketraman]

Thanks so much for the comprehensive reply. I may need to internalize this, but one problem initially comes to mind with this approach, which is that the repositories are loading everything from the server rather than waiting for the UI to request a relevant subset of the data. This may very well be infeasible given the number and size of the entities being fetched.

Maybe with the routing module (which I am not currently using), the routing information can be combined with the auth information to determine which entities need to be fetched.

[Casey Brooks]

Yeah, having many different values fetched and cached in the Repository ViewModels is a problem that I don’t have a great solution for yet, and can definitely lead to a lot of boilerplate. That’s the main thing that needs to be fixed with the ballast-repository module, introducing some kind of API to make these fetch/cache operations simpler and more feasible to do at scale, using this same basic idea