Thread
#android-architecture
    u

    ursus

    3 years ago
    your state data would look like this (1st page list, idle status) (1st page list, started status) (2nd page list, started status) <--- this is a problem (2nd page list, success status)
    f

    florent

    3 years ago
    I don't what are the status for? Why not remove them?
    and be new data received in page two so the loading for page two can be hidden
    u

    ursus

    3 years ago
    status is to show progressbar below the list
    well you dont know that if you can hide it because your queryObservable might reemit for whatever reason, not just that new data inserted
    thats the exact problem
    a

    Al Warren

    3 years ago
    Does the processing for a list have a terminal state?
    u

    ursus

    3 years ago
    what do you mean by processing? runing the refresh or the queryOBservable itself?
    a

    Al Warren

    3 years ago
    Well, what I mean is, what does status of a list represent?
    u

    ursus

    3 years ago
    status of the pagination routine, ie. network call for new page + insert new data
    a

    Al Warren

    3 years ago
    Here's what I use for state:
    sealed class State {
        object InFlight: State()
        object Complete: State()
        object Idle: State()
        object Gone: State()
    }
    InFlight means the process is running, Complete means it finished, Idle means no work is occuring, and Gone means the "observable" is gone. A state object is observed by the fragment which reacts to changes.
    u

    ursus

    3 years ago
    hmm not sure how that helps?
    the problem is that queryObservable observes the database, it emits whenever database changes
    i.e. you cannot infer that the next emit after the inserting of data will be the status change
    imagine you deleting an item while pagination routine is running...
    @Al Warren tldr; queryObservable will emit after youdr data is inserted, no after the refresh routine returns, or emits that State; query will be refreshed before
    a

    Al Warren

    3 years ago
    And how are you making calls to update an entity? And where are you observing?
    u

    ursus

    3 years ago
    not sure what you mean, its sql database thats being observed
    a

    Al Warren

    3 years ago
    Exactly. And how/when is it updated?
    u

    ursus

    3 years ago
    not sure what you are getting at but lets say like this
    fun fetchNewPage()
       = api.newPage(page + 1)
           .flatMap { db.insert(it) }
    a

    Al Warren

    3 years ago
    Where is that being called?
    u

    ursus

    3 years ago
    in the domain layer? or data layer? does it matter?
    you mean when, like after end of list is reached in the uil?
    a

    Al Warren

    3 years ago
    Yes, from the ui layer, where does the process begin?
    u

    ursus

    3 years ago
    not sure how is it relevant, but lets say when end of the list is scrolled to, that generates some kind of EndOfListReached event, to which you react with the fetchNewPage()
    a

    Al Warren

    3 years ago
    No, in the UI (fragment, whatever), how is the process to fetch pages started?
    You're observing somewhere so somewehre youre getting that observable.
    How are you're doing that is what I'm asking.
    u

    ursus

    3 years ago
    recyclerview.scrollListener has onScrolled callback, in which I ask recyclerview for last visible item position, and if that is "close" to the end, I send that event
    is that what you mean?
    I think this is only solvable with Status being saved to the database; i.e. data inserted + status changed both in a sql transaction -- and then data querie also together items + status
    a

    Al Warren

    3 years ago
    This is one reason I don't like observables in a database or api service. 1. A database stores, retrieves, and updates raw data. That is it's sole purpose. It does one thing and does it well. 2. An external API service, such as Retrofit, communicates with an external source. That is it's sole purpose. It does one thing and does it well. 3. A repository uses services such as a database or API to retrieve and/or store data. That is it's sole purpose. It does one thing and does it well. When we ask a database or a repository or an API to convert raw data to an observable we are making them responsible for data conversion. This seems to violate the single responsibility principle.
    u

    ursus

    3 years ago
    Why? How else do you keep your UI up-to-date with database state?
    Observable emiting the current query values seems like a perfect fit
    a

    Al Warren

    3 years ago
    By returning a wrapper object from your repository
    u

    ursus

    3 years ago
    what wrapper?
    you still need to expose Observable of these wrapper objects, dont you?
    not sure how turning onDatabaseDataChanged callback into Observable is making responsible for data conversion, its means to keep the observers with fresh data
    a

    Al Warren

    3 years ago
    Yes, but think about it. What makes the database change? A repository, right? And what calls the repository? Some other class, right? Like some call from a presenter or view model, or something.
    I probably misspoke when I said use a wrapper. I use a wrapper to wrap a database or api result in a class that represents either failure or success (a monad). The data gets converted to an observable elsewhere.
    u

    ursus

    3 years ago
    How else would you do it? If you dont want push, then you can only have pull , i.e. re-running the query manually again
    are you talking about queries as Singles or Observables?
    a

    Al Warren

    3 years ago
    Something touches the database. It doesn't update itself.
    u

    ursus

    3 years ago
    thats what im saying, youre proposing this?
    ViewModel {
       fun doSomething() {
          repository.updateSomething()
          val freshList = repository.querySomethings()  
          // somehow push the freshList to the ui
       }
    }
    a

    Al Warren

    3 years ago
    My point is, a query, from a database or api, in my case returns exactly what is stored in the database or retrieved from the api.
    u

    ursus

    3 years ago
    sure, youre talking about Single
    im talking about the reactive model / Observable
    a

    Al Warren

    3 years ago
    No, I'm not. I'm talking about raw data. Whether it's Rx, LiveData, whatever doesn't matter.
    That's my point, adding reaction in the data layer violates SRP.
    u

    ursus

    3 years ago
    how, its just observer pattern?
    responsibility of data layer is to handle data
    a

    Al Warren

    3 years ago
    No, it's adding responsibility beyond fetching data. It's also adapting the data to something else.
    Yes, but not to adapt/change raw data. That happens in another layer. Or should.
    u

    ursus

    3 years ago
    Cook has single responsibility - to cook food; its not violation of SRP if he cracks egss, puts them in a pan, then puts them on a plate
    which are 3 disctinct actions
    a

    Al Warren

    3 years ago
    Cook is not in the data layer. It's in the adapter layer.
    Cook has to retrieve food.
    u

    ursus

    3 years ago
    still dont see how its adapting the data, its just providing the listener that the data changed
    a

    Al Warren

    3 years ago
    I guess we don't agree on that. It's ok.
    u

    ursus

    3 years ago
    But I dont see your point, providing repository.databaseChangedObservable : Observable<Unit>
    is not doiing anything
    a

    Al Warren

    3 years ago
    It's modifying the data it retrieves.
    u

    ursus

    3 years ago
    what?
    a

    Al Warren

    3 years ago
    That belongs in a different layer
    u

    ursus

    3 years ago
    repository proxying dbStore.someObservable is modifying data how?
    a

    Al Warren

    3 years ago
    Look, I get it. People do things different ways. I've seen Observables and LiveData in repositories. It's just my personal opinion that they don't belong there.
    u

    ursus

    3 years ago
    So youre against DataChangedListener provided by repository?
    a

    Al Warren

    3 years ago
    Yep.
    u

    ursus

    3 years ago
    Then you cannot have event buses, etc, and would have to keep track of everybody touching the database
    so you can pull new data from it
    multiple places can be touching the repository, you should not be aware of that at all, just consume the current state
    a

    Al Warren

    3 years ago
    What is it exactly in your app that listens for changes in the Db?
    u

    ursus

    3 years ago
    Chat UI, it displays messages
    and message can be added but you (via viewmodel), but also via websocket (other user, via totaly different place, nothing to do with ui)
    a

    Al Warren

    3 years ago
    Well, one option is to wire in a LiveData.transformations.switchmap to change your State variable. Just a thought. You;'d have to somehow wire that into your observer.
    u

    ursus

    3 years ago
    it wont work, youll still have the glitch issue, unless you use delay
    a

    Al Warren

    3 years ago
    So delay.
    u

    ursus

    3 years ago
    I dont want to, I worked hard on performance, dont want to introduce arbitrary delays just for stupid paging; which will delay non-paging relayed refreshes too
    a

    Al Warren

    3 years ago
    I think I understand now and think I have a solution.
    It's similar to how I handle onClick in a list adapter. And that's "if" you're observing in your adapter.
    In your adapter:
    var stateChangeListener: (T) -> Unit = { }
    
    // when data or list changes
    // set a state then
    onSomeOtherListener { stateChangeListener(state) }
    In your UI/Fragment/Whatever
    yourAdapter.stateChangeListener = ::stateChanged
    
    private fun stateChanged(state: State) {
        // do something with state
    }
    Just a thought.
    u

    ursus

    3 years ago
    I dont follow -.-
    how does it relate @Al Warren?
    a

    Al Warren

    3 years ago
    It relates to changing your UI/Loading/Etc when the data changes.
    I thought that's what you were trying to do.
    You want to do something in the UI when the data changes, right?
    u

    ursus

    3 years ago
    I am. The issue is list would get updated and only then status would be updated, not together
    its rx combineLatest glitch issue
    if there is reactive query observable, it will emit after data is written, which is before state.success is emitted
    thats the whole issue
    a

    Al Warren

    3 years ago
    Then call the listener when youre ready.
    u

    ursus

    3 years ago
    thats the issue that you cannot, since the room observable emits whenever the db changes automatically, internally. I dont invalidate it
    a

    Al Warren

    3 years ago
    So, room emits, your list updates, and when the list is finished doing it's thing, call that stateChanged listener for the UI.
    Can't you scroll the list and monitor list position?
    I would take State out of the Rx stuff. Only use it when you need it. If that makes sense.
    I typically use two "state" types - one for UI state and one for success/failure (I use an Either monad for that one).
    u

    ursus

    3 years ago
    not sure what you mean, this is how it would look
    fun refresh() {
        statusRelay.accept(Status.STARTED)
        call the api
        insert new data   <------------ db query emits automatically here !
        statusRelay.accept(Status.SUCCESS)
    }
    and we want it to emit only after
    statusRelay.accept(Status.SUCCESS)
    so if you use
    Observable.combineLatest(queryObservable, stateObservable)
    then youd need some sort of
    Observable.combineLatest(queryObservable.filter { !queryRefreshBlocked }, stateObservable)˛
    without it would get 1st page; started 2nd page; started <--- invalid 2nd page; success
    a

    Al Warren

    3 years ago
    So really, you don't have a failure mechanism. You're hard coding that. If I understand correctly.
    u

    ursus

    3 years ago
    what failure?
    a

    Al Warren

    3 years ago
    Isn't this a failure?
    2nd page; started <--- invalid
    u

    ursus

    3 years ago
    failure of what? the refresh routine? no
    its invalid state that should never happen
    a

    Al Warren

    3 years ago
    Then why do you have it marked invalid?
    u

    ursus

    3 years ago
    do you know how combineLatest works?
    its emits a tuple of the currently changed value + previous values of rest of the streams
    if you change both, and its not atomic, youd get 2 emits
    a

    Al Warren

    3 years ago
    And that's the glitch?
    u

    ursus

    3 years ago
    t0:            1st page; started
    t1.000000      2nd page; started <--- invalid
    t1,000001      2nd page; success
    t represents time, 00001 means its very close but same time
    a

    Al Warren

    3 years ago
    What's the second observable?
    u

    ursus

    3 years ago
    what second observable? its this
    Observable.combineLatest(
       queryObservable,
       paginationStateObservable
    )
    a

    Al Warren

    3 years ago
    Where dos paginationStateObservable come from?
    u

    ursus

    3 years ago
    its the observable of State (started, success, error) of paginatioun routine we're talking about the whole time
    a

    Al Warren

    3 years ago
    Then if there's a glitch, you can't rely on it for the UI. You'll have to work around it. Use some other logic in your adapter that just fires the listener I suggested whenever it's appropriate.
    And use a different State type for the UI.
    Btw, somewhere I did a codelab on messaging but it didn't use a db. With that, it's about all I can add. Good luck.
    e

    Eric Martori

    3 years ago
    It seems that you are putting UI logic in a lower layer. The
    { Started, Success }
    states are actually UI logic for showing/hiding the loader (if I understand correctly) this logic should be in the UI layer. If it is not possible to filter the observable to only emit values to the UI when the list actually changes this logic should be in the UI. In the UI layer you already know what items you are displaying in therefore can compare them with the data in the observable. If the received data is different from the currently displayed data the Loader should be hidden.
    you then can encapsulate this logic in a pure function or some other collaborator that can be unit tested and inject it in all the views that have pagination
    a

    Al Warren

    3 years ago
    That's sort of what I was trying to explain. But it wasn't clear what he meant by Status until he finally explained it.
    u

    ursus

    3 years ago
    @Eric Martori the layering is not the issue; maybe youre correct but it doesnt matter for now; the issue is exactly your assumption that the loader should be hidden when the list changes; because it can change because of other reasons (item deleted, item edited - while the "load next page" is going on
    so you cannot infer that "the next observable emit means pagination ended"
    or any kind of heuristic "items added" or whatever
    e

    Eric Martori

    3 years ago
    Then your next option would be to synchronize your observables. I don't know enough about them, but there probably is some operator that let's you do that or you can combine a couple of them to do it.
    u

    ursus

    3 years ago
    @Eric Martori as I said, its hard because the observable has no notion of the status, and its emit happens before, so delay is only option,other than abitrary block
    d

    dewildte

    3 years ago
    Sounds to me like you need some kind of Reducer. It's job is to receive results from database queries, API calls, calculations, and things like that, then take the current ViewState of the feature and produce a new state for the UI to render.
    Or you might find this post by Google to help with your situation:https://developer.android.com/jetpack/docs/guide#show-in-progress-operations
    I personally prefer the Reducer
    u

    ursus

    3 years ago
    @dewildte how would reducer help? You'd have subscription to queryObservable. map that to a reducer QueryChangedAction. Nothing to signal you should not emit it further
    a

    Al Warren

    3 years ago
    Maybe I've completely misunderstood the entire premise of the original question. But it seems like we're trying to update a status indicator whenever the system is updating - "youll have new data with progressbar showing still". And it seems like it's some sort of stream, or flow, or whatever you want to call it. And, honestly, I don't see the value of a status indicator for that type of operation. The only time I use a status indicator is for long running operations. For short frequent updates I think it's disruptive from a user perspective. Just an opinion.
    u

    ursus

    3 years ago
    No, progressbar is needed while pagination is running for sure, it can take indeterminate amount of time