your state data would look like this (1st page list, idle status) (1st page list, started status) (2nd page list, started status) <--- this is a problem (2nd page list, success status)

I don't what are the status for? Why not remove them?

status is to show progressbar below the list

Does the processing for a list have a terminal state?

what do you mean by processing? runing the refresh or the queryOBservable itself?

Well, what I mean is, what does status of a list represent?

status of the pagination routine, ie. network call for new page + insert new data

Here's what I use for state:

sealed class State {
    object InFlight: State()
    object Complete: State()
    object Idle: State()
    object Gone: State()
}

InFlight means the process is running, Complete means it finished, Idle means no work is occuring, and Gone means the "observable" is gone. A state object is observed by the fragment which reacts to changes.

hmm not sure how that helps?

And how are you making calls to update an entity? And where are you observing?

not sure what you mean, its sql database thats being observed

Exactly. And how/when is it updated?

not sure what you are getting at but lets say like this

fun fetchNewPage()
   = api.newPage(page + 1)
       .flatMap { db.insert(it) }

Where is that being called?

in the domain layer? or data layer? does it matter?

Yes, from the ui layer, where does the process begin?

not sure how is it relevant, but lets say when end of the list is scrolled to, that generates some kind of EndOfListReached event, to which you react with the fetchNewPage()

No, in the UI (fragment, whatever), how is the process to fetch pages started?

recyclerview.scrollListener has onScrolled callback, in which I ask recyclerview for last visible item position, and if that is "close" to the end, I send that event

This is one reason I don't like observables in a database or api service. 1. A database stores, retrieves, and updates raw data. That is it's sole purpose. It does one thing and does it well. 2. An external API service, such as Retrofit, communicates with an external source. That is it's sole purpose. It does one thing and does it well. 3. A repository uses services such as a database or API to retrieve and/or store data. That is it's sole purpose. It does one thing and does it well. When we ask a database or a repository or an API to convert raw data to an observable we are making them responsible for data conversion. This seems to violate the single responsibility principle.

Why? How else do you keep your UI up-to-date with database state?

By returning a wrapper object from your repository

what wrapper?

Yes, but think about it. What makes the database change? A repository, right? And what calls the repository? Some other class, right? Like some call from a presenter or view model, or something.

How else would you do it? If you dont want push, then you can only have pull , i.e. re-running the query manually again

Something touches the database. It doesn't update itself.

thats what im saying, youre proposing this?

ViewModel {
   fun doSomething() {
      repository.updateSomething()
      val freshList = repository.querySomethings()  
      // somehow push the freshList to the ui
   }
}

My point is, a query, from a database or api, in my case returns exactly what is stored in the database or retrieved from the api.

sure, youre talking about Single

No, I'm not. I'm talking about raw data. Whether it's Rx, LiveData, whatever doesn't matter.

how, its just observer pattern?

No, it's adding responsibility beyond fetching data. It's also adapting the data to something else.

Cook has single responsibility - to cook food; its not violation of SRP if he cracks egss, puts them in a pan, then puts them on a plate

Cook is not in the data layer. It's in the adapter layer.

still dont see how its adapting the data, its just providing the listener that the data changed

I guess we don't agree on that. It's ok.

But I dont see your point, providing repository.databaseChangedObservable : Observable<Unit>

It's modifying the data it retrieves.

what?

That belongs in a different layer

repository proxying dbStore.someObservable is modifying data how?

Look, I get it. People do things different ways. I've seen Observables and LiveData in repositories. It's just my personal opinion that they don't belong there.

So youre against DataChangedListener provided by repository?

Yep.

Then you cannot have event buses, etc, and would have to keep track of everybody touching the database

What is it exactly in your app that listens for changes in the Db?

Chat UI, it displays messages

Well, one option is to wire in a LiveData.transformations.switchmap to change your State variable. Just a thought. You;'d have to somehow wire that into your observer.

it wont work, youll still have the glitch issue, unless you use delay

So delay.

I dont want to, I worked hard on performance, dont want to introduce arbitrary delays just for stupid paging; which will delay non-paging relayed refreshes too

I think I understand now and think I have a solution.

I dont follow -.-

It relates to changing your UI/Loading/Etc when the data changes.

I am. The issue is list would get updated and only then status would be updated, not together

Then call the listener when youre ready.

thats the issue that you cannot, since the room observable emits whenever the db changes automatically, internally. I dont invalidate it

So, room emits, your list updates, and when the list is finished doing it's thing, call that stateChanged listener for the UI.

not sure what you mean, this is how it would look

So really, you don't have a failure mechanism. You're hard coding that. If I understand correctly.

what failure?

Isn't this a failure?

2nd page; started <--- invalid

failure of what? the refresh routine? no

Then why do you have it marked invalid?

do you know how combineLatest works?

And that's the glitch?

What's the second observable?

what second observable? its this

Observable.combineLatest(
   queryObservable,
   paginationStateObservable
)

Where dos paginationStateObservable come from?

its the observable of State (started, success, error) of paginatioun routine we're talking about the whole time

Then if there's a glitch, you can't rely on it for the UI. You'll have to work around it. Use some other logic in your adapter that just fires the listener I suggested whenever it's appropriate.

It seems that you are putting UI logic in a lower layer. The

{ Started, Success }

states are actually UI logic for showing/hiding the loader (if I understand correctly) this logic should be in the UI layer. If it is not possible to filter the observable to only emit values to the UI when the list actually changes this logic should be in the UI. In the UI layer you already know what items you are displaying in therefore can compare them with the data in the observable. If the received data is different from the currently displayed data the Loader should be hidden.

That's sort of what I was trying to explain. But it wasn't clear what he meant by Status until he finally explained it.

@Eric Martori the layering is not the issue; maybe youre correct but it doesnt matter for now; the issue is exactly your assumption that the loader should be hidden when the list changes; because it can change because of other reasons (item deleted, item edited - while the "load next page" is going on

Then your next option would be to synchronize your observables. I don't know enough about them, but there probably is some operator that let's you do that or you can combine a couple of them to do it.

@Eric Martori as I said, its hard because the observable has no notion of the status, and its emit happens before, so delay is only option,other than abitrary block

Sounds to me like you need some kind of Reducer. It's job is to receive results from database queries, API calls, calculations, and things like that, then take the current ViewState of the feature and produce a new state for the UI to render.

@dewildte how would reducer help? You'd have subscription to queryObservable. map that to a reducer QueryChangedAction. Nothing to signal you should not emit it further

Maybe I've completely misunderstood the entire premise of the original question. But it seems like we're trying to update a status indicator whenever the system is updating - "youll have new data with progressbar showing still". And it seems like it's some sort of stream, or flow, or whatever you want to call it. And, honestly, I don't see the value of a status indicator for that type of operation. The only time I use a status indicator is for long running operations. For short frequent updates I think it's disruptive from a user perspective. Just an opinion.

No, progressbar is needed while pagination is running for sure, it can take indeterminate amount of time