https://kotlinlang.org logo
Join Slack
Powered by
Paging: let’s say I have a RemoteMediator backed b...
# android
s
Paging: let’s say I have a RemoteMediator backed by room dao. If I make a manual change to a single item in database, will updated pagingData be emitted? e.g. marking item as favourite
😶 2
d
remotemediator is just a callback that triggers when pagingsource runs out of data
invalidation is what causes paging to reload and pick up new changes
room automatically triggers invalidation whenever you write to the table
and on invalidation, a new pagingdata is emitted
s
@Dustin Lam I’m super desperate - I had a custom PagingSource defined, paging was fine. But I could not modify existing items in any way, so I implemented remote mediator backed by dao. Now I get initial refresh+append, but after this nothing happens after I scroll. Are there any common reasons for this? If not, would you mind looking at my code?
Also deleting all items from DB doesn’t do anything, flow does not emit and shows all data as they were. Swipe to refresh also doesn’t do anything, just keeps spinning. With PagingSource implemented everything was fine
Took me 3 days - I observed paging flow with 
flowWithLifecycle
from lifecycle 2.4.0-alpha01. This does not work at all 🙂
d
Happy to look if you want to share your project with me 🙂
Changes to backing dataset should be propagated by invalidate + new PagingData emission from the Flow though
Make sure to use collectLatest as well so you can cancel the previous generation as fast as possible once you get a new PagingData
s
I use this to manage all my flow subscriptions, I have quite large app so I’m kind of confident in this:
Copy code
/**
 * Do NOT use with paging - creates a bug
 * */
fun <T> Flow<T>.observeIn(fragment: Fragment): Job {
    return observeIn(fragment.viewLifecycleOwner)
}

fun <T> Flow<T>.observeIn(lifecycleOwner: LifecycleOwner): Job {
    return flowWithLifecycle(lifecycleOwner.lifecycle)
        .launchIn(lifecycleOwner.lifecycle.coroutineScope)
}
It seems to collect only once from Flow≤PagingData<T>> though, all other cases are fine
d
Ah I think launchIn uses collect and not collectLatest under the hood
You also need to actually use the emitted value so maybe I'm missing something?
Where do you actually call submitData?
s
I switched to lifecycleScope and it’s working fine
I just got to new problems 🙂
I’m changing a few items with dao update and my list jumps to top also I’m not sure how to call invalidate when my pagingsource is from non-reusable lambda
d
you have to keep track of your emitted PagingSources and hook up invalidation somewhere in your app
there is a little utility wrapper
InvalidatingPagingSourceFactory which does the tracking for you
but you still need to actually set a listener for invalidation or just pass the factory directly to remotemediator depending on how you want to set it up
as for jumping to top
invalidation calls PagingSource.getRefreshKey() to figure out how to resume loading given previous PagingState
so make sure to implement that
s
I don’t think I can implement this
as PagingSource is created by room
d
ah
Room should automatically invalidate for you
whenever you write to the associated table
so you dont need to do that manually
although getRefreshKey is also implemented by Room
s
I need to invalidate second table from screen involving first table
d
any chance you are using compose?
er
s
nope, I’m using classic UI
d
so you have Paging setup for both tables, but in two separate screens?
s
I guess I could remove all from second table and it will recreate
yes
I have paging of random user profiles on first screen and I can “like” them, on second screen I have just liked profiles
so I need to invalidate second screen’s content
d
I think you only need one table for this
s
and this is not a single table, I have 2 API models for this
😄
I’m basing on backend, local db is only for manipulating items tbh
which is kinda hard to do without room
d
ah.. okay, so PagingSource you get from Room is based on a Query that has no clue about like status
is there any reason you dont just join those two tables?
s
I fetch like status from backend for every profile I see
d
er either way, you can manually invalidate regardless if it's from Room or not
s
I have a separate model from backend for liked profiles, but I think I could merge this, I didn’t think of it
I’m not sure why my list jumps to top though
maybe because of my entity?
Copy code
@Entity(tableName = "profiles")
data class ProfileEntity(
    @PrimaryKey(autoGenerate = true) @ColumnInfo(name = "auto_generated_id") val autoGeneratedId: Long,
    @ColumnInfo(name = "bio") val bio: String,
    @ColumnInfo(name = "birth_date") val birthDate: LocalDate?,
    @ColumnInfo(name = "has_children") val hasChildren: Boolean?,
    @ColumnInfo(name = "describe_tags") val describeTags: List<String>,
    @ColumnInfo(name = "education_level") val educationLevel: EducationLevel,
    @ColumnInfo(name = "id") val id: Long,
    @ColumnInfo(name = "is_liked") val isLiked: Boolean,
    @ColumnInfo(name = "like_tags") val likeTags: List<String>,
    @ColumnInfo(name = "likes") val likes: Long,
    @ColumnInfo(name = "marital_status") val maritalStatus: MaritalStatus,
    @ColumnInfo(name = "photo_id") val photoId: Long?,
    @ColumnInfo(name = "religiosity_level") val religiosityLevel: ReligiosityLevel,
    @ColumnInfo(name = "gender") val gender: Gender,
    @ColumnInfo(name = "username") val username: String,
    @ColumnInfo(name = "api_page_index") val apiPageIndex: Int
)
Copy code
@Query("SELECT * FROM profiles ORDER BY api_page_index")
fun getRandomProfilesPagingSource(): PagingSource<Int, ProfileEntity>
d
does your differ return the right result for two rows to be equal even if the is_liked changes?
and stepping back to your other question about manual invalidation
you can probably do something like
s
to be honest I’m not sure how to debug the differ
or you mean diffutil callback?
d
Copy code
val myPagingSourceFactory = InvalidatingPagingSourceFactory {
  myDao.pagingSource()
}

val pager = Pager(pagingSourceFactory = myPagingSourceFactory)

...

// Whenever you need to:
myPagingSourceFactory.invalidate()
❤️ 1
s
I’m comparing sealed class -> data classes, so I’m sure diffing should be fine
d
yes diffutil callback
are you able to share that?
s
Copy code
sealed class ProfilesAdapterItem : AdapterItem {
    data class ProfileItem(val profile: Profile) : ProfilesAdapterItem()
    object ReadyToStartMatchingItem : ProfilesAdapterItem()
}
private suspend fun handlePagingData(pagingData: PagingData<Profile>) {
    val mappedUiModels: PagingData<ProfilesAdapterItem> =
        pagingData.map { ProfilesAdapterItem.ProfileItem(it) }
    profilesPagingAdapter.submitData(mappedUiModels)
}
class ProfilesAdapterItemDiffCallback : DiffUtil.ItemCallback<ProfilesAdapterItem>() {

    override fun areItemsTheSame(oldItem: ProfilesAdapterItem, newItem: ProfilesAdapterItem): Boolean {
        return when {
            oldItem is ProfileItem && newItem is ProfileItem -> oldItem.profile.id == newItem.profile.id
            oldItem is ReadyToStartMatchingItem && newItem is ReadyToStartMatchingItem -> true
            else -> false
        }
    }

    override fun areContentsTheSame(oldItem: ProfilesAdapterItem, newItem: ProfilesAdapterItem): Boolean {
        return oldItem == newItem
    }
}
Copy code
data class Profile(
    val bio: String,
    val birthDate: LocalDate?,
    val hasChildren: Boolean?,
    val describeTags: List<Tag>,
    val educationLevel: EducationLevel,
    val id: Long,
    val isLiked: Boolean,
    val likeTags: List<Tag>,
    val likes: Long,
    val maritalStatus: MaritalStatus,
    val photoId: Long?,
    val photoUrl: String?,
    val religiosityLevel: ReligiosityLevel,
    val gender: Gender,
    val username: String,
) {
    val age: Int?
        get() = birthDate?.let { Period.between(birthDate, LocalDate.now()) }?.years
}
d
Hmm seems fine.. can you also post the result of 
adapter.snapshot()
before invalidate and after when it jumps?
just do something like
Copy code
adapter.loadStateFlow.collect {
  if (it.source.refresh is NotLoading) {
    println(adapter.snapshot())
  }
}
it will probably be a bit noisy if you have some appends happen
perhaps you could log when invalidate happens too
s
sec
d
Copy code
adapter.loadStateFlow.collect {
  if (it.source.refresh is Loading) {
    println("refresh loading")
  }
  if (it.source.refresh is NotLoading) {
    println(adapter.snapshot())
  }
}
something like that
also typing on phone so excuse me if any errors
s
by the way thanks a lot for helping me
d
no problem, happy to help
s
I really appreciate it
D: DBG: refresh loading D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: refresh loading D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: refresh loading D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio= 😧 DBG: + [ProfileItem(profile=Profile(bio= D: DBG: + [ProfileItem(profile=Profile(bio=
logcat breaks
it doesn’t output full content for some reason
d
blah okay, maybe just override toString in your ProfileItem to just return "ProfileItem(id=$id)"
maybe it will fit then :D
s
D: DBG: [ProfileItem(profile=Profile(id: 31)), ProfileItem(profile=Profile(id: 638)), ProfileItem(profile=Profile(id: 15)), ProfileItem(profile=Profile(id: 17)), ProfileItem(profile=Profile(id: 25)), ProfileItem(profile=Profile(id: 9)), ProfileItem(profile=Profile(id: 3)), ProfileItem(profile=Profile(id: 1)), ProfileItem(profile=Profile(id: 37)), ProfileItem(profile=Profile(id: 13)), ProfileItem(profile=Profile(id: 23)), ProfileItem(profile=Profile(id: 629)), ProfileItem(profile=Profile(id: 19)), ProfileItem(profile=Profile(id: 29)), ProfileItem(profile=Profile(id: 21)), ProfileItem(profile=Profile(id: 5)), ProfileItem(profile=Profile(id: 33)), ProfileItem(profile=Profile(id: 7)), ProfileItem(profile=Profile(id: 636)), ProfileItem(profile=Profile(id: 633))] D: DBG: [ProfileItem(profile=Profile(id: 31)), ProfileItem(profile=Profile(id: 638)), ProfileItem(profile=Profile(id: 15)), ProfileItem(profile=Profile(id: 17)), ProfileItem(profile=Profile(id: 25)), ProfileItem(profile=Profile(id: 9)), ProfileItem(profile=Profile(id: 3)), ProfileItem(profile=Profile(id: 1)), ProfileItem(profile=Profile(id: 37)), ProfileItem(profile=Profile(id: 13)), ProfileItem(profile=Profile(id: 23)), ProfileItem(profile=Profile(id: 629)), ProfileItem(profile=Profile(id: 19)), ProfileItem(profile=Profile(id: 29)), ProfileItem(profile=Profile(id: 21)), ProfileItem(profile=Profile(id: 5)), ProfileItem(profile=Profile(id: 33)), ProfileItem(profile=Profile(id: 7)), ProfileItem(profile=Profile(id: 636)), ProfileItem(profile=Profile(id: 633))] D: DBG: [ProfileItem(profile=Profile(id: 31)), ProfileItem(profile=Profile(id: 638)), ProfileItem(profile=Profile(id: 15)), ProfileItem(profile=Profile(id: 17)), ProfileItem(profile=Profile(id: 25)), ProfileItem(profile=Profile(id: 9)), ProfileItem(profile=Profile(id: 3)), ProfileItem(profile=Profile(id: 1)), ProfileItem(profile=Profile(id: 37)), ProfileItem(profile=Profile(id: 13)), ProfileItem(profile=Profile(id: 23)), ProfileItem(profile=Profile(id: 629)), ProfileItem(profile=Profile(id: 19)), ProfileItem(profile=Profile(id: 29)), ProfileItem(profile=Profile(id: 21)), ProfileItem(profile=Profile(id: 5)), ProfileItem(profile=Profile(id: 33)), ProfileItem(profile=Profile(id: 7)), ProfileItem(profile=Profile(id: 636)), ProfileItem(profile=Profile(id: 633)), ProfileItem(profile=Profile(id: 31)), ProfileItem(profile=Profile(id: 638)), ProfileItem(profile=Profile(id: 15)), ProfileItem(profile=Profile(id: 17)), ProfileItem(profile=Profile(id: 25)), ProfileItem(profile=Profile(id: 9)), ProfileItem(profile=Profile(id: 3)), ProfileItem(profile=Profile(id: 1)), ProfileItem(profile=Profile(id: 37)), ProfileItem(profile=Profile(id: 13)), ProfileItem(profile=Profile(id: 23)), ProfileItem(profile=Profile(id: 629)), ProfileItem(profile=Profile(id: 19)), ProfileItem(profile=Profile(id: 29)), ProfileItem(profile=Profile(id: 21)), ProfileItem(profile=Profile(id: 5)), ProfileItem(profile=Profile(id: 33)), ProfileItem(profile=Profile(id: 7)), ProfileItem(profile=Profile(id: 636)), ProfileItem(profile=Profile(id: 633))] D: DBG: refresh loading D: DBG: [ProfileItem(profile=Profile(id: 31)), ProfileItem(profile=Profile(id: 638)), ProfileItem(profile=Profile(id: 15)), ProfileItem(profile=Profile(id: 17)), ProfileItem(profile=Profile(id: 25)), ProfileItem(profile=Profile(id: 9)), ProfileItem(profile=Profile(id: 3)), ProfileItem(profile=Profile(id: 1)), ProfileItem(profile=Profile(id: 37)), ProfileItem(profile=Profile(id: 13)), ProfileItem(profile=Profile(id: 23)), ProfileItem(profile=Profile(id: 629)), ProfileItem(profile=Profile(id: 19)), ProfileItem(profile=Profile(id: 29)), ProfileItem(profile=Profile(id: 21)), ProfileItem(profile=Profile(id: 5)), ProfileItem(profile=Profile(id: 33)), ProfileItem(profile=Profile(id: 7)), ProfileItem(profile=Profile(id: 636)), ProfileItem(profile=Profile(id: 633))] D: DBG: [ProfileItem(profile=Profile(id: 31)), ProfileItem(profile=Profile(id: 638)), ProfileItem(profile=Profile(id: 15)), ProfileItem(profile=Profile(id: 17)), ProfileItem(profile=Profile(id: 25)), ProfileItem(profile=Profile(id: 9)), ProfileItem(profile=Profile(id: 3)), ProfileItem(profile=Profile(id: 1)), ProfileItem(profile=Profile(id: 37)), ProfileItem(profile=Profile(id: 13)), ProfileItem(profile=Profile(id: 23)), ProfileItem(profile=Profile(id: 629)), ProfileItem(profile=Profile(id: 19)), ProfileItem(profile=Profile(id: 29)), ProfileItem(profile=Profile(id: 21)), ProfileItem(profile=Profile(id: 5)), ProfileItem(profile=Profile(id: 33)), ProfileItem(profile=Profile(id: 7)), ProfileItem(profile=Profile(id: 636)), ProfileItem(profile=Profile(id: 633))] 😧 DBG: [ProfileItem(profile=Profile(id: 31)), ProfileItem(profile=Profile(id: 638)), ProfileItem(profile=Profile(id: 15)), ProfileItem(profile=Profile(id: 17)), ProfileItem(profile=Profile(id: 25)), ProfileItem(profile=Profile(id: 9)), ProfileItem(profile=Profile(id: 3)), ProfileItem(profile=Profile(id: 1)), ProfileItem(profile=Profile(id: 37)), ProfileItem(profile=Profile(id: 13)), ProfileItem(profile=Profile(id: 23)), ProfileItem(profile=Profile(id: 629)), ProfileItem(profile=Profile(id: 19)), ProfileItem(profile=Profile(id: 29)), ProfileItem(profile=Profile(id: 21)), ProfileItem(profile=Profile(id: 5)), ProfileItem(profile=Profile(id: 33)), ProfileItem(profile=Profile(id: 7)), ProfileItem(profile=Profile(id: 636)), ProfileItem(profile=Profile(id: 633)), ProfileItem(profile=Profile(id: 31)), ProfileItem(profile=Profile(id: 638)), ProfileItem(profile=Profile(id: 15)), ProfileItem(profile=Profile(id: 17)), ProfileItem(profile=Profile(id: 25)), ProfileItem(profile=Profile(id: 9)), ProfileItem(profile=Profile(id: 3)), ProfileItem(profile=Profile(id: 1)), ProfileItem(profile=Profile(id: 37)), ProfileItem(profile=Profile(id: 13)), ProfileItem(profile=Profile(id: 23)), ProfileItem(profile=Profile(id: 629)), ProfileItem(profile=Profile(id: 19)), ProfileItem(profile=Profile(id: 29)), ProfileItem(profile=Profile(id: 21)), ProfileItem(profile=Profile(id: 5)), ProfileItem(profile=Profile(id: 33)), ProfileItem(profile=Profile(id: 7)), ProfileItem(profile=Profile(id: 636)), ProfileItem(profile=Profile(id: 633))]
I put those into text-compare and ids are identical
d
so interestingly the same id appears multiple times in the list
s
this is true
I don’t have id as primary key
as I get random profiles from BE each time and they will eventually duplicate
d
but your diffutil only checks id to determine if the items are the same
s
so I rely on autogenerated id for dao order
oh
this is right
I’m lucky to speak to you
❤️
d
Glad it wasn't a bug in paging itself at least 🙂
s
thanks a lot
you are awesome
d
Thanks! Glad to help
s
I got it to work. Item edited by dao update flickers/blinks, usually this would be resolved by setting stable ids in adapter, but this is unsupported by paging. How do I get rid of this in pagingdataadapter?
d
actually setStableIds might actually work in your case
i think paging has that restriction mostly for drops and prepends where the id is based on position
but you might be able to adapter.peek(index) in getItemId and it may "just work'
assuming you don't just return position directly
s
actually I can’t do this, getItemId is final and hasStableIds throws UnsupportedOperationEx
d
Hmm
let me file a bug to look into this and i'll bring it up internally
I actually forgot why we did this and we never got any complaints about it until now
Ah I found the original bug https://issuetracker.google.com/137454910
Looks like we disabled it due to placeholders
You might be able to turn off / remove the flicker animation - I'd need to try in a sample later..
s
thanks a lot
sorry for bumping old discussion - can you guide me in the right direction? I enter the fragment with recycler, after first fetch recycler immediately scrolls from top to the loading placeholder of the second page do you know any common reasons for this?
@Dustin Lam (mentioning for visibility)
d
i believe there is a related bug im looking into for this
the repro i have for this is using a concatadapter via loadstatefooter with remotemediator
s
yes, my setup is remotemediator + loadstate header and footer. not sure if relevant, but in additional to typical setup, it also happens on "chat" mode - in my case, recycler with reverseLayout = true
do you imagine some workaround for this case for now? I tried to stop recycler from scrolling until initial data is loaded, but it doesn’t do anything in my case
13 Views
Open in Slack
PreviousNext
Powered by