Hello! I have a problem with Paging v3 library on ...

# android

[Nikita Khlebushkin]

Hello! I have a problem with Paging v3 library on Android, here's what happens:
When the app is opened, the list is shown correctly and everything is great, but if I call invalidate() method of PagingSource, then it starts spamming load method of PagingSource (and doesn't even wait until the job is finished, cancels it and starts anew). This is the stacktrace:

2021-07-14 02:23:46.189 15215-15254/com.xlebnick.kitties.debug D/OkHttp: --> GET <https://myapi.com/v1/images/search?page=0&limit=24&order=ASC>
2021-07-14 02:23:46.189 15215-15254/com.xlebnick.kitties.debug D/OkHttp: x-api-key: 9b7e282d-2a67-4c7b-a9fd-3f3e4056e949
2021-07-14 02:23:46.189 15215-15254/com.xlebnick.kitties.debug D/OkHttp: --> END GET
2021-07-14 02:23:46.189 15215-15254/com.xlebnick.kitties.debug D/OkHttp: <-- HTTP FAILED: java.io.IOException: Canceled
2021-07-14 02:23:46.189 15215-15215/com.xlebnick.kitties.debug W/System.err: kotlinx.coroutines.JobCancellationException: StandaloneCoroutine was cancelled; job=StandaloneCoroutine{Cancelling}@6c3004

And this is my implementation:

class KittiesPagingSource(
    private val repository: Repository,
    var breedFilter: Breed? = null,
    var likedKitties: List<Like> = listOf()
) : PagingSource<Int, Kitty>() {


    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Kitty> {
        return try {
            // Start refresh at page 1 if undefined.
            val nextPage = params.key ?: 1

            val filters = if (breedFilter != null) listOf(breedFilter!!) else null
            var kitties: List<KittyRemoteModel> = listOf()
            try {
                kitties = repository.fetchKitties(nextPage, KITTIES_PAGE_SIZE, filters)
            } catch (e: Throwable) {
                e.printStackTrace()
            }

            val newKitties = kitties.map { kitty ->
                kitty.asKitty(likedKitties.any { it.imageId == kitty.id })
            }

            LoadResult.Page(
                data = newKitties,
                prevKey = if (nextPage == 1) null else nextPage - 1,
                nextKey = if (newKitties.isEmpty()) null else nextPage + 1
            )
        } catch (e: Exception) {
            LoadResult.Error(e)
        }
    }

    override fun getRefreshKey(state: PagingState<Int, Kitty>): Int? {
        return state.anchorPosition?.let {
            if (it < state.config.initialLoadSize) {
                // if anchor position is less than initial loading count then download from the beginning
                0
            } else {
                // otherwise load a page around anchorPosition using initialLoadSize
                (it - state.config.initialLoadSize / 2).coerceAtLeast(0)
            }
        }
    }
}

private val kittiesPagingFactory = InvalidatingPagingSourceFactory {
        KittiesPagingSource(repository, breedFilter, likedKitties.value ?: listOf())
    }
    private val kittiesPager =
        Pager(PagingConfig(pageSize = 6), pagingSourceFactory = kittiesPagingFactory)
    val kitties: Flow<PagingData<Kitty>> = kittiesPager
        .flow
        .cachedIn(viewModelScope)

What am I doing wrong?

[dustin]

So cancellation seems to be WAI as invalidation cancels the previous generation and creates a new one to reload data.

[dustin]

Do you mean that you are invalidating in a loop over and over again?

[dustin]

How / when are you calling .invalidate() ?

[Nikita Khlebushkin]

No, the invalidation is only called once

[Nikita Khlebushkin]

I invalidate when the second batch of data comes which changes the representation (I take data A and data B and display AB, but until B came, I show only A)

[Nikita Khlebushkin]

I put a breakpoint on kittiesPagingFactory.invalidate() , and it's been only fired once

[Nikita Khlebushkin]

What does WAI mean?

[ephemient]

working as intended

[Dustin Lam]

What is the loadType / subclass of LoadParams passed to .load()? is it looping between prepend and append

[Nikita Khlebushkin]

image.png

[Nikita Khlebushkin]

is it looping between prepend and append

If I understand the question correctly, then yes, because it keeps updating the flow, and therefore, the RecyclerView

[Nikita Khlebushkin]

The type of LoadParams that is sent to load is always androidx.paging.PagingSource$LoadParams$Refresh

[Nikita Khlebushkin]

So no prepend/append

[Dustin Lam]

So this implies an invalidation loop, I can take a look if you have a sample you can share

[Dustin Lam]

To be clear, cancellation is expected, but spamming load is not 🙂

[Nikita Khlebushkin]

Sure, I will create a minimum reproducible code in a bit

[Nikita Khlebushkin]

https://github.com/MrXlebNick/SpammingInvalidate

[Nikita Khlebushkin]

Please let me know if something is missing

[dustin]

I think this is a bug with InvalidatingPagingSourceFactory

[dustin]

To confirm, can you try something like

[dustin]

public class InvalidatingPagingSourceFactory<Key : Any, Value : Any>(
    private val pagingSourceFactory: () -> PagingSource<Key, Value>
) : () -> PagingSource<Key, Value> {
    private val pagingSources = mutableListOf<PagingSource<Key, Value>>()


    override fun invoke(): PagingSource<Key, Value> {
        return pagingSourceFactory().also { pagingSources.add(it) }
    }

    public fun invalidate() {
        for (pagingSource in pagingSources.toList()) {
            if (!pagingSource.invalid) {
                pagingSource.invalidate()
            }
        }

        pagingSources.removeAll { it.invalid }
    }
}

[dustin]

Fix won't make it for next release, but should for the one after (3.1.0-alpha04)

[Nikita Khlebushkin]

Works like a charm, thanks a lot!