Am I right in thinking that even memory cache in A...

# apollo-kotlin

[wasyl]

Am I right in thinking that even memory cache in Apollo is raw cache, and reading from it involves deserializing jsons into objects? I think I'm observing a bit of a performance drop in our app because we're kicking off multiple watchers for some really heavy responses, and I want to confirm that Apollo doesn't do any object-level caching.
And to confirm, I assume there's no API in Apollo to add such object-level cache, right? Asking because implementing it on app level is tricky due to https://kotlinlang.slack.com/archives/C01A6KM1SBZ/p1697617478848379?thread_ts=1684754445.276909&cid=C01A6KM1SBZ, which causes reused and fresh observers to have different data

[mbonnin]

Memory cache is storing `Record`s which are basically Map<String, Any> . These Record s are then read with MapJsonReader which exposes them through a JSON API but under the hood it's not serialized to utf8

[mbonnin]

So there is still calling of all the adapters, etc... which might have a bigger cost than just looking up the instance reference but it's not doing full JSON serialization either

[wasyl]

Interesting, thanks! So it's not doing actual deserialization like I thought, but perhaps still accessing many layers of `Map`s and regular Apollo stuff might take time here, I suppose I need to profile it a bit then.

I'd still consider a full object cache, but I realized that even https://github.com/apollographql/apollo-kotlin/issues/4974 might not help with the linked thread, because the affected cache keys are not only the removed ones, but also everything that depended, directly or indirectly, on the removed keys 😕

[mbonnin]

Problem with object cache is that objects are not shared between queries so you can't have watchers, right?

[wasyl]

Not sure I understand, I'd still base the source of truth on Apollo, I just want to basically deduplicate watch calls

[wasyl]

Instead of kicking off a new watch call in Apollo for a given Query<Query.Data>, I want to attach to a SharedFlow that I'd manage, backed by apollo's watch

[mbonnin]

Ohh to save some time when you have multiple watchers of the same query?

[wasyl]

yes exactly 🙂

[mbonnin]

Can this "just" be a matter of shareIn ?

[wasyl]

It pretty much is, except for the linked thread:

[wasyl]

So if an object changes in Apollo cache, it'd still be emitted to all observers. The challenge I encountered now is that

val query = SomeQuery
val foo = apollo.watch(SomeQuery)
apolloStore.remove(deeplyNestedIdOfObjectInSomeQueryResponse)
val bar = apollo.watch(SomeQuery)

// foo.value = some response
// bar.value = null

[mbonnin]

Right 👍 But that's an issue even with a single listener right? You don't get emissions if you remove a deeply nested object programmatically

[wasyl]

That's right, yes 🙂

[wasyl]

Generally the fresh watch (bar) will try to read from cache, realize it can't satisfy all objects, return null

The existing watch (foo) will not be notified of the deeply nested object removal because remove doesn't publish cache changes for SomeQuery

[wasyl]

And what just happened is that when I added caching layer, my tests started failing because I'm not doing a fresh watch anymore, so the app behavior changed

[mbonnin]

yep, that's annoying

[mbonnin]

We'll need https://github.com/apollographql/apollo-kotlin/issues/4974. I'll give it a try this afternoon. It's been a while I haven't dug into the cache APIs so I wonder how much of a rabbit hole this will be but maybe there'll be a good surprise 🤞

[wasyl]

And I don't know what the proper behavior/solution is, I'm just explaining what I'm trying to do 😄 One option is to accept that we'll keep "stale" responses, there's a chance our app won't break horribly 😛

[wasyl]

I looked briefly at code around https://github.com/apollographql/apollo-kotlin/issues/4974 but I wouldn't know where to start 😕 But yeah just to clarify, it wouldn't be enough to publish changes for all the removed keys, but also for all objects referencing the removed keys. Which kind of sounds like iterating over entire cache 😬

[mbonnin]

Ah yes, that's the rabbit hole there!

[mbonnin]

How does SQL do it?

[wasyl]

Oh and not just objects referencing the removed keys, but entire tree up to the root for everything that touches any of the removed keys somewhere

[wasyl]

no idea 😄

[mbonnin]

Yea, maybe not a 1hour thing then 😅

[mbonnin]

We'd need to keep track of some kind of "backreference" or so....

[wasyl]

How does SQL do it?

Seems like it doesn't https://stackoverflow.com/a/24713124

[mbonnin]

I asked around internally, will update this thread if anyone comes up with a solution

[mbonnin]

For the record, this is what they do on the web: https://www.apollographql.com/docs/react/data/mutations#refetching-after-update

[mbonnin]

So either:
• manually updating all the impacted places
• or passing a list of affected queries that need to be refetched from the network

[wasyl]

or passing a list of affected queries that need to be refetched from the network

that crossed my mind as a workaround, I realized it may not be reasonable to expect Apollo to determine all places to invalidate

[wasyl]

Anyway web approach seems reasonable. And circling back to the original problem, am I right assuming that any sort of SharedFlow-like cache would be out of scope for Apollo itself?

[mbonnin]

I'm not sure what that would look like

[mbonnin]

Something we could do is pass an argument to watchers to listen to all cache updates

[mbonnin]

Then the older watchers would try to re-execute the query from the cache on every change. You'll get all the updates but it's probably quite inefficient

[mbonnin]

or we could add a ApolloStore.refetchAllWatchers() or so

[wasyl]

would that be already possible with something like ApolloStore.publish(rootKey)?

[mbonnin]

If you modify something that is known to have implications in many parts of the graph, then force refetch all the watchers

[mbonnin]

Maybe it's as simple as that indeed. I'd have to double check if the root key is in the list of watched keys

[wasyl]

in any case, thanks for all the insights, I'll word under the assumption I have to either keep old values or figure out how to refresh what makes sense (and only if I absolutely need I'll open an issue for refetchAllWatchers-like functionality)

About

I'm not sure what that would look like

same, just wanted to check it's not on the roadmap somewhere

[mbonnin]

Thanks for the discussion!

[mbonnin]

A few thoughts after letting that soak a bit and discussing it with a few other folks:
• ApolloStore.publish(rootKey) won't work. The watchers only listen to scalar (leaf) fields, not object fields. Typically in the root key case, we don't want to get notified every time a new query queries a new root field
• The web has also a garbage collector. From what I understand, this works by scanning the whole cache. If we can make it fast enough, that could maybe work. Trigger a GC and notify all dependents.
• Another approach is to have every mutation return the modified objects (but requires care when writing the mutation as well as backend support)

[wasyl]

About the mutation, in our case I don't think it helps because we're removing fields without doing any operations (basically as a way of invalidating the cache for some object).

Btw I tried

or passing a list of affected queries that need to be refetched from the network

but realized I don't know what I can pass to publish even if I know the query that should be affected 😕 And I don't really want to refetch from network at this point, just make Apollo emit null for currently observed queries

[wasyl]

(GC definitely sounds interesting regardless of this use case btw)

[mbonnin]

I don't know what I can pass to publish even if I know the query that should be affected

Looks like we need to keep track of query names that have an active watcher. Then you could call something like apolloStore.invalidate("GetProduct")