What is the correct way to implement a cache? I'm...

# coroutines

[CLOVIS]

What is the correct way to implement a cache?

I'm thinking of something like:

val cache = HashMap<Id, Deferred<Value>>()
val lock = Semaphore(1)
val cacheJob = SupervisorJob()
val cacheScope = CoroutineScope(cacheJob)

suspend fun get(id: Id) {
  val r = lock.withPermit {
    if (cache[id] == null) {
      val c = CompletableDeferred()
      cacheScope.launch {
        c.complete(getElementFromNetwork(id))
      }
      c
    } else cache[id]
  }

  r.await()
}

I think that's generally safe, but I'm not sure if it will correctly handle all cases (the ID is invalid, ...)

[Dominaezzz]

Mind using triple back ticks instead of single ones? Makes it easier to read.

[Dominaezzz]

You don't need CompletableDeferred here and you've leaked it by not cancelling it.

[CLOVIS]

Ah, I thought it didn't work on mobile. It should be better now.

[CLOVIS]

How can this work without CompletableDeferred? Where is the leak?

[Dominaezzz]

Actually, nevermind that for now. cache will always be empty.

[CLOVIS]

Ah, indeed:

val cache = HashMap<Id, Deferred<Value>>()
val lock = Semaphore(1)
val cacheJob = SupervisorJob()
val cacheScope = CoroutineScope(cacheJob)

suspend fun get(id: Id) {
  val r = lock.withPermit {
    if (cache[id] == null) {
      val c = CompletableDeferred()
      cacheScope.launch {
        c.complete(getElementFromNetwork(id))
      }
      cache[id] = c
    }
    cache[id]
  }

  r.await()
}

[Dominaezzz]

What's wrong with cache[id] = cacheScope.async { getElementFromNetwork(id) } ?

[Dominaezzz]

(Then you don't need CompletableDeferred)

[CLOVIS]

@Dominaezzz if two coroutines get the same ID, two requests are fired to the server (and the result of the last to terminate will override the other)

[CLOVIS]

Maybe the cache map would be better as:
val cache = HashMap<Id, Pair<StateFlow<Boolean>, StateFlow<Value>>>

This way, the UI can subscribe to either the boolean (whether the value is currently being downloaded) and/or the actual value

This way, it's also possible to get the current value even when another coroutine is updating it

[Dominaezzz]

You have the lock so you won't get two requests

[ephemient]

val cache = ConcurrentHashMap<Id, Deferred<Value>>()
suspend fun get(id: Id) = coroutineScope {
    cache.compute(id) { _, deferred ->
        if (deferred == null || try {
            deferred.getCompletionExceptionOrNull()
        } catch (_: IllegalStateException) {
            null
        } != null) {
            async(start = CoroutineStart.ATOMIC) {
                getElementFromNetwork(id)
            }
        } else deferred
    }.await()
}

[ephemient]

that would effectively give you a per-key lock instead of a global lock

[Dominaezzz]

But if the first caller cancels, other waiting callers never get their values.

[ephemient]

any callers waiting on the same request will see a cancellation, yes, but any subsequent callers will start a new request

[ephemient]

if that's a concern you could add a retry in there

[Richard Gomez]

Related?
https://kotlinlang.slack.com/archives/C1CFAFJSK/p1628709269166700

[CLOVIS]

With that version however, you can't see the old value if another coroutine is currently downloading

[CLOVIS]

I guess having a map of StateFlow is the solution here?

[Dominaezzz]

Old value? Didn't know invalidation was involved.

[CLOVIS]

I might want to force a refresh, for example if I know that the value is outdated