I often find myself having to load a global resource asynchronously, where it only needs to be loade...

# coroutines

[Martin Devillers]

I often find myself having to load a global resource asynchronously, where it only needs to be loaded once, but it could potentially fail. An example is the delta with the server time, so that I can implement an NTP. For those cases, I implemented a DeferredCache with an await method which when called:
- if no coroutine to load the resource has been launched, it starts a new one
- if a coroutine to load the resource has finished successfully, it returns its result
- if a coroutine to load the the resource has finished with an error, it restarts a new one
- if a coroutine to load the resource has been launched and is still active, it awaits its result
My current implementation looks like this

class DeferredCache<T: Any>(
    private val coroutineScope: CoroutineScope = GlobalScope,
    private val block: suspend CoroutineScope.() -> T
) {
    private var deferredValue: Deferred<T>? = null
    private val deferredValueLock = Mutex()

    private val usableDeferredValue: Deferred<T>?
        get() = deferredValue?.takeUnless { it.isCompletedExceptionally }

    suspend fun await(): T {
        val deferredValue = usableDeferredValue ?: deferredValueLock.withLock {
            usableDeferredValue ?: coroutineScope.async(NonCancellable, block = block).also { deferredValue = it }
        }
        return deferredValue.await()
    }
}

Can you think of a more optimal solution? Anyone running into this scenario as well? Might it be worth adding a “standard” way to do this?

[spand]

We do something similar (expect we have an unbounded number of them built on a ConcurrentMap). I can see the use case for a suspendable lazy but anything more than that and you get into details that are heavily domain specific. ie. what to do on an error, how long do you cache the error, how long do you cache the result etc.

[cbruegg]

Question: Shouldn't deferredValue be @Volatile?

[Martin Devillers]

It’s not so much the lazy aspect which is important (because that’s supported by the coroutineStart parameter in coroutine builder functions), but it’s rather the “auto-restart” aspect which is missing from the framework.

[Martin Devillers]

@cbruegg In the blocking world, maybe, but I don’t think it’s an issue with coroutines

[cbruegg]

@Martin Devillers Are you sure? deferredValue may be accessed from any thread and is not accessed using JVM monitor synchronization. So what one thread writes, another doesn't necessarily have to see, from my understanding.

[spand]

@cbruegg If a thread sees null in deferredValue it proceeds to take the lock in which case it should get the updated value (ie. I dont see a problem)

[cbruegg]

@spand The thing is that it's not a JVM lock, but one implemented using coroutines, so there's no happens-before barrier. Though I might also be misunderstanding. See https://stackoverflow.com/a/3519736/1502352

[cbruegg]

Without acquiring a JVM monitor lock, a thread might only write deferredValue to its own cache, not to main memory, which means that other threads wouldn't see the change until the cache is flushed.

[cbruegg]

Thus, if I'm reading this right, there's a chance that block can be executed more than once.

[spand]

I assume the withLock establishes the happens before relation. It javadoc says:

JVM API note:
 * Memory semantic of the [Mutex] is similar to `synchronized` block on JVM:
 * An unlock on a [Mutex] happens-before every subsequent successful lock on that [Mutex].
 * Unsuccessful call to [tryLock] do not have any memory effects.

[cbruegg]

Oh, that's interesting, thanks! I didn't know that.

[spand]

@Martin Devillers The restart logic is still quite domain specific (because it is bad in general) in that it will probably hammer a service if it is already struggling.

[spand]

An exponential back off would be a much safer default

[Martin Devillers]

Good point, it could be configurable with a restart policy though. I ask the question because I’m curious to see whether it’s a common enough problem to warrant a common solution.

[spand]

Maybe. In the general case however it quickly extends to using stuff like the circuit breaker and bulkhead patterns

[spand]

and that is more fitting in a library than coroutines-core 😉

[Martin Devillers]

👍

[spand]

Not saying there isnt a good middle road that could be taken 😉

[elizarov]

You can do much simpler. Class is not needed. Just write a function and use `async(start = LAZY)`: https://gist.github.com/elizarov/d16664152fa65f6bd794319a5b1ed9ab

[Martin Devillers]

It’s not quite the same. In my implementation, when awaiting the value, if the request fails then the error is thrown and there isn’t another request started immediately. Another request will only be started when await is called again.

[elizarov]

Ah. I see.