I wondered if there was a way to have the resource...

# arrow-contributors

[phldavies]

I wondered if there was a way to have the resource bound the to lifecycle of the calling CoroutineScope to avoid any leaked resources if the finaliser isn’t called. I’m not sure if this is entirely safe wrt structured concurrency but it passes some rudimentary tests (and finalizer is called within the current context):

context(CoroutineScope)
suspend fun <A> Resource<A>.bind(): Pair<A, suspend () -> Unit> =
    allocate().let { (acquired, release) ->
        acquired to guaranteeCase(release)
    }

fun CoroutineScope.guaranteeCase(finalizer: suspend (ExitCase) -> Unit): suspend () -> Unit {
    val completion = CompletableDeferred<Unit>()
    val finalize = async(NonCancellable) { guaranteeCase(completion::await, finalizer) }
    val hookHandle = coroutineContext.job.invokeOnCompletion {
        if (it != null) completion.completeExceptionally(it)
        else completion.complete(Unit)
    }
    return {
        hookHandle.dispose()
        completion.complete(Unit)
        finalize.await()
    }
}

or alternatively without the CoroutineScope receiver/context (but requiring a current Job)

suspend fun <A> Resource<A>.bind(): Pair<A, suspend () -> Unit> =
    allocate().let { (acquired, release) ->
        acquired to guaranteeCase(release)
    }

suspend fun guaranteeCase(finalizer: suspend (ExitCase) -> Unit): suspend () -> Unit {
    val completion = CompletableDeferred<Unit>()
    val finalize = async(NonCancellable) { guaranteeCase(completion::await, finalizer) }
    val hookHandle = coroutineContext.job.invokeOnCompletion {
        if (it != null) completion.completeExceptionally(it)
        else completion.complete(Unit)
    }
    return {
        hookHandle.dispose()
        completion.complete(Unit)
        finalize.await()
    }
}

private suspend fun async(
    context: CoroutineContext = EmptyCoroutineContext,
    start: CoroutineStart = CoroutineStart.DEFAULT,
    block: suspend CoroutineScope.() -> Unit
) = Job(coroutineContext.job).let { job ->
    CoroutineScope(coroutineContext + job)
        .async(context, start, block)
        .also { job.complete() }
}

[simon.vergauwen]

That should work yes. Have you seen this project? https://github.com/arrow-kt/suspendapp

It's like IOApp, and I use this in combination with ResourceScope to achieve the same behavior but through Resource.

suspend fun ResourceScope.dependency(): String = Resource({ "Dependency" }) { a, exitCase -> }

fun main() = SuspendApp {
  resource {
    dependency()
  }.use { awaitCancellation() }
}

[simon.vergauwen]

It also allows you to return an unwrapped Resource if you have ResourceScope in your context.

The reason for doing this is so that you can compose the handlers correctly throughout the entire ResourceScope however deeply it may be nested.

[simon.vergauwen]

Actually I'm not sure if binding it to CoroutineScope is (safely) possible 🤔

Relying on coroutineContext.job.invokeOnCompletion is not safe in all cases, since it heavily depends on where it's called, and due to the implicit nature of Job and CoroutineContext there is no type-safety to protect your from incorrect usage.

[phldavies]

re: SuspendApp - Yes I have 🙂 I’ve had a play a few weeks back but wasn’t sure how stable I should consider it at this point. I think the difference here is SuspendApp ties the lifecycle of a Resource (and thus your application) to the platform shutdown hooks (handling sigint etc).

[simon.vergauwen]

Right, but some of the patterns in the docs and what I shared above might also accomplish what you want. If I understood correctly, being able to wrapped unwrapped code for Resource?

[simon.vergauwen]

This PR also has more details about that style, and how I hope to improve it in 2.0. If you have any feedback that be awesome 🙏

https://github.com/arrow-kt/arrow/pull/2786

[phldavies]

re implicit job/context - that was my feeling too (hence initially using context(CoroutineScope) to enforce the usage).
AFAIK the invokeOnCompletion should always be invoked but there’s no guarantee about the scope/thread of the invocation - which is why I only used it to signal via the CompletableDeferred which is being waited on in the correct scope.

[simon.vergauwen]

I consider SuspendApp stable, it was originally developed in production for JVM.

While looking for a solution I also played with job.coroutineContext but it was super delicate if you make any any code changes. The same patterns might apply depending on usage. invokeOnCompletion will always be called, but when implicitly depends on call-site.

[phldavies]

If I understood correctly, being able to wrapped unwrapped code for Resource?

My intent here was to see if we can provide some level of guarantee that a manually allocated resource is finalised at some point given the structured concurrency inherent in kotlin’s coroutines. Mainly to provide some level of parity with cats.effect.Resource.allocated that at least guarantees finalisation on interrupt/failure if not for completion.

[simon.vergauwen]

that at least guarantees finalisation on interrupt/failure if not for completion.

Allocated makes no such guarantees? Once the resource is allocated the responsibility is 100% on the user to call release in all 3 of the cases. Completion, failure or cancellation.

[phldavies]

According to cats documentation it is guaranteed for interrupt or failure. My desire was to have the arrow equivalent provided similar guarantees (or stronger, by ensuring finalization on job/scope completion).

"If the outer F fails or is interrupted, allocated guarantees that the finalizers will be called. However, if the outer F succeeds, it's up to the user to ensure the returned F[Unit] is called once A needs to be released. If the returned F[Unit] is not called, the finalizers will not be run."
https://typelevel.org/cats-effect/api/2.x/cats/effect/Resource.html#:~:text=If%20the%20outer%20F%20fails%20or%20is%20interrupted%2C%20allocated%20guarantees%20that%20the%20finalizers%20will%20be%20called.%20However%2C%20if%20the%20outer%20F%20succeeds%2C%20it%27s%20up%20to%20the%20user%20to%20ensure%20the%20returned%20F%5BUnit%5D%20is%20called%20once%20A%20needs%20to%20be%20released.%20If%20the%20returned%20F%5BUnit%5D%20is%20not%20called%2C%20the%20finalizers%20will%20not%20be%20run.

[simon.vergauwen]

Okay, I think that's phrased a bit ambiguous. They mean failure or cancellation during the allocated operation.

If the outer F fails or is interrupted, allocated guarantees that the finalizers will

be called. However, if the outer F succeeds, it's up to the user to ensure the returned

F[Unit] is called once A needs to be released. If the returned F[Unit] is not called,

the finalizers will not be run.

Can be rephrased, If allocated fails or is interrupted then allocated guarantees calling the finalizers of the already allocated resources within the composed Resource. If allocating the resource succeeds then the responsibility falls unto the user to call the finalizers.

The semantics is the following, and this is the same for Arrow and Cats-effect, and is also why use is recommended by Cats (& Arrow).

Let's say you have a Resource<Persistence> and it exists out of DataSoure and SqlDelight for example. Then we do

val dataSource: Resource<DataSource> = TODO()
fun sqlDelight(dataSource: DataSource): Resource<SqlDelight> = TODO()
fun Persistence(val sqlDelight: SqlDelight): Resource<Persistence> = TODO()

val persistence = resource {
  val ds = dataSource.bind()
  val sqlDelight = sqlDelight(ds).bind()
  Persistence(sqlDelight).bind()
}

Let's say we use the same signature we discussed above, so the same as Cats-effect. Making the types concrete it becomes:

suspend fun Resource<Persistence>.allocated(): Pair<Persistence, suspend (ExitCase) -> Unit>

If this function returns, then the caller is responsibly of calling suspend (ExitCase) -> Unit in the case the user-land-code fails, cancels or completes. It's not possible to call that function based on any failures or cancellations from user-land.

[simon.vergauwen]

So to me it sounds that your usage of the allocated operator of Cats-effect is incorrect, and that is also why I marked it as @DelicateCoroutineApi in Arrow Fx Coroutines. It should only be used from low-level library code.

[simon.vergauwen]

To come back to the other discussion. I think it's possible to attach a Resource to a Job but it feels error-prone to me. In that regard it's probably much safer to use ResourceScope as a marker type to wire this capability in a type-safe way.

[phldavies]

Agreed, this was mostly an investigation of what was possible. Our cats-effect usage of allocated is only within test suites with the suite taking care of finalizing the resource as required (as would our arrow usage be).

Thanks for the detailed response, always appreciated!

[simon.vergauwen]

My pleasure! Thanks for taking such deep looks at our APIs, and brain storming together. This is how we can push Arrow forward together 🙌