https://kotlinlang.org logo
Join Slack
Powered by
hi, is it possible to have a exponential jitter re...
# arrow
k
hi, is it possible to have a exponential jitter retry schedule that has a whileOutput and a untilInput condition? It seem that only the last one configured wins in the chain.
Copy code
val exponentialBackoff: Schedule<Throwable, Double> = Schedule.exponential(TimeUnit.SECONDS.toNanos(2L).toDouble())

val retrySchedule = exponentialBackoff.jittered()
                    .whileOutput {
                       TimeUnit.NANOSECONDS.toSecods(it.toLong()) < 10.0
                    }
                    .untilInput { 
                        when (it) {
                             is ApiException -> it.statusCode in 400 until 500
                             ...
                             else -> false
                        }
the above never stops on the whileOutput condition (using this at work and don't have the code handy, it looks at least similar to above :D)
s
Hey @kartoffelsup, that is strange 🤔 I cannot imagine such limitations, so that might be a bug.
k
I will try to reproduce in a minimal sample later on today, haven't had a chance to do so yet
It is reproducible: https://gist.github.com/kartoffelsup/896edcb154a13f2b5dc9cf639ac08311 but still unsure if that is not expected behaviour 🙈
I've worked around this by combining both in a single 
check
predicate
Should I create an issue for this?
s
Yes, please 🙏
k
One more thing I noticed is that the jitter is not applied to the delay passed to the 
check
predicate function. Is that intentional or am I using it wrong?
Copy code
val exponentialBackoff: Schedule<Throwable, Double> = Schedule.exponential(TimeUnit.MILLISECONDS.toNanos(500L).toDouble())

    val retrySchedule: Schedule<Throwable, Double> = exponentialBackoff.jittered(suspend { 5.0 })
        .whileOutput { del ->
            println(TimeUnit.NANOSECONDS.toMillis(del.toLong()))
            true
        }

    val start: Instant by lazy { Instant.now() }

    val result: Either<Throwable, Unit> = retrySchedule.retryOrElseEither({
        println("Elapsed: ${Duration.between(start, Instant.now())}")
        throw RetryableException("f")
    }) { t: Throwable, _ -> t }

    result.fold(
        ifLeft = { println("Error: $it") },
        ifRight = { println("Success") }
    )
#################################
Elapsed: PT0.000016289S
1000
Elapsed: PT5.019987402S
2000
Elapsed: PT15.02116669S
4000
Is there a way to retry without having to throw an exception? Like returning an either from the function or something?
s
Yes, you can nest 
retry
into 
either
blocks, and it will retry 
Either.Left.bind
as well as exceptions. We're actually not entirely happy with the APIs of 
Schedule
, but finding a better API that matches the same functionality is tricky.
Copy code
fun main() = runBlocking<Unit> {
  either {
    Schedule.recurs<Throwable>(3).retry {
      println("Attempting")
      raise<Int>("failure")
    }
  }.also(::println)
}

Attempting
Attempting
Attempting
Attempting
Either.Left(failure)
k
Nice, thanks!
s
Oh, I use 
raise
here which is Arrow 2.x.x. 
shift
or 
bind
works exactly the same 😉
k
Sorry, I can't seem to get it to work like that. :( Do you have an example with current release arrow with retry only please?
s
Strange.. this works for me with 1.1.3. I included imports.
Copy code
import arrow.core.Either
import arrow.core.continuations.either
import arrow.fx.coroutines.Schedule
import arrow.fx.coroutines.retry
import kotlinx.coroutines.runBlocking

fun main() = runBlocking<Unit> {
  either {
    Schedule.recurs<Throwable>(3).retry {
      println("Attempting")
      Either.Left("failure").bind()
    }
  }.also(::println)
}
This snippet has the same output.
k
Okay I got it working as well now. My 
check
condition had to be extended to return true for 
ShiftCancelledException
^^
s
Ouch.. yes, those are still some quirks we still love to straighten out. It should be simpler to retry / repeat 
Raise/EffectScope
based code.
k
That is arrow 2.0? :) Will have a look at that :D sounds intriguing
s
We don't have anything concrete for Schedule yet. It's a quite tough nut to crack, and it seems I have a bit of tunnel vision on this API but I don't want to give up any features 😅
So far I've written 10+ new implementations, but not entirely happy with any of them
k
I guess until then it's more explicit to throw a exception to force a retry than to have shiftcancelled thrown though:D
I like schedule a lot, I find it quite complex so I understand how that could be a tough nut ^^
s
What are the features you want out of Schedule? If you can completely ignore what it looks like right now. What kind of feature set are you hoping to get out of it?
k
Currently I'm just looking for retries with exponential Backoff and jitter where I can configure the retry decision based on throwable and/or either left with a configurable max elapsed time and max attempts. Not sure if that's schedules purpose though
s
Yes, that is indeed mostly what it's used for but we're thinking that 
Output
it typically not relevant. As long as you have good ways of configuring a MAX timeout. Similarly to logging, there can be other alternatives, for example (looking at my current WIP).
Copy code
repetition(
  2.seconds
  Policy.exponential().max(10.seconds)
).retry { index ->
   Either.Left("failure")
     .tapLeft { println(it) }
     .bind()
}
So far it's missing a good way to add a predicate for 
Throwable
but that could for example be a lambda passed to 
retry
.
k
I'm passing a Singleton of schedule around that I configure once and reuse everywhere. Would probably also work in passing the throwable condition around but I like the central config way in this case. Albeit quite implicit 🙈
s
Right, the 
repetition
is here the singleton value. I guess the 
Predicate
could also fit in there.. The same could not be done with for a predicate with 
E
since you only know that at the call-site.
280 Views
Open in Slack
PreviousNext
Powered by