Use the new raise DSL / context receivers; have se...

# arrow

[Chris Lee]

Use the new raise DSL / context receivers; have several use-cases where a list is processed, the goal being to return the first successful (non-raising) element, or the first error (you can imagine variations on that logic). Having to create variations of this:

@OptIn(ExperimentalTypeInference::class)
@RaiseDSL
public inline fun <Error, A, B> Raise<Error>.mapFirstOrNull(
    iterable: Iterable<A>,
    @BuilderInference transform: RaiseAccumulate<Error>.(A) -> B
): B? {
    val error = mutableListOf<Error>()
    for (item in iterable) {
        fold<NonEmptyList<Error>, B, Unit>(
            { transform(RaiseAccumulate(this), item) },
            { errors -> error.addAll(errors) },
            { return@mapFirstOrNull it }
        )
    }
    return error.toNonEmptyListOrNull()?.let { raise(it.first()) }
}

…are there alternate mechanisms to consider for this?

[Youssef Shoaib [MOD]]

Hmmm. Looks like a withError({ e: Nel<Error> -> e.first() }) { ... } Is what you're looking for, and then mapOrAccumulate on the inside.

[Youssef Shoaib [MOD]]

Alternatively, mapOrAccumulate where the combine function is { a, _ -> a }

[Chris Lee]

close. that doesn’t quite do it - any errors would then be raised, ignoring the success case (accumulate errors until success).

[Youssef Shoaib [MOD]]

Are you sure that wouldn't do it? To me it looks like you have the same structure of mapOrAccumulate, but with the raise being changed to take the first error only. I.e. you're transforming the raised list and taking its first item

[Chris Lee]

perhaps - let me give it a try again…

[Chris Lee]

The challenge is that withError has already decided to raise the Nel<Error> list - the best we can do there is transform it, which doesn’t satisfy this use case (if there is a successful record use that, other wise raise error).

// fails: java.lang.AssertionError: Expected to succeed, but raised 'Error'
        raiseTest<String>("accumulate") {
            val list = listOf("a", "b", "c")
            withError({ errors: Nel<String> -> errors.first() }) {
                mapOrAccumulate(list) {
                    when (it) {
                        "c" -> 1
                        else -> raise("Error")
                    }
                }
            }.shouldBe(1)
        }

        // succeeds
        raiseTest<String>("mapFirstOrNull") {
            val list = listOf("a", "b", "c")
            mapFirstOrNull(list) {
                when (it) {
                    "c" -> 1
                    else -> raise("Error")
                }
            }.shouldBe(1)
        }

[Chris Lee]

the real need here is for some form of raise-enabled iteration that short-circuits on a termination condition:
• failures? count, criteria. first failure. accumulate all failures.
• successes? count, criteria. first success, first that matches criteria, etc.
…similar to takeWhile on Sequence perhaps, with a takeUntil variant (that includes the terminal element) - but handling both success and error paths.
Transforming error is orthogonal and well-handled by withError , assuming we can control what constitutes an iteration error.

[Alejandro Serrano.Mena]

I would suggest using just the list operations

public fun <Error, A, B> Raise<Error>.mapFirstOrNull(
  iterable: Iterable<A>,
  transform: Raise<Error>.(A) -> B
): B? = iterable.firstOrNull().map { transform(it) }

maybe with this you can also see that there's no need to "thread" the Raise<Error>, since it's just moving from the context, so you could even write

public fun <A, B> Iterable<A>.mapFirstOrNull(
  transform: (A) -> B
): B? = firstOrNull().map(transform)

and use it in a place in which you need a Raise<E> without any problem

[Alejandro Serrano.Mena]

takeaway: mapOrAccumulate is not the only function to operate with errors on lists, the regular map is the one embodying the "fail first" strategy

[Chris Lee]

thanks. those options don’t provide the correct behaviour. The goal is to short-circuit the processing of a list on the first successful element.
Looking for better options for handling this - creating a custom method for every combination of short-circuit criteria will be a combinatorial explosion.

class RaiseAccumulateTest :
    FunSpec({
        val list = listOf("a", "b", "c", "d", "e")
        /*
        Goal: return the first successful element, or the raised errors

        For the above list, the first successful element is "c"; this should be returned
        without raising any errors.  Iteration short-circuits at "c": "d" and "e" will not be processed.

        In the event there were no successful elements, all errors should be raised.

        In the event the list is empty (no elements), null should be returned.
         */

        // fails: java.lang.AssertionError: Expected to succeed, but raised 'Error a'
        raiseTest<String>("arrow: mapOrAccumulate") {
            withError({ errors: Nel<String> -> errors.first() }) {
                // issue: mapOrAccumulate does not support short-circuiting iteration; all
                // entries are processed
                // all raised errors are re-raised as a Nel<Error>
                // hence unable to short-circuit on the first successful element
                mapOrAccumulate(list) {
                    when (it) {
                        "c" -> 1
                        else -> raise("Error $it")
                    }
                }
            }
                .shouldBe(1)
        }

        // fails: java.lang.AssertionError: Expected to succeed, but raised 'Error a'
        raiseTest<String>("custom: mapFirstOrNull2") {
            mapFirstOrNull2(list) {
                when (it) {
                    "c" -> 1
                    else -> raise("Error $it")
                }
            }
                .shouldBe(1)
        }

        // java.lang.AssertionError: Expected to succeed, but raised 'Error a'
        raiseTest<String>("custom: mapFirstOrNull3") {
            list
                .mapFirstOrNull3 {
                    when (it) {
                        "c" -> 1
                        else -> raise("Error $it")
                    }
                }
                .shouldBe(1)
        }

        // succeeds
        raiseTest<Nel<String>>("custom: mapFirstOrNull") {
            mapFirstOrNull(list) {
                when (it) {
                    "c" -> 1
                    else -> raise("Error $it")
                }
            }
                .shouldBe(1)

            mapFirstOrNull(emptyList<String>()) {
                when (it) {
                    "c" -> 1
                    else -> raise("Error $it")
                }
            }
                .shouldBe(null)
        }
    })

// as supplied on slack, corrected with "?.let" instead of ".map"
internal fun <Error, A, B> Raise<Error>.mapFirstOrNull2(
    iterable: Iterable<A>,
    transform: Raise<Error>.(A) -> B
): B? = iterable.firstOrNull()?.let { transform(it) }

// as supplied on slack, corrected with "?.let" instead of ".map"
internal fun <A, B> Iterable<A>.mapFirstOrNull3(transform: (A) -> B): B? =
    firstOrNull()?.let(transform)

/**
 * Returns the first success (non-error-raising) result of the supplied Iterable or the raised
 * errors. In the event the Iterable is empty null is returned.
 */
@OptIn(ExperimentalTypeInference::class)
@RaiseDSL
internal inline fun <Error, A, B> Raise<Nel<Error>>.mapFirstOrNull(
    iterable: Iterable<A>,
    @BuilderInference transform: RaiseAccumulate<Error>.(A) -> B
): B? {
    val error = mutableListOf<Error>()
    for (item in iterable) {
        fold<NonEmptyList<Error>, B, Unit>(
            { transform(RaiseAccumulate(this), item) },
            { errors -> error.addAll(errors) },
            {
                return@mapFirstOrNull it
            }
        )
    }
    return error.toNonEmptyListOrNull()?.let { raise(it) }
}

[Alejandro Serrano.Mena]

I think this is a bit complicated one, because you want the Raise of each element to "depend" on the previous one

internal fun <Error, A, B> Raise<Nel<Error>>.mapFirstOfNull(
  iterable: Iterable<A>,
  @BuilderInference transform: Raise<Error>.(A) -> B
) {
  val errors = mutableList<Error>()
  iterable.forEach { element ->
    withError({ e -> errors.add(e) }) { 
      return@mapFirstOfNull transform(element)
    }
  }
  return errors.toNonEmptyListOrNull()
}

[Chris Lee]

Thanks; with a few tweaks that provides a working, more concise implementation; however, the underlying problem remains of hard-wiring the logic in each case, not being able to compose it.

@OptIn(ExperimentalTypeInference::class)
internal fun <Error, A, B> Raise<Nel<Error>>.mapFirstOrNull4(
    iterable: Iterable<A>,
    @BuilderInference transform: Raise<Error>.(A) -> B
): B? {
    val errors = mutableListOf<Error>()
    iterable.forEach { element ->
        // doesn't compile; withError must return a Raise<Error> to be raised unconditionally
        //        withError({ e -> errors.add(e) }) {
        //            return@mapFirstOrNull4 transform(element)
        //        }

        recover({
            return@mapFirstOrNull4 transform(element)
        }) {
            errors.add(it)
        }
    }
    if (errors.isEmpty()) return null
    raise(errors.toNonEmptyListOrNull()!!)
}

[Youssef Shoaib [MOD]]

You could do a method wrapping mapOrAccumulate where in the transform you early-return. I think that would give the desired behaviour

[Alejandro Serrano.Mena]

in theory you should also be able to do

fun <Error, A, B> Raise<Nel<Error>>.mapFirstOrNull5(
    iterable: Iterable<A>,
    @BuilderInference transform: Raise<Error>.(A) -> B
): B? {
  mapOrAccumulate(iterable) { return@mapFirstOrNull5 transform(it) }
  return null
}

• if we get a correct value after transform, the function returns early
• we get to return null only if mapOrAccumulate is not erroneous nor has returned early, which means the iterable must be null

[Chris Lee]

thanks; that’s a working, even-more-concise solution. Still seems that Arrow is missing capabilities around short-circuiting iterations, requiring custom logic.

@OptIn(ExperimentalTypeInference::class)
internal inline fun <Error, A, B> Raise<NonEmptyList<Error>>.mapFirstOrNull5(
    iterable: Iterable<A>,
    @BuilderInference transform: RaiseAccumulate<Error>.(A) -> B
): B? =
    mapOrAccumulate<Error, A, B>(iterable) {
            return@mapFirstOrNull5 transform(it)
        }
        .firstOrNull()

[Alejandro Serrano.Mena]

I don't think that's completely fair: here you are short-circuiting the map using an early return, which is exactly what you would do using regular Kotlin

[Chris Lee]

In regular Kotlin I’d have all kinds of operations such as first {…} etc; those aren’t available in conjuction with Arrow error accumulation.

[Alejandro Serrano.Mena]

I see, what you are missing then is some kind of firstOrAccumulate...

[Youssef Shoaib [MOD]]

Perhaps a forEachAccumulating method would be desirable as a primitive? It would allow for such functions to be easily defined, and thus Arrow could have firstOrAccumulate, lastOrAccumulate etc easily

[Chris Lee]

hmmm. forEachAccumulating is interesting. Would be nice if it had the flexibility to support various operations - first, last, takeWhile, etc.

[Alejandro Serrano.Mena]

mapOrAccumulate can work as such primitive, since forEachAccumulating is just the version in which each element has a Unit transformation

[Chris Lee]

I don’t believe so. mapOrAccumulate processes the whole list, with limited means of short-circuiting - we hacked that for first by returning, but that won’t easily work for say takeWhile.

[Alejandro Serrano.Mena]

what I mean is that if you can write it using forEachAccumulating, you can write it using mapOrAccumulate

[Chris Lee]

yes, I suppose you could make that work - wouldn’t be as elegant as having slightly-lower-level support from forEachAccumulating.

[Chris Lee]

another dimension to consider for forEachAccumulating - would/should it support criteria on the errors themselves?
atm stdlib provides list.takeWhile { … } to stop at a certain point - we can provide an Arrow version for the success path, but what if the criteria is something like “stop after 5 errors” or “stop after error x encountered” on the error path?

[Alejandro Serrano.Mena]

I don't think this is the right mindset: Raise is about working implicitly with the errors, in the case you mention I think you should rather use either and build this logic yourself

[Alejandro Serrano.Mena]

you cannot have a function which cover every possible case

[Chris Lee]

btw, here is one use-case that now reads quite well w/ firstOrAccumulate (formerly known as mapFirstOrNull5); it loops through constructors on a class, trying each one in turn until one succeeds (or all fail).

return withError({ it.first() }) {
            firstOrAccumulate(singleArgConstructors) { constructor ->
                val args =
                    constructor.convertArguments(
                        mapOf(constructor.parameters[0] to sourceNode),
                        context,
                        ConversionError::ValueClassParameterErrors
                    )
                catch({ constructor.callBy(args) }) { thrown ->
                    raise(ValueClassConstructorException(target.type, thrown))
                }
            }
        } ?: raise(ConversionError.NoValueClassConstructor(valueClass))

[Chris Lee]

thanks for all the insights.