mitch
04/01/2023, 10:26 PMFailure<F>
and Success<S>
(F
and S
being generic types) specifically to address and / or recommend how error handling should be done? perhaps cc @elizarov, @simon.vergauwen and @raulraja
I believe there are other production applications like ours for which exceptions are too risky to use. We need to find a formal way to have ergonomic language & compiler support to model successes and failures as generic sealed types. There are currently confusion internally on what is considered idiomatic Kotlin for this topic of error handling which I’m personally am also struggling to answer without pointing to @elizarov’s blog post about kotlin exceptions. That’s the reason why I’m seeking advice internally as well as externally.
For context we are running a relatively large (~180k LOC) mission critical kotlin jvm application in production. For many years now we have been using arrow’s Either<L, R>
to model that with great success. For example, contributors find the code below to be relatively ergonomic.
suspend fun getManagedUser(id: UserId): Either<Fail, ManagedUser> = either {
val user = getUser(id).toEither { Fail.UserNotFound }.bind()
val managedStatus = getManagedStatus(user)
ensure(managedStatus.isManaged) {
Fail.UserIsNotManaged
}
ManagedUser(user, managedStatus)
}
// private functions
private suspend fun getUser(id: UserId): Option<User> = ...
private suspend fun getManagedStatus(user: User): ManagedStatus = ...
For those who aren’t aware what arrow is. They provide some types and constructs that are very useful to cover this gap. either { }
is a type-safe builder for Either<L, R>
, which exposes functions such as Either<L, R>.bind()
and ensure(...)
that fires a special cancellation exception. This builder makes sure that those extension functions can only be used against a common failure type, therefore getting help from the compiler enforcing domain boundaries. The builder safely catch that and project the propagated error into a specific domain-specific Fail
type.
I’ve been following the discussion around this for a couple of years. I’m aware of the limitation of Result<T>
and the risk therein to use throwables when dealing with coroutines (mistakenly catching cancellation exceptions, etc), as well as the recommendation for users to model them as sealed types, e.g. Outcome<F, S>
, Either<L, R>
or a custom Result<E, A>
which I know exists as a published library.
My questions are therefore:
• Is this approach considered to be aligned with the overall best practices with the kotlin community?
◦ If it isn’t why not? Can we document the alternative and the alternative recommendation therein?
◦ If it is, then is there any plans to absorb / introduce something similar e.g. via kotlinx.typesafe
or something like so that would be the gold standard, and to finally close the gap on our need of a typed failure channel?
• If it is considered aligned and idiomatic but there isn’t a plan to introduce it to standard library, would there be any chance from Jetbrains team to express this and perhaps concede and recommend / endorse specific community driven libraries? e.g. something along the lines that “For those looking for error-handling best practices, the team is currently working on it. However, we recommend these community-driven libraries which we endorse as following the recommended best practices and might consider absorbing the patterns to the kotlin language if there’s enough demand within the community: … “. For instance Arrow.kt #arrow https://github.com/arrow-kt/arrow is a fairly popular library right now within the Kotlin community that fills this gap.George
04/02/2023, 3:29 PMsimon.vergauwen
04/02/2023, 3:40 PMIO
in favor of suspend
, embracing DSLs, etc the happier we've become writing FP code in Kotlin (and I think the same goes for Arrow users).
Personally, I don't think the language has to make an official recommendation. on Arrow, FP, or any OSS library / framework. is not a one-fits-all solution. I.e. Spring vs Ktor vs Quarkus vs Vert.X vs ... all are great candidates in Kotlin.
In some cases I also use Result
, and in others Either
, nullable types or whatever fits my use-case best. So an official recommendation might even be counterproductive, "pushing" people to a single solution whilst it might not fit for their use-case.
Arrow is also not exclusive around Either
, and offers many other patterns well known in FP. I am personally not even a fan of the term FP anymore, same for OOP. Since with modern languages these things have faded a bit, and often a mix of put styles is used / found with great success.mitch
04/02/2023, 11:39 PMEither<L, R>
type instead of hand-writing our own.
So.. with that said, I’d like to go back to the proposition so to make sure the intention is not lost. Why doesn’t Kotlin have this (imo) highly relevant type in the language? I believe one thing that we all would likely agree is for domain failure states to be modelled as an explicit return type. Exceptions are exceptional, and should only be used for exceptional circumstances. catch { }
is possibly dangerous as it might catch other unwanted exceptions such as CancellationException
or OutOfMemoryError
. Let’s introduce something like Result<E, T>
- instead of having throwable as its failure type, users will have the liberty to type the error channel. Something like below hypothetical type:
// Sorry, the name clashes with Result<T>,
// decision of using value class, and identifier orders are hypothetical implementation detail
// the language implementer can choose to implement this as sealed type or native union type accordingly
value class Result<E, A> private constructor(val value: E | A) {
fun valueOrNull(): A? = ...
fun isSuccess(): Boolean = ... // kotlin contract, implies value is A
fun isFailure(): Boolean = ... // kotlin contract, implies value is E
// etc...
}
Afterwards perhaps we can then leverage the kotlin type-safe builder to then create a DSL, similarly to how kotlinx flow { }
and Arrow’s either { }
can be constructed and composed. Here I’m drawing inspirations from Arrow:
interface ResultContext<E> {
fun fail(failure: E): Nothing
fun <B> Result<E, B>.valueOrFail(): B // this extension name is hypothetical, in Arrow it's called bind()
}
inline fun <E, A> result(@BuilderInference fn: ResultContext<E>.() -> A): Result<E, A> = ...
// in the caller's code
suspend fun getManagedUser(id: UserId): Result<Fail, ManagedUser> = result {
val user = getUser(id).toResult { Fail.UserNotFound }.valueOrFail()
val managedStatus = getManagedStatus(user)
if (!managedStatus.isManaged) {
fail(Fail.UserIsNotManaged)
}
ManagedUser(user, managedStatus)
}
Having this type will solve a lot of problems and possibly alleviate some confusion on the topic of error handling. I want to understand more on where is the language heading towards?
On this regards, I 100% agree to this.
Personally, I don’t think the language has to make an official recommendation. on Arrow, FP, or any OSS library / framework. is not a one-fits-all solution. I.e. Spring vs Ktor vs Quarkus vs Vert.X vs ... all are great candidates in Kotlin.The reason why I mentioned recommendation is perhaps because in my view error handling is a common principle of software development lifecycle. One can choose how to handle errors which is agnostic to frameworks, but adhere to best practices and principles. In my opinion Arrow provides good example for that and some patterns (e.g. the
either { }
builder) may need to be brought into attention of a broader audience. i.e. it’s not recommending a library per se but recommending the approach taken to be aligned with idiomatic kotlin. If I were to draw some parallels between languages which employs a similar principle around exceptions. Rust has this principle, and also a solution. It provides a language level solution with Result<T, E>
which is either a Ok<T>
or Err<E>
- a typed error channel. This allows segregation of domain-level errors E
and encourage safe programming with values, translating and handling errors between boundaries - i.e. each boundaries would have different E
types. Kotlin in other hand has the right principles but lack clear guidance or language solution. Would be great to get some alignment on the solution might look like. I won’t be surprised if we find many users may have tried reinventing or have reinvented this wheel multiple times… This is where I believe we can benefit from a slightly clearer guidance.ephemient
04/03/2023, 4:14 AMResult<T, E>
type used widely throughout its ecosystem, has issues. There is a wide divergence between library authors who want to provide actionable and recoverable errors (leading to solutions like thiserror to help reduce the boilerplate required), and application authors who want manage many different errors without having to wrap all of their dependencies' error types into a common hierarchy (leading to other solutions like anyhow - from the same author!).ephemient
04/03/2023, 4:16 AMNetworkResult<out T>
|- Success<out T>(value: T) : NetworkResult<T>
\- Error : NetworkResult<Nothing>
|- AuthenticationError : Error
|- ClientError : Error
|- TransportError : Error
\- ServerError : Error
CacheResult<out T>
|- Success<out T>
| |- FromNetwork<out T>(value: T) : Success<T>
| |- FromDisk<out T>(value: T) : Success<T>
| \- FromMemory<out T>(value: T) : Success<T>
\- Error : CacheResult<Nothing>
\- ...
DialogResult<out T>
|- Positive<out T>(value: T) : DialogResult<T>
|- Neutral : DialogResult<Nothing>
\- Negative : DialogResult<Nothing>
etc., plus a boatload of FooParseResult
for different types of Foo
. It just made sense, coming from our pre-Kotlin experiences. Don't use exceptions for anything recoverable, and the modeling is dependent on context. Which I think ends up being pretty similar to elizarov's article? (It hadn't been published yet at the time we were laying out this design.)ephemient
04/03/2023, 4:17 AMmitch
04/03/2023, 6:13 AMXyzResult
type in that case be modelled as either a failure or success with generic type? i.e.
• wouldn’t NetworkResult<out T>
would then be Result<NetworkError, NetworkValue<T>>
?
• wouldn’t CacheResult<out T>
be Result<CacheFailure, CacheSuccess<T>>
?
• and DialogResult<T>
is Result<DialogFailure, Dialog<T>>
The reason why I point this out is because all of a sudden because result is a provided type, we can leverage kotlin’s very own type-safe builder to compose this ergonomically and handle the error types safely between domain boundaries. consider
interface ResultContext<E> {
fun fail(failure: E): Nothing
fun <B> Result<E, B>.valueOrFail(): B // this extension name is hypothetical, in Arrow it's called bind()
}
inline fun <E, A> result(@BuilderInference fn: ResultContext<E>.() -> A): Result<E, A> = ...
then we can write something like below where translation between domain boundaries can happen organically
suspend fun getPositiveDialog(id: DialogId): Result<GetPositiveDialogFailure, CustomDialog> = result {
val dialogData: DialogData? = getDialogDataFromCacheById(id).map { it.value }
.recover { getDialogDataFromNetworkById(id).map { it.value } }
.mapFailure { failure -> /* map the getDialog failure to GetPositiveDialogFailure */ }
.valueOrFail()
ensureNotNull(dialogData) {
GetPositiveDialogFailure.DialogNotFound
}
// kotlin contract, dialogData is no longer null, update cache async
cacheUpdaterCtx.launch { updateCache(dialogData) }
val dialog: Dialog<CustomDialog> = extractDialog(dialogData)
.let { convertDialog<CustomDialog>(it) }
?: fail(GetPositiveDialogFailure.DialogConversionFailure)
when (dialog) {
is Positive -> dialog.value // CustomDialog
is Neutral -> GetPositiveDialogFailure.NeutralDialog
is Negative -> GetPositiveDialogFailure.NegativeDialog
}
}
ephemient
04/03/2023, 6:17 AMResult
type, but every module has its own Error
hierarchy. it does mean you get standard map
functions etc. for the outer Result
(ditto Arrow's Either
) but it doesn't help at all with converting the errors between domains. our codebase has custom mappers going up the chain, and those would still remain even with a typed Result
or Either
.mitch
04/03/2023, 6:22 AMephemient
04/03/2023, 6:24 AMResult<T, E1 | E2 | E3 ...>
then it would be like Java's checked exception propagation (and actually Rust kind of allows you to do that, where ?
will automatically .into()
if possible) but there's a lot of other impacts to the language with thatephemient
04/03/2023, 6:25 AMmitch
04/03/2023, 6:30 AMvalue class Result<E, A> private constructor(val value: E | A) {
fun valueOrNull(): A? = ...
fun isSuccess(): Boolean = ... // kotlin contract, implies value is A
fun isFailure(): Boolean = ... // kotlin contract, implies value is E
// etc...
}
then yeah java code can just call result.valueOrNull()
as pretty much how it does it with java very own Optional<T>
Taking a step back. It’s actually this conversation that I’m after. Notice how we finally are both talking about domain boundaries and how errors should be translated, by whom, where, and when. Not only that - we can compose complex control flows of programs by leveraging kotlin language with result { }
builder.
This is what I’m after of this conversation. Having some type that has a typed error channel gives this.mitch
04/03/2023, 6:49 AMResult<E, A>
I guess I need to give proper kudos to @simon.vergauwen and arrow maintainers for introducing the concept of builders for options/eithers/result. As a user I can’t emphasize on how inspiring and mindblowing this is. I have to be honest, this syntax is amazing, it allows writing flat program with very minimal nesting. This program looks like it’s using guard clauses through early returns, but actually under the hood it can use coroutines cancellations. A “short-circuit” event then not does it return early, it also cancels all the coroutines in the scope. I really think that something like this needs to be considered by the Kotlin language.
suspend fun getManagedUser(id: UserId): Result<Fail, ManagedUser> = result {
val user = getUser(id).toResult { Fail.UserNotFound }.valueOrFail()
val managedStatus = getManagedStatus(user)
if (!managedStatus.isManaged) {
fail(Fail.UserIsNotManaged)
}
// ... insert more rules here
ManagedUser(user, managedStatus)
}
simon.vergauwen
04/03/2023, 7:33 AMResult<T, E1 | E2 | E3 ...>This is already possible without
Union
types if you combine Arrow's DSL with context receivers.
context(Raise<E1>, Raise<E2>, Raise<E3>
fun myExample(): T = ...
Furthermore, either { }
is just an implementation of the Raise
DSL and Arrow allows you to provide custom DSL for all your data types.
Taking one of your examples @ephemient.
NetworkResult<out T>
|- Success<out T>(value: T) : NetworkResult<T>
\- Error : NetworkResult<Nothing>
|- AuthenticationError : Error
|- ClientError : Error
|- TransportError : Error
\- ServerError : Error
inline fun <T> networkResult(
block: Raise<NetworkResult.Error>.() -> T
): NetworkResult<T> =
recover(
{ NetworkResult.Success(block()) }
) { it }
Or even with context receivers and no need for a customised DSL (or Success
wrapper).
context(Raise<NetworkResult>)
fun example(): Int = ...
We've done tremendous work in Arrow (for 2.0) to reduce the binary size, decrease the API size, documentation, etc such that is should not be troublesome to include it on the classpath.
I'll talk more about this in detail on KotlinConf, but I stand with my original points. I also agree with @ephemient, it's not a 1-fits-all solution. While in Arrow with Kotlin we try to serve all use-cases for typed error handling, and FP inspired solution. Even within the FP community there is a lot of debate how things should be done.
The the DSL approach Arrow takes vs monads with transformers, algebraic effect systems, final tagless, etc.mitch
04/03/2023, 7:46 AMsimon.vergauwen
04/03/2023, 7:49 AMmitch
04/03/2023, 8:04 AMmono { }
or flow { }
builders. So I do believe the way errors are handled with types and values instead of exception to be idiomatic.
In which case.. there is a definite gap in the language about a missing Result<E, A>
type. Yes Arrow did fill that gap, but I think this might be because it’s a systemic problem.. I’d like to understand better the reason why isn’t this type available in the kotlin library. Surely there is a reason behind it and it has probably being considered as I do find articles and comments online and even in the blog itself that asks for this. If this type is introduced to the standard library or as part of opt-in package in kotlinx.typesafe. That would solidify this and alleviate all the ambiguity on the best practice on error handling and how that should be done in Kotlin.Joffrey
04/03/2023, 12:44 PMI believe there are other production applications like ours for which exceptions are too risky to useExceptions are present on the JVM, and even if you never throw any, you always have to remember that anything could throw an exception anywhere. So there is no (sane) way to completely get rid of exceptions. In general people setup exception handling at a high level (usually in the main framework they're using) to deal with unhandled exceptions, and avoid app death where necessary. Of course this doesn't mean we should use exceptions everywhere for error handling. As @ephemient pointed out, you may represent business-level errors with sealed classes (or nullability for the simplest cases). From what I have gathered, Roman's blog post is describing the idiomatic way of handling errors in Kotlin at most levels. I recommend also reading the initial KEEP for the design of
kotlin.Result
, which describes the intended style of Kotlin as well.Joffrey
04/03/2023, 12:51 PMmitch
04/03/2023, 9:40 PMsuspend fun bakeCake(...): Outcome<Fail, Cake> =
outcome {
val butter: Butter = buyButter(...).valueOrFail()
val carrot: Carrot? = buyCarrot(...).valueOrFail()
val sugar: Sugar = buySugar(...).valueOrFail()
val flour: Flour = buyFlour(...).valueOrFail()
carrot?.let { carrot ->
ensure (carrot.isFresh()) {
Fail.StaleCarrot
}
}
bakeCarrotCake(recipe, carrot, sugar, butter, flour).valueOrFail()
}
simon.vergauwen
04/04/2023, 9:33 AMfun Raise<String>.one(): Int = 1
vs fun one(): Either<String, Int> = 1.right()
.
There is no need for bind
or valueOrFail
as shown in the example from @mitch. This sadly currently occupies the receiver, but with context receivers that is resolved.
context(Raise<UserAlreadyExists>)
fun User.save(): Unit = ...
context(Raise<UpdateRejected>)
fun Remote.createProfile(
user: User,
update: Update
): Unit = ...
context(
Raise<UserAlreadyExists>,
Raise<UpdateRejected>
)
fun serviceMethod(...): Unit {
...
user.save()
...
remote.createProfile(user, ...)
...
}
This supports all patterns of railway programming, still supports the style of coding that Mitchell shared above, and is more in line with more modern functional languages are evolving towards. It completely eliminates the need for flatMap
or more complex patterns such as monad transformers, traverse, etc.
It's also future proof with other potential upcoming features, such as union types:
context(
Raise<UserAlreadyExists | UpdateRejected>
)
fun serviceMethod(...): Unit
As I mentioned, IMO nothing is a one-fits-all solution, that is something I definitely learned as having worked half my career as a fanatic OOP Java developer. I guess that also seems to be the conclusion of this discussion.
I think both styles fit well in Kotlin, and I am proud of the improvements we made in Arrow.mitch
04/05/2023, 11:33 AM.bind()
are no longer needed and developers will get help by the compiler to recover
error boundaries and translate between them.
context(Raise<UpdateUserFailure>) // this is a different error boundary
suspend fun updateUser(...): User = ...
context(Raise<ProvisioningFailure>)
suspend fun checkManagedStatus(...): Unit = ensure(user.isManaged) {
ProvisioningFailure.NotManaged
}
context(Raise<ProvisioningFailure>)
suspend fun checkProvisioningPolicy(...): Unit = ...
context(Raise<ProvisioningFailure>, Logging)
suspend fun provision(): User {
checkManagedStatus(...)
checkProvisioningPolicy(...)
val updatedUser = recover({ updateUser(...) }) {
when (it) {
is UserNotFound -> raise(ProvisioningFailure.InvalidUser)
is UpdateRejected -> raise(ProvisioningFailure.InvalidUpdate)
}
}
log { info("user provisioning successful") }
updatedUser
}
Gosh this is mindblowing! thank you @simon.vergauwenmitch
04/05/2023, 11:34 AMsimon.vergauwen
04/05/2023, 11:35 AMelizarov
04/05/2023, 12:06 PMcontext(Raise<UserAlreadyExists>)
is a kind of coeffect — a “capability to raise an error” that the function requires from its context. Coeffects are totally equivalent to effects, that are traditionally used in functional languages to model exceptions and things like this. However, coeffects (contexts) are more ergonomic and fit way better into Kotlin programming style.