ianbrandt
03/14/2024, 3:50 AMfun interface Converter<in I, out O> {
fun convert(input: I): O
}
My system does a lot of domain mapping, so this interface has 100+ implementations.
Some Converter
implementations are trivial, and are declared to return non-nullable or nullable types. Others can result in logical errors, and so I've been declaring them to return an `Either`:
object NonTrivialConverter : Converter<Int, Either<Error, Int> {
fun convert(input: Int): Either<Error, Int> = either {
ensure(input <= LIMIT) { Error() }
input * SCALING_FACTOR
}
}
I'm currently using the `either`/`bind()` approach to compose logic that calls such functions, but it's not ideal:
• The extra bind()
calls increase verbosity.
• They can be omitted without producing a compile-time error, which is contrary to the shift-left value proposition of typed errors.
◦ I'm aware of the Detekt rule, but additional build and IDE plugins add overhead and risk compared to native language and IDE support.
• The ergonomics of lambdas as function bodies isn't great.
◦ For one, there's the jarring mix of value-of-last-expression returns for lambda function bodies vs. explicit `return`s for regular function bodies.
◦ Also, at least in the latest version of IntelliJ, return type issues in lambdas result in the entire lambda body being underlined in red, unlike the precise error highlighting in regular function bodies.
I've experimented with the Raise
DSL. That wonderfully addresses all of the above issues, with one problem. If I understand correctly, to use the Raise DSL with my generic converters I need to update my functional interface to have a Raise
receiver or context parameter, e.g.:
context(Raise<Error>)
fun interface Converter<in I, out O> {
fun convert(input: I): O
}
The issue is that I've now colored my functional interface, not unlike if I'd made it `suspend`ing. I can't migrate my codebase to it incrementally, and all implementers must be called from a Raise
context regardless of whether they actually need it.
Have you all encountered this problem, and is there a solution for it besides sticking with the `either`/`bind()` approach?
If that latter is the only option, I'd start providing feedback on the code coloring downside of the current context parameters proposal, or advocating for the revisiting of for-comprehensions in Kotlin (KT-18861) as a native solution to `either`/`bind()`.Alejandro Serrano.Mena
03/14/2024, 8:29 AMfun interface Converter<in I, out O> {
fun Raise<Error>.convert(input: I): O
}
simon.vergauwen
03/14/2024, 8:32 AMobject Error
fun <I, O> Converter(
block: context(Raise<Error>) (input: I) -> O
): Converter<I, O> =
object : Converter<I, O> {
override fun Raise<Error>.convert(input: I): O =
block(this@convert, input)
}
@OptIn(ExperimentalTypeInference::class)
@OverloadResolutionByLambdaReturnType
fun <I, O> Converter(
block: (input: I) -> Either<Error, O>
): Converter<I, O> =
object : Converter<I, O> {
override fun Raise<Error>.convert(input: I): O =
block(input).bind()
}
fun interface Converter<in I, out O> {
fun Raise<Error>.convert(input: I): O
fun convert(input: I): Either<Error, O> =
either { convert(input) }
}
I don't think having for-comprehensions make any sense. You can discuss if Raise
colors the function or not, it's the same as Either
. It also colors your functions in a way.Alejandro Serrano.Mena
03/14/2024, 8:33 AMfun interface FailingConverter<in I, out O> {
fun Raise<Error>.convert(input: I): O
}
fun interface PureConverter<in I, out O>: FailingConverter<I, O> {
fun convertNoFailure(input: I): O
override fun Raise<Error>.convert(input: I): O = convertNoFailure(input)
}
for a similar example, this is how `suspend`/non-suspend
is dealt with in Arrow Collectors https://github.com/arrow-kt/arrow/blob/arrow-2/arrow-libs/fx/arrow-collectors/src/commonMain/kotlin/arrow/collectors/Collector.kt#L160simon.vergauwen
03/14/2024, 8:34 AMAlejandro Serrano.Mena
03/14/2024, 8:38 AMFailingConverter
and Converter
as the relation between Converter<A, Either<E, B>>
and Converter<A, B>
in your original example. You are marking whether they fail or not somehow. However, the two-interfaces approach give you the syntax you are looking for: no bind
object NonTrivialConverter : FailingConverter<Int, Int> {
fun Raise<Error>.convert(input: Int): Int {
ensure(input <= LIMIT) { Error() }
return input * SCALING_FACTOR
}
}
simon.vergauwen
03/14/2024, 8:41 AMConverter<String, Either<Error, Int>
into ContextConverter<Raise<Error>, String, Int>
and it should not really change at the call site if you're using the DSL.Alejandro Serrano.Mena
03/14/2024, 8:42 AMContext
an extension receiver (fun Context.convert(input: I): O
) instead of a contextual onesimon.vergauwen
03/14/2024, 8:42 AMsimon.vergauwen
03/14/2024, 8:43 AMconverter.convert
and you replace the receiver with the extension ..simon.vergauwen
03/14/2024, 8:43 AMsimon.vergauwen
03/14/2024, 8:44 AMcontext(Nothing)
is like having no context, so you can generically "erase" the fact that you have a context receiverAlejandro Serrano.Mena
03/14/2024, 8:44 AMoverride
one with the other; this is the reason I have convert
vs. convertPure
in my example)simon.vergauwen
03/14/2024, 8:46 AMraulraja
03/14/2024, 8:47 AMraulraja
03/14/2024, 8:47 AMsimon.vergauwen
03/14/2024, 8:49 AMcontext
on type/constructor going to stay @Alejandro Serrano.Mena? I thought it was going to be removed on interfaces, and classes.simon.vergauwen
03/14/2024, 8:50 AMAlejandro Serrano.Mena
03/14/2024, 8:52 AMcontext
on the class doesn't do what you expect
context(Raise<E>)
fun interface Converter<out E, in I, out O> {
operator fun invoke(input: I): O
}
the reason is that (in the current prototype) the context is taking from the place you call the constructor; instead of the place where you call the function. This is why the faithful encoding is instead
fun interface Converter<out E, in I, out O> {
operator fun Raise<E>.invoke(input: I): O
// or, if you prefer
context(Raise<E>) operator fun invoke(input: I): O
}
and indeed, the first block is not going to compile anymore once context parameters get implemented (one of the main reasons is exactly the confusion between when the context is "captured")simon.vergauwen
03/14/2024, 8:56 AMval conv = Convertor<Error, String, Int>
either {
conv.convert(...) // expected convert to be called on Raise.
with(conv) { convert(..) } // this works.
}
raulraja
03/14/2024, 8:57 AM@raulraja does your compile, and run?just prototyped it quickly on the editor but haven't tested it. We use this class level context on other places though to avoid
with(instance) { callInstanceMethod }
so I'm guessing all that is going to break once context parameters are inAlejandro Serrano.Mena
03/14/2024, 8:58 AMRaise<E>
in the context (but in invoke, not on top of the class)
fun interface Converterout E, in I, out O {
context(RaiseE) operator fun invoke(input: I): O
}simon.vergauwen
03/14/2024, 9:04 AMianbrandt
03/14/2024, 3:12 PMianbrandt
03/16/2024, 12:38 AMianbrandt
03/16/2024, 1:19 AMAlejandro Serrano.Mena
03/16/2024, 8:15 AMfun interface ContextConverter<C, in I, out O> {
context(C)
fun ctxConvert(input: I): O
}
fun interface Converter<in I, out O> : ContextConverter<Unit, I, O> {
fun convert(input: I): O
override context(Unit) fun ctxConvert(input: I): O = convert(input)
}
Alejandro Serrano.Mena
03/16/2024, 8:17 AMcontext(Nothing)
is not a good option here, because once the implementation is finished, it won't work (you cannot create values of type Nothing
); so Unit
is a best choiceraulraja
03/16/2024, 10:54 AMContext(Unit)
have the same semantics as no context
and get automatically erased or synthesized by the compiler as needed? The compiler has guarantees that it can provide the only singleton instance value for Unit as it does in functions with no explicit return.
If it does not get automatically synthesized how do you call functions with context(Unit)
?
Is it like ?
Unit.run { convert(i) }
Since context(Unit)
is the same as the raw environment I would expect to be able to call that function without a context.
Similarly I expected this should have been a valid override:
fun interface Converter<in I, out O> : ContextConverter<Unit, I, O> {
override fun convert(input: I): O = convert(input)
}
or if we want to leave this explicit.
fun interface Converter<in I, out O> : ContextConverter<Unit, I, O> {
override context(Unit) fun convert(input: I): O = convert(input)
}
But at the call site I don't see the point of providing my own value of Unit or carrying the context(Unit)
because Unit
can only have one inhabitant and the compiler knows which one it is.ianbrandt
03/16/2024, 4:10 PMfun interface ContextConverter<C, in I, out O> {
context(C)
fun ctxConvert(input: I): O
}
fun interface Converter<in I, out O> : ContextConverter<Unit, I, O> {
fun convert(input: I): O
context(Unit)
override fun ctxConvert(input: I): O = convert(input)
}
I can't say I love the extra method subjectively (for example, it wouldn't work with overloading invoke
). Regardless, I ran into a Kotlin/Spring reflection issue with one of my unmodified (i.e. not context) converters:
Caused by: kotlin.reflect.jvm.internal.KotlinReflectionInternalError: Inconsistent number of parameters in the descriptor and Java reflection object: 3 != 2
Calling: context(kotlin.Unit) public open fun ctxConvert(input: my.A?): my.B defined in my.PlainAToBConverter[DeserializedSimpleFunctionDescriptor@79a201cf]
Parameter types: [class mt.PlainAToBConverter, class kotlin.Unit, class java.lang.Object])
Default: false
at kotlin.reflect.jvm.internal.calls.ValueClassAwareCallerKt.checkParametersSize(ValueClassAwareCaller.kt:263)
at kotlin.reflect.jvm.internal.calls.ValueClassAwareCallerKt.access$checkParametersSize(ValueClassAwareCaller.kt:1)
at kotlin.reflect.jvm.internal.calls.ValueClassAwareCaller.<init>(ValueClassAwareCaller.kt:116)
at kotlin.reflect.jvm.internal.calls.ValueClassAwareCallerKt.createValueClassAwareCallerIfNeeded(ValueClassAwareCaller.kt:304)
at kotlin.reflect.jvm.internal.calls.ValueClassAwareCallerKt.createValueClassAwareCallerIfNeeded$default(ValueClassAwareCaller.kt:293)
at kotlin.reflect.jvm.internal.KFunctionImpl$caller$2.invoke(KFunctionImpl.kt:99)
at kotlin.reflect.jvm.internal.KFunctionImpl$caller$2.invoke(KFunctionImpl.kt:64)
at kotlin.SafePublicationLazyImpl.getValue(LazyJVM.kt:107)
at kotlin.reflect.jvm.internal.KFunctionImpl.getCaller(KFunctionImpl.kt:64)
at kotlin.reflect.jvm.ReflectJvmMapping.getJavaMethod(ReflectJvmMapping.kt:64)
at kotlin.reflect.jvm.ReflectJvmMapping.findKFunction(ReflectJvmMapping.kt:155)
at kotlin.reflect.jvm.ReflectJvmMapping.getKotlinFunction(ReflectJvmMapping.kt:144)
at org.springframework.core.MethodParameter$KotlinDelegate.getGenericReturnType(MethodParameter.java:916)
at org.springframework.core.MethodParameter$KotlinDelegate.access$000(MethodParameter.java:867)
at org.springframework.core.MethodParameter.getGenericParameterType(MethodParameter.java:510)
at org.springframework.core.SerializableTypeWrapper$MethodParameterTypeProvider.getType(SerializableTypeWrapper.java:291)
at org.springframework.core.SerializableTypeWrapper.forTypeProvider(SerializableTypeWrapper.java:107)
at org.springframework.core.ResolvableType.forType(ResolvableType.java:1420)
at org.springframework.core.ResolvableType.forMethodParameter(ResolvableType.java:1341)
at org.springframework.core.ResolvableType.forMethodParameter(ResolvableType.java:1323)
at org.springframework.core.ResolvableType.forMethodParameter(ResolvableType.java:1290)
at org.springframework.core.ResolvableType.forMethodReturnType(ResolvableType.java:1250)
at org.springframework.core.GenericTypeResolver.resolveReturnType(GenericTypeResolver.java:80)
at org.springframework.beans.GenericTypeAwarePropertyDescriptor.<init>(GenericTypeAwarePropertyDescriptor.java:110)
at org.springframework.beans.CachedIntrospectionResults.buildGenericTypeAwarePropertyDescriptor(CachedIntrospectionResults.java:431)
at org.springframework.beans.CachedIntrospectionResults.<init>(CachedIntrospectionResults.java:308)
at org.springframework.beans.CachedIntrospectionResults.forClass(CachedIntrospectionResults.java:181)
at org.springframework.beans.BeanWrapperImpl.getCachedIntrospectionResults(BeanWrapperImpl.java:174)
at org.springframework.beans.BeanWrapperImpl.getPropertyDescriptors(BeanWrapperImpl.java:248)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.filterPropertyDescriptorsForDependencyCheck(AbstractAutowireCapableBeanFactory.java:1594)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.filterPropertyDescriptorsForDependencyCheck(AbstractAutowireCapableBeanFactory.java:1574)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1434)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
This is with Java 11 and Spring 5. To try it with the latest Spring 6, I'll either need to create a reproducer project, or upgrade my main project. Either (no pun intended) is going to take a bit of time.ianbrandt
03/16/2024, 4:19 PMConverter
and ContextConverter
inheritance relationship around:
fun interface Converter<in I, out O> {
fun convert(input: I): O
}
fun interface ContextConverter<C, in I, out O> : Converter<I, O> {
context(C)
override fun convert(input: I): O
}
That resulted in a different internal error: https://youtrack.jetbrains.com/issue/KT-66658/Backend-Internal-error-Exception-during-IR-lowering#focus=Comments-27-9492449.0-0simon.vergauwen
03/16/2024, 5:07 PMfun interface Converter<in I, out O> : ContextConverter<Unit, I, O> {
You should be using Nothing
here instead of Unit
, and then you can remove the context(Nothing)
. That way you don't need it at all, and there is also no need for an extra method. In case C
is Raise<E>
then convert
will only be callable from within either { }
or any other code with Raise
available.
fun interface ContextConverter<C, in I, out O> {
context(C)
fun convert(input: I): O
}
fun interface Converter<in I, out O> : ContextConverter<Nothing, I, O> {
fun convert(input: I): O
}
Can you try this?ianbrandt
03/16/2024, 5:16 PMsimon.vergauwen
03/16/2024, 5:17 PMianbrandt
03/16/2024, 5:39 PMConverter { ... }
is being done. The other in a test that's satisfying a Converter
constructor dependency with converterArg = { mockk() }
. These seem like edge cases to me, so I'm not too shocked by the errors.simon.vergauwen
03/16/2024, 5:40 PMsimon.vergauwen
03/16/2024, 5:41 PMsimon.vergauwen
03/16/2024, 5:42 PMmocck
caseianbrandt
03/16/2024, 5:43 PMNothing
vs. Unit
above: https://kotlinlang.slack.com/archives/C5UPMM0A0/p1710576940925629?thread_ts=1710388229.201949&cid=C5UPMM0A0.
His comments sort of dissuaded me from pursuing this approach, since it won't do me much good if it's not going to work in the release version of context parameters.simon.vergauwen
03/16/2024, 5:44 PMianbrandt
03/19/2024, 3:49 PMContextConverter : Converter
doesn't really make sense. It violates the Liskov Substitution Principle. A ContextConverter
can't be substituted when clients expect a Converter
, as those clients wouldn't provide the needed context.
From that same perspective, it does seem like a subtype should be able to somehow override a context parameter to erase or widen it. Such a subtype would be substitutable for its supertype. If a subtype doesn't need a Raise
, Logger
, etc. in context, or could work with a supertype of the context parameter defined by its supertype, one would think it should be able to be declared as such to enable incremental migration use cases like mine here. This would seem to be in conflict with, "In the JVM a function with context parameters is represented as a function with additional parameters." Subtraction or widening of function parameters by a subtype would be an overload in Kotlin and Java, not an override.
This is a bit hand-wavy, but if context parameters were say implemented as open class properties, then they could be contravariant. Is the currently chosen approach to context parameter implementation on the JVM constraining them to be invariant, whereas they would otherwise have better conceptual integrity with inheritance if they were contravariant?
If context parameters were to support variance, it would seem overriding context(r: Raise<Error>)
with context(r: Nothing)
as above would be incorrect. That would be a covariant override (viable for return types, but not method arguments or class properties), as Nothing
is the subtype of all types. Overriding with context(r: Any?)
would be a contravariant override, with Any?
being the supertype of all types. You'd be saying that r
can be anything that has an equals
, hashCode
and toString
(e.g. Unit
, etc.), or null
(i.e. not be in context).
I gave this a try, and it sort of worked, with some caveats noted inline:
import arrow.core.raise.Raise
import arrow.core.raise.either
data class ConverterError(
val e: Exception,
)
fun interface ContextConverter<C, in I, out O> {
context(C)
fun convert(input: I): O
}
// A `Converter` is `ContextConverter` with anything in context
fun interface Converter<in I, out O> : ContextConverter<Any?, I, O>
object StringToIntConverter :
ContextConverter<Raise<ConverterError>, String, Int> {
context(Raise<ConverterError>)
override fun convert(input: String): Int =
try {
input.toInt()
} catch (e: NumberFormatException) {
raise(ConverterError(e))
}
}
object IntToStringConverter : Converter<Int, String> {
context(Any?)
override fun convert(input: Int): String = input.toString()
}
// We can generalize over the `ContextConverter` supertype
context(C)
fun <C, I, O> withContextConverter(
input: I,
converter: ContextConverter<C, I, O>,
): O = converter.convert(input)
fun main() {
val stringToIntSuccess = either {
withContextConverter("1", StringToIntConverter)
}
println(
"withContextConverter(\"1\", StringToIntConverter): $stringToIntSuccess"
)
val stringToIntFailure = either {
withContextConverter("Nope", StringToIntConverter)
}
println(
"withContextConverter(\"Nope\", StringToIntConverter): $stringToIntFailure"
)
// "Not enough information to infer type variable Error"...
val intToStringSuccess = either<ConverterError, String> {
// The `Converter` subtype is substitutable
withContextConverter(1, IntToStringConverter)
}
println(
"withContextConverter(1, IntToStringConverter): $intToStringSuccess"
)
// Direct call to the `Converter` subtype seems to require a `with` for
// `null` or any object (e.g `Unit`). Otherwise, "No required context
// receiver found".
with(null) {
val directIntToStringSuccess = IntToStringConverter.convert(1)
println("direct IntToStringConverter(1): $directIntToStringSuccess")
}
}
ianbrandt
03/19/2024, 9:31 PMwith
above, this approach doesn't really address my incremental migration use case. All Converter
clients would need to be updated at once when extracting the ContextConverter
supertype.simon.vergauwen
03/20/2024, 6:47 AMI'm probably just catching up to you all here, but thinking on this, I realized my attempt that flips the inheritance relationship toIs that truly the case? Because if we can erase context, thendoesn't really make sense. It violates the Liskov Substitution Principle. AContextConverter : Converter
can't be substituted when clients expect aContextConverter
, as those clients wouldn't provide the needed context.Converter
Converter
is a contextless Converter
which is exactly precise, and respects the Liskov substitution principle, since you also need to respect the generic of course. To take it further, Converter
doesn't need to be its own type typealias Converter = ContextConverter<Nothing, Input, Output>
so Liskov doesn't even come into play really.
This would seem to be in conflict with, "In the JVM a function with context parameters is represented as a function with additional parameters." Subtraction or widening of function parameters by a subtype would be an overload in Kotlin and Java, not an override.Well, for this to ever work Kotlin would need to ignore
context(Nothing)
since a parameter of Nothing
won't allow you to ever call this function. fun impossible(nothing: Nothing)
there is no way to call the impossible
function since you can never provide an instance of Nothing
. This is I guess why @Alejandro Serrano.Mena said it won't be added, which I think is a shame because I agree that generically erasing a context is a super power. I guess I'd need to add a official request for that on the KEEP if I'd want something like this to be taken into consideration.simon.vergauwen
03/20/2024, 6:51 AMGiven the requiredYes, theabove, this approach doesn't really address my incremental migration use case. Allwith
clients would need to be updated at once when extracting theConverter
supertypeContextConverter
with(null)
shouldn't be needed here. That's a shame, because if you put main
in a class it works 🙃
context(Any?)
fun example() = 1
class Test {
fun main() {
val x: Test = this
example() // Any? is satisfied by this: Test
}
}
Alejandro Serrano.Mena
03/20/2024, 8:24 AMcontext(Nothing)
situation is correct: having such a context means that you cannot call that function, since you can never create a Nothing
value
(fwiw, this has spawned a discussion with my colleagues about a proper way to "erase contexts". From a theoretical point of view context(Unit)
means a "useless context", since Unit
holds no information, but the fact that we also use it for side-effectful computations in Kotlin makes everything a bit harder)simon.vergauwen
03/20/2024, 8:32 AMFrom a theoretical point of viewThat makes perfect sense to me, also through my original understanding ofmeans a "useless context", sincecontext(Unit)
holds no information, but the fact that we also use it for side-effectful computations in Kotlin makes everything a bit harderUnit
Unit
and Nothing
as a set with 1 value and a set with no values. On the other hand, context(Nothing)
makes sense that no has to be provided. Introducing a special type like EmptyContext
would probably a bad idea, but I think so is requiring with(null)
or using Any?
which also kind-of makes sense in a variance way, but besides that it doesn't make sense to me.ianbrandt
03/21/2024, 7:59 PMcontext(Any?)
actually does make sense to me (noting that contravariance is in general not particularly intuitive). Functions with context parameters are in effect "consumers" of them. Even without generic type parameters, subtyping can be done more flexibly when consumed types are contravariant (and produced types are covariant). We give up that flexibility with non-generic method arguments as a tradeoff to enable overloading, but do we need to make that tradeoff for context parameters too?
Take this simpler example:
object Error
open class Gizmo {
context(Raise<Error>)
open fun doIt(): String =
"I can do it, but I might raise an error."
}
class SubGizmo : Gizmo() {
// This compiles as a contravariant override with Kotlin
// 1.9.23 and `kotlin.experimental.tryK2=false`.
// With `kotlin.experimental.tryK2=true` it fails stating,
// "'doIt' overrides nothing".
context(Raise<Error>?)
override fun doIt(): String =
"I can do it without raising an error, " +
"so the Raise context parameter can be null."
}
fun main() {
// The `with(null)` here shouldn't be needed.
with(null) {
println(SubGizmo().doIt())
}
}
"Erasing" a context parameter can be done by simply overriding it with a nullable supertype, be that e.g. Raise<Error>?
, or Any?
. Context parameters just need to be able to resolve to null when they're of nullable type and no match is found. Then there'd be no need for the with(null)
above.simon.vergauwen
03/22/2024, 12:30 PMContext parameters just need to be able to resolve to null when they're of nullable type and no match is found.Hmm, that seems abit dangerous but nullability in context receivers. Is that allowed @Alejandro Serrano.Mena? Does it make sense? A context being optional might be useful 🤔 but how to deal with mutiple nullable contexts?
context(Raise<Error>?, ResourceScope?)
does with(null) { }
cover both?Alejandro Serrano.Mena
03/22/2024, 12:59 PMwith(null) { ... }
or context(null) { ... }
. If you do so, that would work for any nullable context, indeedianbrandt
04/02/2024, 10:35 PM