This is more of a missing optimisation that it is ...

# language-proposals

[Youssef Shoaib [MOD]]

This is more of a missing optimisation that it is a feature per se, but it's still quite useful. Currently when you have a very simplistic usecase like this:

inline fun <T> identity(block: () -> T): () -> T = block
inline fun <T> perform(block: () -> T): T = block()
fun main() {
    println(perform(identity { "hello" })
}

A useless lambda object gets created due to the compiler believing that because the lambda is returned by identity then it must exist as an actual object. This prevents certain wrapping tricks for lambdas that can provide interesting behaviours without overhead (think for example a lambda being passed into multiple functions one by one that affect whether it executes or not and even does suspend calls in between and ultimately gets inlined into the call site directly with all that new behaviour being abstracted away from the user). I propose that, whenever you have a lambda being returned by an inline function, and all of the subsequent usages of that lambda are only passed as inline parameters, that the lambda gets entirely inlined through all of the inline functions that calls it and ultimately ends up at the call site with all of the wrapping code from the inline functions around it, just as if you used that exact lambda but rewrote the functions to pass it down instead. Effectively, the above usecase would get compiled down into roughly the equivalent of this:

fun main () {
    println("hello")
}

I'd love to know if there's anything currently preventing this from working as expected, because it seems to me that the current inline mechanism should be enough. Yes this will probably mess up debugging possibly, but that could be circumvented just as if you would debug an inline function today.
This is basically just inversing the call stack in a way of how you would do this normally and allows for better decomposition. For example, the usecase above can currently be rewritten to this:

inline fun <T> identity(block: () -> T): T = block()
inline fun <T> perform(block: () -> T): T = identity(block)
fun main() {
    println(perform { "hello" })
}

and if you imagine this with actual logic surrounding the lambdas you can basically recreate any example of that form, but it ruins composability and requires the lambda call to always be at the very very bottom of the call hierarchy instead of allowing for more complex lambda-returning functions with no overhead.
For another example, consider this maybe:

val shouldPerform = true
val shouldRunSpecial = true 
inline fun <T> specialLogic(block: () -> T): () -> T? = if(shouldRunSpecial) block else { null } // can also be converted to one big lambda instead that does the check on shouldRunSpecial when it's actually called like this: = { if(shouldRunSpecial) block() else null }. Those 2 styles should be roughly equal in the end since all the lambdas get inlined
inline fun <T> performOrThrow(block: () -> T): T = if(shouldPerform) block() else TODO()
fun main() {
    println(perform(specialLogic { "hello" })
}
// Turns into this:
fun main() {
    println(
        if(shouldPerform) {
            if (shouldRunSpecial) "hello" 
            else null 
        } 
        else TODO())
}

This is currently what this code runs like logically today, but the only difference is that it'll be all inlined without the need for any lambda objects. Keep in mind that the object itself will need to materialise if it gets used in a non-inline context (i.e. if you pass it to a regular function or query its class for example). This could possibly be extended to fun interfaces themselves being inlined if you use a lambda to call them and then pass them to an inline function, which can have a lot of great impacts for some functional abstractions since it allows you to, for example, have a reference to a fun interface Functor that gets passed around to only inline functions and in the actual compiled code completely disappears since it isn't needed in any non-inlined contexts. I'm sure the Arrow team and other similar functional proponents would be interested in this because it decreases the cost of those functional abstractions even more (Note: if this can result in some backwards-compatibility issues, then this can instead be a new feature under the name inline fun interface that works similarly to inline classes but utilises the fact that lambdas that are used in only inline contexts don't ever need to be actual objects and can instead just be copied and inlined everywhere at compile time).
I do realise that this is kind of multiple proposals in one but these 2 features are very connected and kinda go hand in hand because they require similar compile-time machinery, however I personally think that at the very least the first one should be implemented first if it isn't possible to implement both at the moment because then libraries like Arrow can temporarily sacrifice some of the type safety that fun interface brings in lieu of the optimisation of regular lambdas being inlined when returned by a function. If the whole inline fun interface idea just sounds too compilcated, then instead we could possibly have `inline class`es with lambdas for their backing field that also receive the same optimisations in this proposal of inlining lambdas along with type safety of course. Personally however I think that inline fun interface s make more sense because they mirror how inline functions look like. Any thoughts or criticisms about this are greatly appreciated😄!

Again I'd love to know if I made an oversight here and if this could even be impossible, so don't hesitate to point out any flaws in my reasonings.

[Youssef Shoaib [MOD]]

Regarding the Arrow thing that I mentioned, currently AFAIK Arrow is using `fun interface`s to allow for easily moving around behaviours like Foldable and stuff like that (I might be remembering incorrectly btw but I am sure that they're used somewhere in the current snapshot version of the library) so this basically allows for using that abstraction with no overhead if the value of these fun interfaces is known at compile time and both the functions that return them and the functions that consume them are inline

[Youssef Shoaib [MOD]]

This optimisation also allows for a bit more freedom in DSLs. I have an example for this but it's a pretty silly use case (I was implementing a DSL that looks like if expressions just for fun using inline classes and inline functions everywhere and to implement syntax like If (condition) { code } ElseIf (condition) { code } and for the ElseIf part to achieve that exact infix syntax you need to have an extension invoke on Boolean that wraps the lambda after it in an actual if and returns that wrapped lambda and then have the ElseIf as an infix fun that accepts the wrapped lambda. It's kinda hard to explain but it's because of some operator priority things with infix funs lol, but basically that was to ensure that the ElseIf branch will never run if the first If succeeded. The alternative to this that we can do today is having ElseIf as a normal extension function on the return type of If that accepts 2 paramters: the boolean and the block to run if the first If failed and the condition of this ElseIf succeeded. Again this was all a very stupid example but there are most likely some legitimate usecases for this kind of optimisation out in the wild for DSLs and other abstractions).

[Youssef Shoaib [MOD]]

I also have a gut feeling (with no real examples right now) that this feature could bring more power to decorators possibly to allow for even more interesting and complex behaviours that feel idiomatic to Kotlin. Keep in mind that this proposal doesn't really add any new features per se and that it's just more about allowing the current possibilities of behaviours to be more optimised so that they can be used in performance-critical code.

[Youssef Shoaib [MOD]]

I just had a eureka moment that this optimisation can be used to create struct-like value types with zero overhead that ultimately get inlined into the call site. For example, here's a very quick demonstration that I threw together to create a zero cost pair (that is, assuming that this optimisation gets implemented and that you're allowed to keep the returned lambda in a val that only gets passed to inline functions, which should indeed be possible with this proposal) Try it out here :

// Using a dirty trick to ensure that the type information about F and S gets passed with the lambda.
// Alternatively, one can just use an inline fun interface or an inline class with a lambda val or whatever
// the accepted part of this proposal would be. I cobbled this together using a plain old lambda just to 
// demonstrate how cruicial the core optimisation itself is and that an inline fun interface or an inline
// class with a lambda would just ensure type safety over this so that users don't misuse it.
typealias ZeroCostPair<F, S> = (PairCall, F, S) -> Any?

// Enum to represent, well, which call the pair received. One can instead just use an Int or a Boolean even
// if performance is needed, but regardless of what you use if this proposal is implemented then the lambda 
// will be inlined which means that any smart shrinker should optimise away the resulting "First == First"
// and "Second == Second" calls, or the JIT can even optimise them then cuz they're a constant expression.
enum class PairCall {
    First,
    Second
}

// Mimicking a constructor for the type.
inline fun <F, S> ZeroCostPair(first: F, second: S): ZeroCostPair<F, S> = { call, _, _ -> // Those parameters are solely a trick to carry type information, and so they should be ignored
    when (call) {
        PairCall.First -> first
        PairCall.Second -> second
    }
}

// Again, the parameters are useless, so just pass in null for them since during runtime the JVM won't actually know what
// F and S are since they get erased.
// We can safely cast the result of invoking the function as F or S because we know that ZeroCostPairs created using 
// the factory function always follow the pattern of returning an F if PairCall.First is passed in and likewise for S.
// However, if stricter type safety is needed (which it is if you're building a library), then inline classes with lambdas should
// be implemented so that one can make the constructor internal and then have a factory function that takes in F and S values
inline val <F, S> ZeroCostPair<F, S>.first get() = invoke(PairCall.First, null as F, null as S) as F 
inline val <F, S> ZeroCostPair<F, S>.second get() = invoke(PairCall.Second, null as F, null as S) as S

fun main(){
    val values = computeValues(37, "Hello")
    println(values.second)
    println("The answer to life, the universe, and everything = " + values.first)
    
    computeValues(37, "Hello") { first, second ->
        println(second)
        println("The answer to life, the universe, and everything = " + first)
    }
}


inline fun computeValues(number: Int, text: String): ZeroCostPair<Int, String> {
    return ZeroCostPair(number + 5, text + " World!")
}

inline fun computeValues(number: Int, text: String, block: (Int, String) -> Unit){
    block(number + 5, text + " World!")
}

Notice that at the bottom I included an example of how this can be done today with zero cost but by compromising the call site with ugly callback-style code. IMHO the ZeroCostPair callsite looks much much much nicer and with this proposal would have no performance overhead just like the callback-style callsite but with much cleaner semantics. This could possibly be used to implement some form of @AutoInline value class kind of thing where the compiler generates all this boilerplate including the PairCalls enum and stuff like that. This class would work in the same exact spirit that lambdas work right now: it will only get inlined if the function using it is inline and otherwise it will be boxed (but in this case it'll be boxed just as a normal POJO (/KOJO) instead of being boxed as a lambda). It will basically take on the semantics of inlining lambdas including the optimisations introduced in this proposal, and for "mutating" it the lambda will just be prefixed with an if(call == ChangedCallName) return newValue (if this isn't clear enough I can write a function that does this).

[Youssef Shoaib [MOD]]

I recognise that I keep going on tangents about other possible proposals from this so I want to reemphasise the point that the vital part of this proposal is just the plain lambda optimisation, everything else is syntactic sugar or type safety on top. For example, that last idea of @AutoInline doesn't require the inline fun interfaces or the inline classes part of this proposal as is very clear from the example. The idea of @AutoInline itself doesn't even have to be a language feature; it can be implemented as an annotation processor that generates that boilerplate for you; it can be implemented as an Arrow Meta compiler plugin that rewrites the class to use a lambda; hell, it can even just be manually written out for those performance-critical cases where you need something like this and then you can carry on with your life since it doesn't require any more work than is shown in the example above (which, mind you, is exactly just (7 + 3 * the number of properties) lines of boilerplate code (and that number would be the same or very very close if you used an inline class if that part of the proposal was implemented) which, if I'm being honest, isn't really the end of the world if you're just making a few `Rect`and Point types for a high-performance graphics library. The important thing to take here is that core lambda-returning optimisation opens up the doors for many many interesting concepts that don't need to compromise between ergonomics, type safety, and performance since lambdas are highly composable and can be to represent a myriad of situations.

[Ilmir Usmanov [JB]]

Can you create a youtrack issue? This is definitely something we can support, since it does not have backward compatibility issues, and it does not introduce new syntax.

[Youssef Shoaib [MOD]]

Perfect, I'll do so ASAP. I posted here just to gather my ideas and to get feedback on whether there's something that could be blocking this. Should I also include the inline class lambda/inline fun interfaces optimisation in the proposal? Or should I only focus on just the core part for now and then suggest that later on after the core optimisation is implemented?

[Ilmir Usmanov [JB]]

You can post inline fun interface part as well, noting, that the optimization can be extended to support them.

[Youssef Shoaib [MOD]]

Perfect thank you so much I'll create the issue ASAP with a clear section for "Future Possibilities" that include everything else other than the core lambda return thing

[Youssef Shoaib [MOD]]

Just created a Youtrack issue under the name KT-44530. Go vote!