Hi all.. I'd like to propose Kotlin nullable types...

# language-proposals

[mitch]

Hi all.. I'd like to propose Kotlin nullable types T? to adopt swift-style optionals that nests correctly e.g. T? is optional and T?? means optional of optional. That perhaps can be enabled with a compiler option (because of the possibility for breaking interop with Java). I just realized how misleading nullables can be when paired with generic codes due to the null auto-flattened... I believe this is known as the nested nullability problem in Kotlin (i.e. T? and T??????? are the same thing: null). I'm surprised not many online sources that talk about this problem...

Use-case:

to hopefully explain my pain.. a toy example with `List<T?>`:

val myList: List<T?> = ....

/** 
 * if the result is null, does it mean:
 * - the first element is null, or 
 * - [myList] is empty?
 */
val first: T? = myList.firstOrNull()
println(first) // null :( but why???

what I really need (thinking of swift here)

val myList: List<T?> = ....

// notice ?? - I can then know
val first: T?? = myList.firstOrNull()
println(first) // some(nil) list isn't empty, first value is nil

[jw]

What representation are you using to model this in bytecode, JS, and native?

[jw]

I actually think the current design, while not monadic, is the smarter choice since it harmonizes with the underlying memory better. And when you need an actual nestable optional you can choose to use one explicitly

[mitch]

I understand about the argument about memory footprint. However, this is a big, very painful footgun that really hurts when you were caught off-guard. Swift, in comparison solves this very elegantly in the language.. https://www.avanderlee.com/swift/optionals-in-swift-explained-5-things-you-should-know/#5-nested-optionals-is-a-thing

Representation: my current best proposal if it is a single ? then it'll do it like it is right now (nulls). However, in cases where engineers declare the need of nesting e.g. ?? the compiler would be smart enough to determine that needs to be desugared into a generic wrapper object akin to Optional<T> in respective target platform.

[mikhail.zarechenskiy]

Let me throw one more thing in here. Similar questions arise in every discussion of union types and their design: tagged vs untagged unions, T? = T | null, or T? = Some(T) | None . While Kotlin's type-system design of T? has good compromises, we would like to address issues with tagging unions, especially before adding more unions to Kotlin... For one more real example, you can take a look at our Sequence<T>.last implementation, it uses not the prettiest way to find an element and, what is more important, it's quite error-prone because it's easy to forget about this fact and write straightforward code:

// incorrect implementation
fun <T> Sequence<T>.last(predicate: (T) -> Boolean): T {
  var result: T? = null
  for (element in this) if (predicate(element)) result = element
  return result ?: throw NoSuchElementException("Not found")
}

[mitch]

Thank you @mikhail.zarechenskiy 💯 .. that captures the problem very well. I'm very glad that the Kotlin team is actively discussing this. T? = Some(T) | None would be so amazing. It'll solve all the nested nullability issues. 🙌

Is there a KEEP or youtrack ticket that I can follow? Or otherwise I'm happy to open one.

[mikhail.zarechenskiy]

JFTR: I highly doubt that representation of T? will be changed to Some(T) | None, we think of slightly other approaches. There is no KEEP at the moment, only this generic issue, feel free to add your case there: https://youtrack.jetbrains.com/issue/KT-13108/Denotable-union-and-intersection-types

[mitch]

Thanks @mikhail.zarechenskiy I understand. It may somehow has to be an encoding that allows maintaining backward compatibility otherwise a lot of codes that currently uses nulls may break...

I mentioned this idea above: perhaps during transitionary period if it is a single ? then it'll do what it does right now (nulls). However, in cases where engineers declare the need of nesting e.g. ?? - that'll be optional of optional, the compiler would be smart enough to determine that needs to be recursively desugared into the new approach (i.e. the later chosen equivalent of Some(T) | None )

With the new encoding, calling functions that has nested nullability effect, compiler and IDE can help show adequate warnings to e.g. type it with ?? or something like so. And this can be set to strict in the compiler settings.

What are the candidate approaches that the team is currently looking at if you have any insights?

I've left a comment about my use case.
https://youtrack.jetbrains.com/issue/KT-13108/Denotable-union-and-intersection-types#focus=Comments-27-8229776.0-0

[Youssef Shoaib [MOD]]

An important point to make: if we get union types, we could simply just do:

object NotFound
fun <T> Sequence<T>.last(predicate: (T) -> Boolean): T {
  var result: T | NotFound = NotFound
  for (element in this) if (predicate(element)) result = element
  return result.ifNotFound { throw NoSuchElementException("Not found") }
}

inline fun <A> (A|NotFound).ifNotFound(block: () -> A) = if (this is NotFound) block else this

In other words, we could have types that mean a bit more than just "can be null", and if NotFound is file private, this won't leak anywhere or cause any issues at all

[Youssef Shoaib [MOD]]

In fact, this seems to be the team's preference for what to do for https://youtrack.jetbrains.com/issue/KT-59047/Support-for-custom-null-objects

[mikhail.zarechenskiy]

It's somewhat similar to what <@U019AG342E5> wrote but the key point here is that I think we should make it possible to return some union type from the function, and this is where questions about type inference with its undecidability and memory-wise representation come in (as well as ergonomics in general). We're leaning a bit towards into material shape design (see this paper for inspiration). In other words, maybe we should have a separation for "regular" types and "error" types with their restriction for type inference and so on, but, again, it's too early to tell