I'm having a hard time figuring out how to use Arr...

# arrow

[leonhardt]

I'm having a hard time figuring out how to use Arrow Optics on nullable values. When the set and modify operations are used with nullable properties, the functions do not accept nullable values where I would expect them.

[leonhardt]

Here's an example:

import arrow.optics.copy
import arrow.optics.optics

@optics
data class Person(val name: String?) {
  companion object
}

fun main() {
  val person = Person(name = "Matt")
  person.copy {
    // this does not compile, not able to return null:
    Person.name.transform { if (it.isNullOrBlank()) null else it.trim() }

    // even this does not compile, not able to pass null:
    Person.name.set(null)
  }
}

[leonhardt]

Is this a known issue or limitation? Or am I approaching this the wrong way?

I tried to find information in the docs but the only nullable value in the example (the street number) is not mutated by the optics operations.

[Kev]

Would it cause less issues to have a person that doesn't have a nullable name, and by this, having 2 types of people; a Person and a NamedPerson?

[leonhardt]

For the patterns in our codebase I don't believe that would work well. We're using optics with data classes that represent deserialized request bodies made by users in a web interface. When a value is omitted, the user interface will naively send an empty string. If blank/empty values are allowed, we transform them in the server to null on the backend (or send an error back to the user when a blank/empty value is not allowed). And we have many fields on theses requests, which is why I don't believe your suggestion would work (many types would be required to represent all the combinations of present and not-present values). But thanks for the suggestion!

[leonhardt]

It seems the set/modify operations are built to only operate on nullable fields with non-null values. Any idea if there's an alternative that doesn't exclude null values like this? Or if there's some alternative that we've missed?

[Gemy]

hmm
I'm thinking using option instead of nullable?

[leonhardt]

Hello @simon.vergauwen and @Alejandro Serrano.Mena. I remember @raulraja mentioned you were to thank for the great DSL in Optics! I really appreciate the suggestions I've received in this thread but I'm not really looking for a fix to this particular bit of example code but rather to understand the intended way to use the Optics DSL with null values.

So a question for you two as the authors, are the operations in the Optics DSL intended to only be invoked in conjunction with non-null values? Or am I missing something?

[Alejandro Serrano.Mena]

Long story short: on 1.2 nullable fields are represented by optionals, on 2.0 by lenses

Long story long: there has been a lot of discussions about this over the years. Indeed, the current approach makes it impossible to turn a nullable value into something not nullable; on the other hand makes it easier to query the information inside a big structure, because nullable are just ignored.

At some moment around the development of 2.0 we figured out a way to make optimal fields into lenses, and still keep a nice query language (basically where in 1.2 you write nullableField now you write nullableField.notNull in 2.0). Alas, this is a big breaking change, this is why we decided against including it in the (backwards-compatible) 1.2

We’re happy to discuss other migration strategies which would give users this feature, yet not invalidate any code from previous versions

[leonhardt]

Wow, thank you for all this context. I wasn't aware of the big changes to optics coming in 2.0 but I feel quite caught up now. I found the details you described in the PR and for what it's worth I'm excited for that change. The new type signature is much more intuitive from my perspective and I'm looking forward to working with it.

It's quite helpful to know now that I'm not just missing something (working with 1.2). I'll keep using my workarounds for now and subscribe for your big 2.0 release. For my projects where this is relevant I'm not personally concerned about migration strategies but appreciate how important that is for Arrow.

Thank you for taking the time to reply and for all the care you put into this fantastic library. 🍻

[Alejandro Serrano.Mena]

if I may ask: we tried to discuss a bit these coming changes in https://arrow-kt.io/learn/immutable-data/optional/#nullable-types (the big red thing); do you think this is enough, or should we treat this behavior on nullable types a bit more in depth in the docs?

[leonhardt]

Great question. It was recent enough I still remember! Here's how I managed to miss that information:

1. The immutable data page made me feel that these components (lenses, traversals, prisms, etc) were all optional (no pun intended) and I should choose what I need for my project. I was eager to focus on just what I needed because I'm still learning much of the terminology.
2. I consciously skipped over Optionals because I thought "oh we don't use optionals for anything in our codebase, we always use nullable values."
3. When I found the Lenses page I believed that was the only relevant page to using Optics. I did find it strange "Optics" wasn't the title of one of the pages but I figured I was still learning the mental model of your library and carried on.
4. I hadn't considered that the only way to interact with nullable values using optics might be through using optionals. When I finally figured that out through experimentation in my IDE, it had been a long time since I visited the documentation and I didn't connect it with the page I'd previously thought irrelevant.