https://kotlinlang.org logo
#arrow
Title
# arrow
d

dave08

03/15/2023, 12:10 PM
Is there a way to use Optics to work with KotlinX Serialization's JsonObject hierarchy? i'd like to pinpoint on certain parts of a big, very nested json and just modify it...
🙌 1
s

simon.vergauwen

03/15/2023, 12:27 PM
Yes, I've build a specialised DSL based on Optics for it. https://github.com/nomisRev/kotlinx-serialization-jsonpath
It offers special functions to more easily access deeper parts of a very nested
JsonElement
.
d

dave08

03/15/2023, 12:28 PM
It's production-ready?
s

simon.vergauwen

03/15/2023, 12:29 PM
Yes, of course. What would you consider not-production ready?
d

dave08

03/15/2023, 12:30 PM
Well, it's not 1.0.0 😊... in some libraries that's considered experimental or WIP...
s

simon.vergauwen

03/15/2023, 12:31 PM
Ah, I guess I could/should release it as 1.0.0 😅
The API is stable, and tested.
d

dave08

03/15/2023, 12:37 PM
I hope you'll reference all these great libraries in the new site 😃... I keep on thinking that something's not possible/not implemented just to discover that you've already written a library for it 😅.
s

simon.vergauwen

03/15/2023, 12:38 PM
We have a page for this already 😉 I am pushing today again to have it released in preview, but it'll probably be for in 2 weeks. I have some much needed PTO next week 😂
d

dave08

03/15/2023, 12:44 PM
Minor detail... my JsonObject keys contains a
.
... is there a way to escape them?
Is there a reference for the JsonPath select syntax you're using there?
Ok, I think I got it JsonPath["..."]["..."]...
s

simon.vergauwen

03/15/2023, 12:50 PM
I'm not sure I understood your question
d

dave08

03/15/2023, 12:51 PM
Copy code
{
  "some.key": {
    "other-key": [
      "value"
    ]
  }
}
Copy code
JsonPath.select("some.key.other-key") // would look for { "some": { "key": { "other-key" ...
Another thing... to modify
"value"
I'd have to map my set of strings to `JsonElement`s I guess... I wouldn't be able to create a function that allows me to set the list from a collection of strings directly?
s

simon.vergauwen

03/15/2023, 12:55 PM
You can by calling
.string
or
.int
on the DSL
It currently it's possible to escape
.
, but that is something we could quite easily add. We can parameterise splitting on a parameterised
Char
rather than hardcoding
.
Otherwise using
[][]
can currently be used as a workaround. Feel free to open a issue, and I'll try to get to is asap ☺️ A PR fixing it is of course also very welcomed. The codebase is quite small, so it's less intimidating to approach than Arrow itself.
Would give a good excuse to bump to 1.0.0 I guess 😄
d

dave08

03/15/2023, 1:01 PM
I'm still not sure how to set the list "value" is in:
Copy code
val listRef = JsonPath["some.key"]["other-key"].every.string

listRef.set(jsonElem, listOf("other-value")) // this doesn't work
s

simon.vergauwen

03/15/2023, 1:02 PM
Copy code
JsonPath["some.key"]["other-key"].every.string
  .modify(jsonElem) { original: String -> "other-value" }
d

dave08

03/15/2023, 1:02 PM
I want to overwrite the whole list
s

simon.vergauwen

03/15/2023, 1:05 PM
JsonPath["some.key"]["other-key"].jsonArray()
would give you
Optional<JsonElement, List<JsonElement>>
but then you'd want to turn
List<JsonElement>
into
List<String>
🤔
Not sure how to do that conveniently
Copy code
JsonPath["some.key"]["other-key"].jsonArray().modify(jsonElem) {
  listOf(JsString("other-value"))
}
this definitely works
Not sure if it'd be easy to make that better, since JsonArray can contain JsElement of different types
d

dave08

03/15/2023, 1:08 PM
Yeah, so in my case, it doesn't...
s

simon.vergauwen

03/15/2023, 1:08 PM
But the type system doesn't know that 🧌
d

dave08

03/16/2023, 11:39 AM
It seems like Intellij doesn't like your setup for some reason... There's no code navigation, it doesn't seem to catch that those are source sets...
s

simon.vergauwen

03/16/2023, 11:44 AM
That's really weird.. I am having the same issue now but it has the same layout as all my other projects 😕
d

dave08

03/16/2023, 11:46 AM
I go to project structure, mark them as source sets... they get highlighted (with red lines under arrow deps).. I try to resync with the gradle file, it goes back to not being source sets...
Maybe Gradle 8 changed something for KMM projects?
I wanted to give a PR a shot... but with my little knowledge of optics (I was hoping to learn more about optics along the way...), and no code navigation... I guess I'll wait until there's a solution for this...
s

simon.vergauwen

03/16/2023, 12:02 PM
Might be related to missing Kotlin sources after all:
Could not find kotest-common.klib (io.kotest:kotest-common-iosx64:5.5.5).
I currently don't have time to investigate this
d

dave08

03/16/2023, 12:02 PM
No rush 😊, just reporting...
@simon.vergauwen Can regular optics be combined with that libraries optics?
Copy code
@JvmInline
value class Bar(val json: JsonObject) {
  companion object {
     val path = JsonPath....
  }
}

@optics data class Foo(val bar: Bar)

foo.copy { Foo.Bar.path set ... }
s

simon.vergauwen

03/16/2023, 2:04 PM
(Foo.bar compose Bar.path) set ...
Not sure if
+
is still available as alias for
compose
.
d

dave08

03/16/2023, 2:05 PM
compose is red
None of the following functions can be called with the arguments supplied. compose(Fold<in CustomRestrictions, out TypeVariable(C)>) where C = TypeVariable(C) for fun <C> compose(other: Fold<in CustomRestrictions, out C>): Fold<RestrictionAction, C> defined in arrow.optics.POptional compose(PEvery<in CustomRestrictions, out CustomRestrictions, out TypeVariable(C), in TypeVariable(D)>) where C = TypeVariable(C), D = TypeVariable(D) for fun <C, D> compose(other: PEvery<in CustomRestrictions, out CustomRestrictions, out C, in D>): PEvery<RestrictionAction, RestrictionAction, C, D> defined in arrow.optics.POptional compose(POptional<in CustomRestrictions, out CustomRestrictions, out TypeVariable(C), in TypeVariable(D)>) where C = TypeVariable(C), D = TypeVariable(D) for infix fun <C, D> compose(other: POptional<in CustomRestrictions, out CustomRestrictions, out C, in D>): POptional<RestrictionAction, RestrictionAction, C, D> defined in arrow.optics.POptional compose(POptionalGetter<in CustomRestrictions, RestrictionAction, out TypeVariable(C)>) where C = TypeVariable(C) for fun <C> compose(other: POptionalGetter<in CustomRestrictions, RestrictionAction, out C>): POptionalGetter<RestrictionAction, RestrictionAction, C> defined in arrow.optics.POptional compose(PSetter<in CustomRestrictions, out CustomRestrictions, out TypeVariable(C), in TypeVariable(D)>) where C = TypeVariable(C), D = TypeVariable(D) for fun <C, D> compose(other: PSetter<in CustomRestrictions, out CustomRestrictions, out C, in D>): PSetter<RestrictionAction, RestrictionAction, C, D> defined in arrow.optics.POptional compose(PTraversal<in CustomRestrictions, out CustomRestrictions, out TypeVariable(C), in TypeVariable(D)>) where C = TypeVariable(C), D = TypeVariable(D) for fun <C, D> compose(other: PTraversal<in CustomRestrictions, out CustomRestrictions, out C, in D>): PTraversal<RestrictionAction, RestrictionAction, C, D> defined in arrow.optics.POptional
s

simon.vergauwen

03/16/2023, 2:06 PM
What type is
path
?
d

dave08

03/16/2023, 2:07 PM
Optional<JsonElement, List<JsonElement>>
Ohh... the value class.
It should have been Optional<Bar, List<JsonElement>>
But how do I adapt that? value classes can't have optics.
And it's a bit overkill.
s

simon.vergauwen

03/16/2023, 2:09 PM
You could write an
Iso
for it and put it in between
d

dave08

03/16/2023, 2:09 PM
?
s

simon.vergauwen

03/16/2023, 2:11 PM
Copy code
@JvmInline
value class Bar(val json: JsonElement) {
  companion object {
     val path: Optional<Bar, List<JsonElement>> =
       json compose JsonPath....

     val json: Iso<Bar, JsonElement> = Iso(
       get = { it.json },
       reverseGet = { Bar(it) }
     )
  }
}

@optics data class Foo(val bar: Bar)

foo.copy { (Foo.Bar compose Bar.path) set ... }
d

dave08

03/16/2023, 2:12 PM
Thanks!
So I could also make such an Iso for my previous List<JsonElement> to List<String>, no?
s

simon.vergauwen

03/16/2023, 2:19 PM
It would be in violation of the laws which is a set of tests that the Optics type should adhere in order for them to work correctly. You could create an unsafe
Iso
which makes a type-unsafe assumption that
List<JsonElement> == List<String>
.
TL;DR it will requires an unsafe cast, which violates the type system but yes anything is possible in code 😄
d

dave08

03/16/2023, 2:22 PM
Setting has no problems with types... only reading. But I guess it's all one big package deal in optics?
If there would be a way to implement only reverseGet and just use the setter part of optics for such cases.
Or alternatively, even possibly have some kind of Either to return for the get case which would just return Left if the cast wasn't successful
But I guess that would make things much more complex
s

simon.vergauwen

03/16/2023, 2:26 PM
That is possible with
Prism
but I'm not sure on the top of my head how to do it for
List<JsonElement> -> List<String>
. We have it for
JsonElement -> String
.
d

dave08

03/16/2023, 2:27 PM
Ok, I just learned what Iso is... but what's Prism?
s

simon.vergauwen

03/16/2023, 2:28 PM
d

dave08

03/16/2023, 2:39 PM
Ok... so a Prism allows to return an Either on
get
... so there's no such concept for each element in a collection?
s

simon.vergauwen

03/16/2023, 2:40 PM
Nope, but I though you didn't want that? Otherwise you can just use
path.every.string
You can still compose
.every
after you go from
List<JsonElement>
to
List<String>
though, and it should be possible to write a Prism for that
d

dave08

03/16/2023, 2:44 PM
I need to set the whole list (overwrite the current contents)... every does that?
Something like val baz = Optional<Bar, List<String>>
that allows
baz.set(bar, listOf("one", "two"))
s

simon.vergauwen

03/16/2023, 2:45 PM
Ok... so a Prism allows to return an Either on
get
... so there's no such concept for each element in a collection?
Then I don't understand your question
d

dave08

03/16/2023, 2:46 PM
Well, I don't know Prism's return type so I just expressed it as Optional...
But whatever gives that set(...)
s

simon.vergauwen

03/16/2023, 2:47 PM
Both Optional and Prism have
set
d

dave08

03/16/2023, 2:57 PM
I guess this should do the trick:
Copy code
private val prism: Prism<List<JsonElement>, List<String>> = Prism(
    getOption = { orig -> 
            orig.mapNotNull { it.jsonPrimitive.contentOrNull }
                .takeIf { orig.size == it.size }?.some() ?: none() 
        },
    reverseGet = { it.map { value -> JsonPrimitive(value) } }
)
Maybe I'm mixing up Optional here... but there IS getOption on Prism... 🤔, Maybe I'm not doing this correctly? Or maybe the better practice would be the Either alternative...?
s

simon.vergauwen

03/16/2023, 3:49 PM
is something not working?
d

dave08

03/16/2023, 4:09 PM
No, everything's fine 😃, just wondering if I'm doing things "right".
s

simon.vergauwen

03/16/2023, 4:10 PM
Yes, you're doing it right ☺️ You can think of
Prism
as
when/is
and
Optional
as
?.
. The new website is planned to launch end of this month btw 😉
d

dave08

03/16/2023, 4:11 PM
You can think of
Prism
as
when/is
and
Optional
as
?.
.
Not sure I understand that -- they both have the same getOption...?
Couldn't I have just used Optional instead of Prism in the same way?
Or would that be abusing Optional...
s

simon.vergauwen

03/16/2023, 4:13 PM
Yes, but Optics always exists out of 2 operations.
Prism
doesn't have
set
but
reverseGet
which is required to go from
List<String>
back to
List<JsonElement>
.
d

dave08

03/16/2023, 4:29 PM
Just a bit curious... why does the
@optics
annotation generate inline vals? Wouldn't it be more efficient to have regular vals...?
s

simon.vergauwen

03/16/2023, 4:30 PM
It's not possible to generate regular val with backing field as an extension on
Companion
. In order for use to do that we need stable compiler plugins, if we get those than we can apply more optimisations.
8 Views