Is this the best way to handle deserialization of ...

# ktor

[Lawik]

Is this the best way to handle deserialization of lists with the http client on the JS platform? https://stackoverflow.com/a/54714777
I see a setListMapper function but that doesn't seem to do anything.

[ribesg]

For top level lists, yes. Because top level lists aren’t valid json and kotlinx.serialization only supports valid json

[ribesg]

(Which is stupid)

[Lawik]

This isn't valid json 😐?

[{
	"id": 2,
	"name": "test",
	"age": 20
}, {
	"id": 3,
	"name": "test",
	"age": 21
}, {
	"id": 4,
	"name": "test2",
	"age": 18
}]

[ribesg]

No it’s not. Valid json always starts with {

[ribesg]

{
    "results": [...]
}

That’s how you should do it. There are various reasons for that... but in the real world there are APIs who return arrays. But the folks working on kotlinx.serialization don’t really care about the real world. It’s the same for the null problem.

[Lawik]

I saw that you used the same solution for this problem, were you able to create a generic wrapper for this or do I have to create a new list wrapper class for every single DTO?

[ribesg]

You need a new serializer every time if you compile to native

[ribesg]

I’m not sure how generics would work there, but it doesn’t work on native so I didn’t try

[ribesg]

If you’re just on the JVM you could try

[Lawik]

I work on JS.

[ribesg]

Unlucky. If I remember correctly it’s the same problem for JS, there’s no reflection

[Lawik]

I'll give it a shot.

[Lawik]

Shoot, that just brings me back to last week's problem 😔

[Lawik]

What's your opinion on this solution (this is just a start, still need to figure out how to ensure T is serializable, and add proper handling for request settings):

@ImplicitReflectionSerializer
    suspend inline fun <reified T : Any> HttpClient.getList(): List<T> = request<HttpResponse> {
        // other settings: url, etc
        method = HttpMethod.Get
    }.use {
        if (it.status.isSuccess()) {
            Json.parse(T::class.serializer().list, it.readText())
        } else {
            throw BadResponseStatusException(it.status, it)
        }
    }

This allows me to perform the request as follows:

actual suspend fun getAll(): List<PersonDTO> = client.getList(/*request settings*/)

?

[ribesg]

Looks ok to me

[Lawik]

Alright, I'll go with that. Thanks a lot for all the help!

[r4zzz4k]

@ribesg could you please point out a link mentioning the fact JSON should be an object and not an array or another value? Grammar defined on json.org seem to allow any JSON element to be valid JSON.

[jvassbo]

RFC 7159 states that valid JSON must be a valid JSON value, not just an object. This also goes on top level. So array or just a value is also valid json per spec

[ribesg]

@r4zzz4k @jvassbo I’m aware of that. The grammar is clear, multiple RFCs allow for example 42 to be returned as a valid json response. But that’s not what the untold standard is, because of various historical reasons, some of which are still valid today

[ribesg]

A very quick google search returns this one for example https://haacked.com/archive/2009/06/25/json-hijacking.aspx/

[ribesg]

All the vulnerabilities are nice, but my favorite reason is the fact that arrays can’t evolve. When you return an object, you can add a new entry to the returned payload without breaking every client. You can’t do that when your root element is an array.

[ribesg]

Nowadays you could implement your API returning arrays and objects, and use versioning for evolution. But for historical reasons it’s still best to use only objects today. Even a deserializer as young as kotlinx.serialization doesn’t support top level arrays... Do you think it’s the only one?

[ribesg]

Just look where we are and why we’re here: the developers of kotlinx.serialization ever forgot that top level arrays were a thing, or thought it was not something important to implement. There are reasons behind this, conscious or not