I've got a problem that I'm pretty sure could be s...

# serialization

[zt]

I've got a problem that I'm pretty sure could be solved with polymorphism but I'm not sure what specifically since its a pretty broad topic and I dont know what I would do in the code to make it possible.

heres the scenario I would like to try to make:

@Serializable
sealed interface Test {
  @Serializable
  @SerialName("a")
  class A : Test
  
  @Serializable
  @SerialName("b")
  class B : Test

  @Serializable
  @SerialName("c")
  class C : Test
}

@Serializable
class Container(val contents: List<Test>)

with such json data:

{
  "contents": [
    {
      "a": { ... }
    },
    {
      "b": { ... }
    },
    {
      "c": { ... }
    }
  ]
}

Elements of contents will always contain a single key, with an object as its value

A temporary solution to this was just to use nullable fields with the matching object names, but its not ideal.

[jw]

That's not valid JSON

[jw]

Either you need contents to be an object with curly braces or the elements of the array need to be valid JSON values.

[zt]

Oh you're right, I'll correct it in a moment

[zt]

@jw Now it's correct

[jw]

This is a form of polymorphism, but I don't think you need to bother.

What I would do is instead write a custom serializer that is applied to the List<Test> which actual deserializes the array as a List<SomeOtherType> where SomeOtherType has the nullable properties a, b, and c. Then, perform a map on the list where you pull out the single non-null value and return that from the serializer.

[zt]

I'm not sure how I could make that

[Nicolas Patin]

Something like this:

data class SomeOtherType(
    val a: Test.A? = null,
    val b: Test.B? = null,
    val c: Test.C? = null,
)

fun List<SomeOtherType>.mapToTests(): List<Test> =
    this.mapNotNull {
        when {
            it.a != null -> it.a
            it.b != null -> it.b
            it.c != null -> it.c
            else -> null
        }
    }

[Nicolas Patin]

(this is the naive implementation, see https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serializers.md#custom-serializers to perform this operation during deserialization)

[zt]

I would need to be able to make that intermediary class during runtime, somehow by reading the json

[ephemient]

class SingletonMapPolymorphicSerializer<T : Any>(
    tSerializer: KSerializer<T>,
    private val discriminator: String = tSerializer.descriptor.annotations.firstNotNullOfOrNull { it as? JsonClassDiscriminator }?.discriminator ?: "type",
) : JsonTransformingSerializer<T>(tSerializer) {
    override fun transformDeserialize(element: JsonElement): JsonElement = buildJsonObject {
        val (type, value) = element.jsonObject.entries.single()
        put(discriminator, type)
        value.jsonObject.forEach { (k, v) -> put(k, v) }
    }

    override fun transformSerialize(element: JsonElement): JsonElement = buildJsonObject {
        putJsonObject(element.jsonObject.getValue(discriminator).jsonPrimitive.content) {
            element.jsonObject.forEach { (k, v) -> if (k != discriminator) put(k, v) }
        }
    }
}

@Serializable
sealed interface Test {
    @Serializable
    @SerialName("a")
    data class A(val foo: String) : Test
    @Serializable
    @SerialName("b")
    data class B(val bar: String) : Test
}

object TestSerializer : KSerializer<Test> by SingletonMapPolymorphicSerializer(Test.serializer())

@Serializable
data class Container(
    val contents: List<@Serializable(with = TestSerializer::class) Test>
)

Json.decodeFromString<Container>("""{"contents":[{"a":{...}},{"b":{...}}]}""")

[zt]

Whats the delegation in TestSerializer for? I haven't found any use for that so I don't know much about it's function

[ephemient]

@Serializable(with = ) needs to be an object for types with no type parameters, and a class with constructor taking the KSerializer of each type parameter for types with type parameters

[ephemient]

alternately you could make open class SingletonMapPolymorphicSerializer<T> { ... }; object TestSerializer : SingletonMapPolymorphicSerializer(...) instead of delegation

[ephemient]

in any case, SingletonMapPolymorphicSerializer doesn't need to know anything specific about your type, just how to map between how the Json serializer wants to represent polymorphism and how you want to represent polymorphism

[ephemient]

thus if you turn on useArrayPolymorphism you'll need a different implementation

[zt]

Is there any way I can make it skip attempting to de-serialize elements present in the JSON, but with no matching class?

[zt]

also this is amazing, I really appreciate it cause this is gonna make my code a lot nicer

[zt]

I'm also curious why does this work

List<@Serializable(with = TestSerializer::class) Test>

But this doesn't and throws the following error

@Serializable(with = TestSerializer::class)
sealed interface Test {

Caused by: java.lang.NullPointerException: Attempt to invoke interface method 'kotlinx.serialization.descriptors.SerialDescriptor kotlinx.serialization.KSerializer.getDescriptor()' on a null object reference

[Dominaezzz]

I was feeling generous so I wrote you some code.

public object TestSerializer : KSerializer<Test> {
    private val subtypes = listOf(Test.A::class, Test.B::class, Test.C::class)
    private val subclasses = listOf(Test.A.serializer(), Test.B.serializer(), Test.C.serializer())

    override val descriptor: SerialDescriptor by lazy(LazyThreadSafetyMode.PUBLICATION) {
        buildClassSerialDescriptor("packagename.Test") {
            for (subclass in subclasses) {
                element(subclass.descriptor.serialName, subclass.descriptor, isOptional = true)
            }
        }
    }

    override fun serialize(encoder: Encoder, value: Test) {
        val index = subtypes.indexOf(value::class)
        @Suppress("UNCHECKED_CAST")
        val subclass = subclasses[index] as KSerializer<Test>
        encoder.encodeStructure(descriptor) {
            encodeSerializableElement(descriptor, index, subclass, value)
        }
    }

    override fun deserialize(decoder: Decoder): Test {
        return decoder.decodeStructure(descriptor) {
            val index = decodeElementIndex(descriptor)
            check(index != CompositeDecoder.DECODE_DONE) {
                "No field in json object"
            }
            val subclass = subclasses[index]
            decodeSerializableElement(descriptor, index, subclass)
        }
    }
}

[ephemient]

@Serializable(with = TestSerializer::class)

because the SingletonMapPolymorphicSerializer I gave you, depends on the original Test.serializer(), which you've prevented from being generated

[ephemient]

Dominic's solution would be a last resort to me… you're bypassing the serialization plugin's automatic handling of sealed hierarchies, and you can screw yourself over by missing a subclass

[Dominaezzz]

Missing a subclass can be solved by using a when statement in the serialiser but I do agree that it's not ideal to bypass the code gen. It's a shame that the current state of polymorphic serialisation is rather opinionated.

[ephemient]

anyhow, if you're going to make a serializer that works only for one class, I'd choose

@Serializable(with = TestSerializer::class)
sealed interface Test {
    @Serializable
    class A : Test
    @Serializable
    class B : Test
    @Serializable
    class C : Test
}

@Serializable
class TestSurrogate(
    val a: Test.A? = null,
    val b: Test.B? = null,
    val c: Test.C? = null,
)

object TestSerializer : KSerializer<Test> {
    override val descriptor: SerialDescriptor = SerialDescriptor("my.package.name.Test", TestSurrogate.serializer().descriptor)
    override fun serialize(encoder: Encoder, value: Test) {
        encoder.encodeSerializableValue(
            TestSurrogate.serializer(),
            when (value) {
                is Test.A -> TestSurrogate(a = value)
                is Test.B -> TestSurrogate(b = value)
                is Test.C -> TestSurrogate(c = value)
            }
        )
    }
    override fun deserialize(decoder: Decoder): Test {
        val value = decoder.decodeSerializableValue(TestSurrogate.serializer())
        return value.a ?: value.b ?: value.c ?: throw SerializationException()
    }
}

[zt]

Well I'll have to use it for multiple classes

[zt]

@ephemient I found this would let me skip missing serializers, or is there a better way?

serializersModule = SerializersModule {
    polymorphic(Any::class) {
        default { null }
    }
}

[ephemient]

data class TypedSerializer<T : Any>(val kclass: KClass<T>, val serializer: KSerializer<T>)
inline fun <reified T:  Any> TypedSerializer(serializer: KSerializer<T>): TypedSerializer<T> = TypedSerializer(T::class, serializer)
class PolymorphicOrNullSerializer<T : Any>(serialName: String, vararg types: TypedSerializer<out T>) : KSerializer<T?> {
    private val types = types

    override val descriptor: SerialDescriptor = buildClassSerialDescriptor(serialName) {
        for ((kclass, serializer) in types) element(serializer.descriptor.serialName, serializer.descriptor, isOptional = true)
    }

    override fun serialize(encoder: Encoder, value: T?) {
        encoder.encodeStructure(descriptor) {
            types.forEachIndexed { index, (kclass, serializer) ->
                if (kclass.isInstance(value)) {
                    encodeSerializableElement(descriptor, index, serializer as KSerializer<T>, value as T)
                    return@encodeStructure
                }
            }
        }
    }

    override fun deserialize(decoder: Decoder): T? = decoder.decodeStructure(descriptor) {
        when (val index = decodeElementIndex(descriptor)) {
            CompositeDecoder.DECODE_DONE -> null
            CompositeDecoder.UNKNOWN_NAME -> null
            else -> decodeSerializableElement(descriptor, index, types[index].serializer)
        }
    }
}

@Serializable
data class Container(
    val contents: List<@Serializable(with = TestSerializer::class) Test?>,
)

object TestSerializer : KSerializer<Test?> by PolymorphicOrNullSerializer(
    "my.package.name.Test",
    TypedSerializer(Test.A.serializer()),
    TypedSerializer(Test.B.serializer()),
    TypedSerializer(Test.C.serializer()),
)

Json { ignoreUnknownKeys = true }.decodeFromString<Container>("""{"contents":[{"a":...},{"z":...}]}""")

[zt]

Is it not possible to have it use all the subclasses of the sealed interface without defining them explicitly? Sorry that I'm asking so many questions

[ephemient]

Test::class.sealedSubclasses.map { it.serializer() }

but this depends on kotlin-reflect, doesn't take sub-subclasses into account (can be fixed), will be slow or outright fail on Android, and doesn't provide any compile-time feedback if you mistakenly miss a @Serializable annotation

[ephemient]

also KClass<T>.serializer() is @InternalSerializationApi

[zt]

yea that would be a problem since im developing mainly for Android right now

[zt]

@ephemient what about using a transforming serializer to convert
ex:

{
  "list": [
    {
      "x": {
        "a": true
      }
    },
    {
      "y": {
        "b": false
      }
    }
  ]
}

to

{
  "list": [
    {
      "type": "x",
      "a": true
    },
    {
      "type": "y",
      "b": true
    }
  ]
}

would that be simpler?