https://kotlinlang.org logo
Join Slack
Powered by
How would I go about encoding/decoding one type to...
# serialization
s
How would I go about encoding/decoding one type to another, where the second type just needs a few functions called to insert the elements. e.g: input:
Copy code
@Serializable
data class ChangeOrderInput(
    val oldProposal: OldProposal,
    val newProposals: List<NewProposal>
)
output:
Copy code
@Serializable
data class ChangeOrderUnit(
    val oldProposal: SingletonStore<OldProposal>,
    val newProposals: ListDataStore<NewProposal>,
) : RuleUnitData
where 
ChangeOrderUnit
will always be deserialized from 
ChangeOrderInput
, never the other way around, and where SingletonStore/ListDataStores are created just by calling 
oldProposal.set()
or 
newProposals.add()
on the stores, with the element. e.g. the only difference between the objects are that the RuleUnit object has the objects contained in stores.
this needs to act exactly like the 
convertValue
method from Jackson. I'm not sure if I need to write a custom serializer or if there's someway I can force generic deserialization here. It seems like I would need something like this:
Copy code
object SingletonStoreSerializer : KSerializer<SingletonStore<*>> {
    override val descriptor: SerialDescriptor = ChangeOrderUnit.serializer().descriptor

    override fun serialize(encoder: Encoder, value: SingletonStore<*>) {
        encoder.encodeSerializableValue(String.serializer(), value.toString())
    }

    override fun deserialize(decoder: Decoder): SingletonStore<*> {
        val string = decoder.decodeString()
        decoder.decodeStructure(descriptor) {
            decodeNullableSerializableElement()
        }
        return Color(string.toInt(16))
    }
}
but I can't figure out exactly what I need to do from the documentation. I don't really ever need to serialize the SingletonStore, I just need to deserialize it from either json or the ChangeOrderInput itself.
r
s
Thanks for the links @Richard Gomez. It was almost 1am when I posted this thread and I definitely should have just waited until morning. 😴 You are correct, it is Kogito.
e
if you can transform between ChangeOrderInput and ChangeOrderUnit, then you can declare a custom serializer for ChangeOrderUnit that uses ChangeOrderInput.serializer() and just performs some operations before/after
note that while in theory SerializationStrategy and DeserializationStrategy are two separate things, most of the kotlinx.serialization API wants both to be the same (a.k.a. KSerializer), so you would be best off defining both serialization and deserialization
s
@ephemient I've got this so far, but my app is failing to start, seems like I'm doing something wrong (I know I'm doing something wrong with the list serializer, but slowly working toward that)
Copy code
class SingletonStoreSerializer<T>(private val dataSerializer: KSerializer<T>) : KSerializer<SingletonStore<T>> {
    override val descriptor: SerialDescriptor = dataSerializer.descriptor
    override fun serialize(encoder: Encoder, value: SingletonStore<T>) = dataSerializer.serialize(encoder, value.first())
    override fun deserialize(decoder: Decoder) =
        DataSource.createSingleton<T>().apply {
            set(dataSerializer.deserialize(decoder))
        }
}

class ListDataStoreSerializer<T>(private val dataSerializer: KSerializer<T>) : KSerializer<ListDataStore<T>> {
    override val descriptor: SerialDescriptor = dataSerializer.descriptor
    override fun serialize(encoder: Encoder, value: ListDataStore<T>) = dataSerializer.serialize(encoder, value.first())
    override fun deserialize(decoder: Decoder) =
        ListDataStore<T>().apply {
            add(dataSerializer.deserialize(decoder)) // TODO this is wrong
        }
}
I feel like this should work, so I might be doing something else wrong.
e
you want something along the lines of
Copy code
private val surrogateSerializer = ListSerializer(dataSerializer)
override val descriptor: SerialDescriptor = surrogateSerializer.descriptor
override fun serialize(encoder: Encoder, value: ListDataStore<T>) {
    encoder.encodeSerializableValue(surrogateSerializer, value.toList())
}
override fun deserialize(decoder: Decoder): ListDataStore<T> {
    val list = decoder.decodeSerializableValue(surrogateSerializer)
    return ListDataStore<T>().apply {
        addAll(list)
    }
}
s
ahh. this is where the surrogates come in, ok thanks. I find the docs quite confusing and not too comprehensive, but that might just be my inexperience with custom deserialization.
thanks!
e
well, in the singleton case, the 
dataSerializer
is your "surrogate", that's fine
just in the second case you want 
KSerializer<List<T>>
, not 
KSerializer<T>
itself
s
yeah I thought the singleton one looked fine, it was the list one that confused me.
Oh!?
wait, like I should change my serializer type?
Copy code
class ListDataStoreSerializer<T>(dataSerializer: KSerializer<T>) : KSerializer<ListDataStore<T>> {
    private val surrogateSerializer = ListSerializer(dataSerializer)
    override val descriptor: SerialDescriptor = surrogateSerializer.descriptor
    override fun serialize(encoder: Encoder, value: ListDataStore<T>) {
        encoder.encodeSerializableValue(surrogateSerializer, value.toList())
    }
    override fun deserialize(decoder: Decoder): ListDataStore<T> {
        val list = decoder.decodeSerializableValue(surrogateSerializer)
        return ListDataStore<T>().apply {
            list.forEach { add(it) }
        }
    }
}
e
that looks fine
if your app is still crashing, what is the error?
I mean that 
dataSerializer: KSerializer<T>
doesn't (de)serialize a list, that's why 
surrogateSerializer: KSerializer<List<T>> = ListSerializer(dataSerializer)
is needed
then you just adapt 
KSerializer<List<T>>
to a 
KSerializer<ListDataStore<T>>
anyhow, next thing is you'll likely need is to make sure 
SingletonStoreSerializer
and 
ListDataStoreSerializer
are registered; easiest way will likely be
Copy code
@file:UseSerializers(SingletonStoreSerializer::class, ListDataStoreSerializer::class)
at the top of the file that you define 
@Serializable class ChangeOrderUnit
in, that will cause its generated serializer to use those custom serializers for its properties
or just skip that altogether, instead directly write a custom serializer for 
ChangeOrderUnit
which uses 
ChangeOrderInput.serializer()
as its surrogate. since it's your own type, you can register the serializer right there, e.g. 
@Serializable(with = ChangeOrderUnitSerializer::class) data class ChangeOrderUnit(...)
and it'll work for all usages
s
I was using 
@Contextual
to register those serializers. is that incorrect?
Copy code
@Serializable
data class ChangeOrderUnit(
    @Contextual val oldProposal: SingletonStore<OldProposal>,
    @Contextual val newProposals: ListDataStore<NewProposal>,
) : RuleUnitData {
    constructor() : this(
        DataSource.createSingleton<OldProposal>(),
        ListDataStore()
    )
}
and one of the main points of writing these serializers was that I wanted to be able to contribute them back to the kogito project so that anyone could serialize using kotlinx rather than having to write all these custom serializers. Ideally I'd have something that could serialize any 
RuleUnitData
, but haven't figured that one out yet.
the failure in my app doesn't indicate any kotlinx stuff, so I'm guessing it's just interacting with kogito weirdly. I need to dig deep.
e
as code to be shared, serializers for those is sensible
if they're not going to have different serializers under different formats, I wouldn't use Contextual. just have it bind to the one and only serializer
s
Ah, I misunderstood Contextual then. So Contextual is for when you might serialize Color as either 
#FFFFFF
or 
r=255, g=255, b=255
etc?
e
right, it gives you the ability to
Copy code
val json = Json {
    serializersModule = SerializersModule {
        contextual(if (oldVersion) ColorAsHashSerializer else ColorAsTripletSerializer)
    }
}
and change all 
@Contextual Color
throughout the whole tree
s
Ha, so the build/run errors I was getting was actually due to adding 
@Contextual
 as an annotation. Apparently kogito tries to read the annotation for some reason and fails. It looks like I am past those failures. Thank you for the help! I still have some stuff to figure out, it's not working yet, but this has gotten me much further than last night. 🎉 🙇🏽
@ephemient it seems that my SingletonDeserializer fails to work when the object it's deserializing is a polymorphic type using sealed classes. I can't seem to figure out why though...
Copy code
2021-12-08 11:31:13,310 ERROR [io.qua.ama.lam.run.AbstractLambdaPollLoop] (Lambda Thread (DEVELOPMENT)) Failed to run lambda (DEVELOPMENT): kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 553: Expected start of the array '[', but had ':' instead
JSON input: ....."Standard"}]}},"oldPricePoint":{"type":"Monthly","id":"","pr.....
	at kotlinx.serialization.json.internal.JsonExceptionsKt.JsonDecodingException(JsonExceptions.kt:24)
	at kotlinx.serialization.json.internal.JsonExceptionsKt.JsonDecodingException(JsonExceptions.kt:32)
	at kotlinx.serialization.json.internal.AbstractJsonLexer.fail(AbstractJsonLexer.kt:524)
	at kotlinx.serialization.json.internal.AbstractJsonLexer.fail$kotlinx_serialization_json(AbstractJsonLexer.kt:221)
	at kotlinx.serialization.json.internal.AbstractJsonLexer.unexpectedToken(AbstractJsonLexer.kt:204)
	at kotlinx.serialization.json.internal.StringJsonLexer.consumeNextToken(StringJsonLexer.kt:74)
	at kotlinx.serialization.json.internal.StreamingJsonDecoder.beginStructure(StreamingJsonDecoder.kt:41)
	at kotlinx.serialization.internal.AbstractPolymorphicSerializer.deserialize(AbstractPolymorphicSerializer.kt:130)
	at com.sunrun.pricing.changeorder.helpers.SingletonStoreSerializer.deserialize(Serialization.kt:29)
	at com.sunrun.pricing.changeorder.helpers.SingletonStoreSerializer.deserialize(Serialization.kt:24)
	at kotlinx.serialization.json.internal.PolymorphicKt.decodeSerializableValuePolymorphic(Polymorphic.kt:59)
	at kotlinx.serialization.json.internal.StreamingJsonDecoder.decodeSerializableValue(StreamingJsonDecoder.kt:36)
	at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:43)
	at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableElement(AbstractDecoder.kt:70)
	at com.sunrun.pricing.changeorder.ChangeOrderUnit$$serializer.deserialize(ChangeOrderUnit.kt:17)
	at com.sunrun.pricing.changeorder.ChangeOrderUnit$$serializer.deserialize(ChangeOrderUnit.kt:17)
	at kotlinx.serialization.json.internal.PolymorphicKt.decodeSerializableValuePolymorphic(Polymorphic.kt:59)
	at kotlinx.serialization.json.internal.StreamingJsonDecoder.decodeSerializableValue(StreamingJsonDecoder.kt:36)
	at kotlinx.serialization.json.Json.decodeFromString(Json.kt:100)
	at com.sunrun.pricing.changeorder.ChangeOrderFnHandler$handleRequest$1.invoke(ChangeOrderFnHandler.kt:70)
	at com.sunrun.pricing.changeorder.ChangeOrderFnHandler$handleRequest$1.invoke(ChangeOrderFnHandler.kt:26)
	at com.sunrun.pricing.changeorder.helpers.serialization.RequestStreamWrapperKt.requestStreamWrapper(RequestStreamWrapper.kt:17)
	at com.sunrun.pricing.changeorder.ChangeOrderFnHandler.handleRequest(ChangeOrderFnHandler.kt:26)
	at io.quarkus.amazon.lambda.runtime.AmazonLambdaRecorder$1.processRequest(AmazonLambdaRecorder.java:183)
	at io.quarkus.amazon.lambda.runtime.AbstractLambdaPollLoop$1.run(AbstractLambdaPollLoop.java:111)
	at java.base/java.lang.Thread.run(Thread.java:829)
it seems it's because it's expecting a 
[
when deserializing the polymorphic type, though I don't understand why. from 
kotlinx.serialization.json.internal.WriteMode
Copy code
internal enum class WriteMode(@JvmField val begin: Char, @JvmField val end: Char) {
    OBJ(BEGIN_OBJ, END_OBJ),
    LIST(BEGIN_LIST, END_LIST),
    MAP(BEGIN_OBJ, END_OBJ),
    POLY_OBJ(BEGIN_LIST, END_LIST);
}

@OptIn(ExperimentalSerializationApi::class)
internal fun Json.switchMode(desc: SerialDescriptor): WriteMode =
    when (desc.kind) {
        is PolymorphicKind -> WriteMode.POLY_OBJ
        StructureKind.LIST -> WriteMode.LIST
        StructureKind.MAP -> selectMapMode(desc, { WriteMode.MAP }, { WriteMode.LIST })
        else -> WriteMode.OBJ
    }
why does 
PolymorphicKind
want to have a list handed to it? How can I force it to write with 
.OBJ
instead?
actually the error is 
java.lang.IllegalArgumentException: Polymorphic value has not been read for class null
...
116 Views
Open in Slack
PreviousNext
Powered by