Hi all, we are seeing some serialization issues an...
# serialization
m
Hi all, we are seeing some serialization issues and we would like to hear from leaders and the broad community. Full details here: https://github.com/temporalio/sdk-java/issues/2665 The TLDR is our kotlin project is having challenges integrating with Jackson/Gson serialization. Are these compatible to each other? Is it possible to write a custom Jackson/Gson adapter that entirely delegates encode/decode to kotlin?
g
We used a lot of approaches during our mgiration from Gson, but what we do now, is to always rely on Serializers, in all API, they must be explicitly passed It maked all our serialization logic completely type safe
So in general, we had all those toData(Object value) but it's obviously doesn't work well There is serializerOrNullT(), which allows to get serializer from any T, but it's really just a monkey patching And after back and forth we realized that serializer always must be explicit and passed on all levels and not rely on class We are very happy with current solution
Maybe you could do the same and introduce overload which allows to pass serializer
m
thanks @gildor -- our current challenge is that we need to adapt a serializer coming from java-land, so we can't run anything
reified
--or at least I haven't find any ways to do so Right now I can only see two options to move forward: • Option A: Temporal exposes the types (ie.
toData(Object value, Class<T> class)
) so kotlin can do some gymnastics to get the right serializer • Option B: Entirely bypass jackson by using blobs payloads and then manually encode/decode at kotlin (ie.
typealias Blob<T> = String
, which surprisingly works quite well) Any other ideas?
g
so we can't run anything
reified
--or at least I haven't find any ways to do so
But you don't have to
Copy code
SerializersKt.serializer(value.getClass());
There is nothing reified for parsing
> so kotlin can do some gymnastics to get the right serializer You can call it gymnastics, but really it's just a runtime, type unsafe, way to retrieve serializer from object.
There are issues though, for example if your payload is list of serializable classes and so on This why I just suggest to go with: toData(Object value, KSerializer<T> serializer) KSerializer can be retrieved on Java side
m
here's my impl.. which like you said wont work
Copy code
@OptIn(ExperimentalSerializationApi::class)
    private fun toPayloadImpl(value: Any?): Payload? {
        if (value == null) return null

        val serializer = json.serializersModule.serializer(value::class.java) // this wont work
        val data = json.encodeToString(serializer, value)
            .toByteArray()
            .let(ByteString::copyFrom)

        return Payload.newBuilder()
            .setData(data)
            .build()
    }
Finding the right serializer at runtime is the problem, all that java is sending is the object so kotlin works just fine on concrete objects it will fail on very common things, like
List<T>
or anything with generics
it also fails on any class that uses
@Serializable(with = CUSTOM)
I'm starting to think that bypassing java is the way to go; only expose blobs to them, and handle serialization fully at kotlin --the ergonomics are awful
g
> it will fail on very common things, like
List<T>
or anything with generics It's correct, you can try to do some custom thing to resolve it, but I actually think it doesn't worth it
I'm starting to think that bypassing java is the way to go
Kotlin will have all the same problems, that why I just suggest and let client to pass serializer every time
Not sure that I follow, what is json.serializersModule? val serializer = json.serializersModule.serializer(value::class.java) // this wont work
and I don't see a problem to bypass java completely, but maybe you have some specific use cases of course