https://kotlinlang.org logo
Join Slack
Powered by
Is it possible to tell kotlinx serialization not t...
# serialization
v
Is it possible to tell kotlinx serialization not to escape a particular string? So if I have a class:
Copy code
@Serializable
data class ContainsJsonString(val name: String, val jsonString: String)
Is there some annotation which will let me tell kotlinx not to escape or otherwise mess about with the 
jsonString
property? I don't need to worry about deserialization, only interested in writing the class `ConstainsJsonString`to a json file. Output would be something like:
Copy code
{
  "name": "Liam",
  "jsonString": {
    "address": "1 Acacia Avenue",
    "favouriteColour": "blue"
  }
}
Right now I'm getting something like:
Copy code
{
  "name": "Liam",
  "jsonString": "{ \\\"address\\\" : \\\"1 Acacia Avenue\\\":\\\"favouriteColour\\\":\\\"blue\\\"}"
}
(Posted in this the wrong channel earlier, sorry!)
a
v
I saw that in the documentation but wasn't sure if it worked in my case. I'll try it out. This is all wrapped in a Sealed Class which is serialized normally.
a
I suspect this will work, but I’ve not tried it UPDATE: this doesn’t work, keep reading!
RawJsonStringSerializer.kt
Copy code
@Serializable
data class ContainsJsonString(val name: String, val jsonString: RawJsonString)
hmm probably not actually, it will intercept the JsonElement too late in the process… or maybe not. I can’t remember!
v
It's a start! Use is case I'm writing an API which returns an existing JSON file, but wrapped in a 
Result
sealed class with some status, etc values. I tried loading the json, deserializing that, adding the object to the Result class, and then serializing everything, but that went horribly wrong and produced invalid JSON. As this is just one-way, I thought I could just return the JSON file content directly as a String in the Result sealed class.
Would I need to build 1.5.1-RC myself?
a
no 1.5-RC was released yesterday
v
I was being dumb. Typo in my version number...
I don't think it's working, though I could be making mistakes. Based on your 
JsonUnquotedLiteral
code, I've written a test case which is failing.
Copy code
@Test
    fun `can serialize an API JsonSuccess result containing a literal json string`() {
        val expected = """
            {"jsonString":{"name":"Json Literal","count":1}}
        """.trimIndent()

        val jsonObject = WillBeJson(name = "Json Literal", count = 1)
        val jsonObjectString = Json.encodeToString(jsonObject)
        val jsonSuccess = APIResult.JsonSuccess(jsonString = jsonObjectString)

        val result = Json.encodeToString(jsonSuccess)

        println(result)
        assertEquals(expected, result)
    }
Test fails:
Copy code
Expected :{"jsonString":{"name":"Json Literal","count":1}}
Actual   :{"jsonString":"{\"name\":\"Json Literal\",\"count\":1}"}
a
yeah, the example I gave won’t work - 
JsonUnquotedLiteral
needs to be used before the content is converted into a 
JsonPrimitive
(I think). OR maybe the 
typealias
doesn’t work for primitives. Or both!
I’m having a closer look now. A value class will work, but I’m trying to see if I can make it more succinct
v
Tricky to debug into these custom serializers!
a
this works, but it’s a little messy
RawJsonStringSerializer.kt
if you only have a few properties that you want to raw-encode, here’s a more succinct version. It requires adding 
@Serializable(with = RawJsonStringSerializer::class)
to the property. If you have lots of properties I’d recommend using the 
value class RawJsonString(...)
method, which reduces the annotation spam. I’m not sure why the 
typealias
doesn’t work - I’ll make an issue. EDIT: this is a known issue that will be fixed in 1.8.20, and might be backported to 1.8…?
RawJsonStringSerializer.cpp
v
There should only ever be one of these raw Json 'objects' in the class.
It's working in the simple cases but I've got a lot of generics flying around which is complicating real-world use. Nonetheless, i think I have a good place to start, so thank you!
a
yeah generics are really difficult to encode/decode! I try and avoid them as much as possible.
v
I think I've lost track of whether his is working or not! My final, serialized output is currently sitting at:
Copy code
{"statusCode":200,"headers":{"Content-Type":"application/json"},"body":"{\"jsonString\":{\r\n  \"layouts\": .... // rest of my raw json here
So a couple of things: the body is still being escaped, but I think that's actually AWS API Gateway doing this, because an APIGatewayProxyResponseEvent's body should be escaped to a single string, not a Json entity. But what I think is wrong is the appearance of 
jsonString
in the body. That's coming from the serializer surrogate which isn't really understanding that this is a value class. From the docs, I think I want the RawJsonStringSerializer to call something like 
encoder.encodeInlineElement().encodeJsonElement(JsonUnquotedLiteral(value.content))
but I know that isn't how these things work. Anyway, progress, and I could live with it.
I've raised a bug/feature request with the specifics of my code. I might be asking too much, of course, and I've a feeling that if it could work for serialization, it might not be at all possible for deserialization, and hence not supported. Still, I thought I'd check. https://github.com/Kotlin/kotlinx.serialization/issues/2177
a
could you say more about why you expect a JSON object with a 
body
property? I don’t see any property named 
body
in your code. But there is a 
jsonString
property (in the 
APIResultSurrogate
class), so it looks correct that resulting JSON has a 
jsonString
property
v
It's all a complex chain of classes, and I've cut out some of the outer classes, which relate to AWS API Gateway lambda functions (which is where the 
body
and other related parts come from). I mostly understand why I get the result I get, I just hoped the value class would give me the result I hoped for. If I strip away the sealed class and the surrogate, it works with the value class.
s
sorry to open an old thread, but this seems like exactly the same type of problem I am trying to solve, even down to the api gateway lambda functions bit, but in the opposite direction. Essentially I want to be able to decode an object like this (this is just the structure of the aws lambda proxy payload)
Copy code
@Serializable
sealed class LambdaProxyRequest<T>(
    val version: String? = null,
    val routeKey: String? = null,
    val rawPath: String? = null,
    val rawQueryString: String? = null,
    val cookies: List<String>? = null,
    val headers: Headers? = null,
    val queryStringParameters: QueryStringParameters? = null,
    val requestContext: RequestContext? = null,
    val body: T? = null,
    val pathParameters: PathParameters? = null,
    val isBase64Encoded: Boolean? = null,
    val stageVariables: StageVariables? = null,
)

object Request1: LambdaProxyRequest<CalculateRequest>()
object Request2: LambdaProxyRequest<DoOtherStuffRequest>()
where the 
body
actually comes across the wire as an escaped JSON string, but I want to deserialize it as the generic object, rather than manually pull out the escaped string, then unescape it, then decode it again. I don’t exactly see how the Json primitive stuff y’all talked about above would solve this, but I do see how it could be done with a custom deserializer that simply searches each index until it finds the 
body
and then unescapes it, then finally deserializes the whole object as if nothing was changed. But I’m having trouble implementing that. Am I off base here? Is it not possible to do this?
For example, I can do something like this
Copy code
object SomeProxyDeserializer: KSerializer<Some> {
    fun unescapeJson(escapedJson: String): String {
        return escapedJson.replace("""\"""", "\"")
    }
    override val descriptor: SerialDescriptor
        get() = PrimitiveSerialDescriptor("body", PrimitiveKind.STRING)

    override fun deserialize(decoder: Decoder): Some {
        val unescapedString = unescapeJson(decoder.decodeString())
        return Json.decodeFromString(Some.serializer(), unescapedString)
    }

    override fun serialize(encoder: Encoder, value: Some) {
        TODO("No need to ever serialize this, probably. XD")
    }
}
but it doesn’t use the original Json object that was used to start the deserialization process, thus immediately failing as some of the serialization modules aren’t available.
and it doesn’t solve the generics problem at all, i have to manually set the serializer to a single instance.
a
I think it'd be better if you started a new thread :)
s
Will do
thank you color 1
774 Views
Open in Slack
PreviousNext
Powered by