https://kotlinlang.org logo
#serialization
Title
# serialization
j

janvladimirmostert

01/29/2023, 7:30 PM
I'm looking to mask certain values like
email
if it's serialized for the purpose of logging, but not if it's serialized for the purpose of sending it over the wire, so I'm looking for a way to swop out Serializers depending on context. Example:
Copy code
@JvmInline
@Serializable
value class Email(override val value: String) : Sensitive {
    override fun mask(): String {
        return value.split("@").let {
            "_".repeat(it.first().length) + "@" + it.last()
        }
    }
    object SensitiveSerializer : KSerializer<Email> {
        override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Email", PrimitiveKind.STRING)
        override fun deserialize(decoder: Decoder): Email = Email(decoder.decodeString())
        override fun serialize(encoder: Encoder, value: Email) = encoder.encodeString(value.mask())
    }
}
Copy code
val wireMapper = Json {
    prettyPrint = true
    ignoreUnknownKeys = false
}

val sensitiveMapper = Json {
    prettyPrint = true
    ignoreUnknownKeys = false
    serializersModule = SerializersModule {
        this.contextual(Email::class, Email.SensitiveSerializer)
    }
}
Copy code
val testMasking = QuoteRequestTeamB.QuoteV1(email = Email("<mailto:test@gmail.com|test@gmail.com>"))
// expected { "email": "<mailto:____@gmail.com|____@gmail.com>" }
sensitiveMapper.encodeToString(testMasking).also(::println)        
// expected { "email": "<mailto:test@gmail.com|test@gmail.com>" }
wireMapper.encodeToString(testMasking).also(::println)
If I replace the
@Serializable
with
@Serializable(with = Email.SensitiveSerializer)
, then it always masks and if I remove it, it never masks which mens the serializersModule is not working here. Any idea what I'm missing?
a

Adam S

01/29/2023, 7:46 PM
cool idea! it will probably work if you annotated the field with
@Contextual
https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serializers.md#contextual-serialization What you could also do is create a custom encoder. This won’t be too difficult - use StringFormat and as it’s an interface you can delegate to a
Json
instance. And then you just need to intercept any properties you want to mask. edit: Hmm maybe not StringFormat… I’ll have another look
j

janvladimirmostert

01/29/2023, 8:00 PM
doing
Copy code
@Serializable
    data class QuoteV1(
        @Contextual
        val email: Email
    ) {
        override fun toString(): String {
            return sensitiveMapper.encodeToString(this)
        }
    }
works for that one field, but this means all fields will need to be annotated with
@Contextual
which is exactly what I'm trying to avoid by using the value classes I guess it's a start, at least I know why it wasn't working
I see the docs has another way of doing this which is cool, but not 100% suited for my use-case
Copy code
typealias DateAsLong = @Serializable(DateAsLongSerializer::class) Date

typealias DateAsText = @Serializable(DateAsSimpleTextSerializer::class) Date
Copy code
@Serializable          
class ProgrammingLanguage(val stableReleaseDate: DateAsText, val lastReleaseTimestamp: DateAsLong)

fun main() {
    val format = SimpleDateFormat("yyyy-MM-ddX")
    val data = ProgrammingLanguage(format.parse("2016-02-15+00"), format.parse("2022-07-07+00"))
    println(Json.encodeToString(data))
}
a

Adam S

01/29/2023, 8:08 PM
typealias serialization doesn’t work yet :( https://github.com/Kotlin/kotlinx.serialization/issues/2083
try this! It detects if a specific
jsonMasked
Json encoder instance is used during encoding, and if so, then it masks the data
e

ephemient

01/29/2023, 10:47 PM
I think it would be less hackish to do something like this, which doesn't rely on any specific format instances
to be used like
Copy code
val jsonUnmasked = Json {
    ...
}
val jsonMasked = Json {
    serializersModule = SerializersModule {
        contextual(MaskedSerializer)
    }
}
of course you could flip the logic around if you wanted it to default to masked unless a specific contextual serializer is given
j

janvladimirmostert

01/29/2023, 11:02 PM
these are both pretty cool solutions, thanks!!
I'm already using a
Copy code
object RequestSerializer : JsonContentPolymorphicSerializer
which doesn't support overriding anything other than the deserializer seems like I'll need to write my own from scratch to support both polymorphic + masking serializing + deserializing
the source of JsonContentPolymorphicSerializer is extending KSerializer, so that should be a good place to dig
this is a little boilerplate-heavy, but it's self-contained and since we only have a few such fields, the "is masked encoder" hack works brilliantly
Copy code
@JvmInline
@Serializable(with = Email.SensitiveSerializer::class)
value class Email(override val value: String) : Sensitive {
    override fun mask(): String {
        return value.split("@").let {
            "_".repeat(it.first().length) + "@" + it.last()
        }
    }

    object SensitiveSerializer : KSerializer<Email> {
        override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Email", PrimitiveKind.STRING)
        override fun deserialize(decoder: Decoder): Email = Email(decoder.decodeString())
        override fun serialize(encoder: Encoder, email: Email) {
            if (encoder is JsonEncoder && encoder.json == sensitiveMapper) {
                encoder.encodeString(email.mask())
            } else {
                encoder.encodeString(email.value)
            }
        }
    }
}
e

ephemient

01/29/2023, 11:50 PM
I think that's a bad idea - you would expect
Json(from = json)
to behave like
json
, but it won't with that
j

janvladimirmostert

01/29/2023, 11:50 PM
you mean I should swop it that it defaults to masking unless I use a special mapper that doens't mask?
Copy code
override fun serialize(encoder: Encoder, value: Email) {
            if (encoder is JsonEncoder && encoder.json != wireMapper) {
                encoder.encodeString(value.mask())
            } else {
                encoder.encodeString(value.value)
            }
        }
Copy code
Json.encodeToString(myTestData).also(::println)
now masks as well which is perfect I'll build the correct serializer into the HTTP function so that it doesn't mask there
e

ephemient

01/29/2023, 11:54 PM
no. I mean that
.json ==
is bad
there's already a mechanism for customizing behavior, and it works even when the format instances are extended.
j

janvladimirmostert

01/29/2023, 11:56 PM
agreed your solution addresses that, but I need to experiment with your solution first to fully understand what it does
ah, ok, I see what yours is doing differently you pass in which contexts are available in the Json builder and then based on that figure out if it should mask or not Since I have multiple classes, I can't just have one SensitiveSerializer, but I can still pass that in as a context and use that as a flag. Sounds like a nice avenue for further experimentation
I've gone for a hybrid based on your idea
Copy code
@JvmInline
@Serializable(with = Email.SensitiveSerializer::class)
value class Email(override val value: String) : Sensitive {

    override fun mask(): String {
        return value.split("@").let {
            value.first() + "_".repeat(max(0, it.first().length - 1)) + "@" + it.last()
        }
    }

    @ExperimentalSerializationApi
    object SensitiveSerializer : KSerializer<Email> {
        override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Email", PrimitiveKind.STRING)
        override fun deserialize(decoder: Decoder): Email = Email(decoder.decodeString())
        override fun serialize(encoder: Encoder, value: Email) {
            encoder.serializersModule.getContextual(Sensitive::class)?.let {
                encoder.encodeString(value.mask())
            } ?: encoder.encodeString(value.value)
        }
    }

    override fun toString(): String {
        return mask()
    }

}
Copy code
val wireMapper = Json {
    prettyPrint = true
    ignoreUnknownKeys = false

}

val sensitiveMapper = Json {
    prettyPrint = true
    ignoreUnknownKeys = false
    serializersModule = SerializersModule {
        this.contextual(MaskedSerializer)
    }
}
thanks for the ideas @ephemient and @Adam S
p

pdvrieze

02/01/2023, 2:38 PM
A different approach (esp. if you have a deep hierarchy) is to have the following: • Have a custom format that wraps the original but adds a property that specifies the needed status. It forwards to the base format, but will wrap return values as needed • Have a custom serializer that uses
is
to determine whether the format is this special format (the same case you can special case the
Jon
format). It can then read the property and do what is appropriate. This technique could be used for any case that needs dynamic serialization. The advantage is that you don't need to rely on the right serializer being added to the context.
8 Views