I'm looking to mask certain values like `email` if...

# serialization

[janvladimirmostert]

I'm looking to mask certain values like email if it's serialized for the purpose of logging, but not if it's serialized for the purpose of sending it over the wire, so I'm looking for a way to swop out Serializers depending on context.

Example:

@JvmInline
@Serializable
value class Email(override val value: String) : Sensitive {
    override fun mask(): String {
        return value.split("@").let {
            "_".repeat(it.first().length) + "@" + it.last()
        }
    }
    object SensitiveSerializer : KSerializer<Email> {
        override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Email", PrimitiveKind.STRING)
        override fun deserialize(decoder: Decoder): Email = Email(decoder.decodeString())
        override fun serialize(encoder: Encoder, value: Email) = encoder.encodeString(value.mask())
    }
}

val wireMapper = Json {
    prettyPrint = true
    ignoreUnknownKeys = false
}

val sensitiveMapper = Json {
    prettyPrint = true
    ignoreUnknownKeys = false
    serializersModule = SerializersModule {
        this.contextual(Email::class, Email.SensitiveSerializer)
    }
}

val testMasking = QuoteRequestTeamB.QuoteV1(email = Email("<mailto:[e-mail]|[e-mail]>"))
// expected { "email": "<mailto:[e-mail]|[e-mail]>" }
sensitiveMapper.encodeToString(testMasking).also(::println)        
// expected { "email": "<mailto:[e-mail]|[e-mail]>" }
wireMapper.encodeToString(testMasking).also(::println)

If I replace the @Serializable with @Serializable(with = Email.SensitiveSerializer) , then it always masks and if I remove it, it never masks which mens the serializersModule is not working here.

Any idea what I'm missing?

[Adam S]

cool idea!

it will probably work if you annotated the field with @Contextual https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serializers.md#contextual-serialization

What you could also do is create a custom encoder. This won’t be too difficult - use StringFormat and as it’s an interface you can delegate to a Json instance. And then you just need to intercept any properties you want to mask.

edit: Hmm maybe not StringFormat… I’ll have another look

[janvladimirmostert]

doing

@Serializable
    data class QuoteV1(
        @Contextual
        val email: Email
    ) {
        override fun toString(): String {
            return sensitiveMapper.encodeToString(this)
        }
    }

works for that one field, but this means all fields will need to be annotated with @Contextual which is exactly what I'm trying to avoid by using the value classes
I guess it's a start, at least I know why it wasn't working

[janvladimirmostert]

I see the docs has another way of doing this which is cool, but not 100% suited for my use-case

typealias DateAsLong = @Serializable(DateAsLongSerializer::class) Date

typealias DateAsText = @Serializable(DateAsSimpleTextSerializer::class) Date

@Serializable          
class ProgrammingLanguage(val stableReleaseDate: DateAsText, val lastReleaseTimestamp: DateAsLong)

fun main() {
    val format = SimpleDateFormat("yyyy-MM-ddX")
    val data = ProgrammingLanguage(format.parse("2016-02-15+00"), format.parse("2022-07-07+00"))
    println(Json.encodeToString(data))
}

[Adam S]

typealias serialization doesn’t work yet :( https://github.com/Kotlin/kotlinx.serialization/issues/2083

[Adam S]

try this!

It detects if a specific jsonMasked Json encoder instance is used during encoding, and if so, then it masks the data

kxsMask.kt.cpp

[ephemient]

I think it would be less hackish to do something like this, which doesn't rely on any specific format instances

Untitled.kt

[ephemient]

to be used like

val jsonUnmasked = Json {
    ...
}
val jsonMasked = Json {
    serializersModule = SerializersModule {
        contextual(MaskedSerializer)
    }
}

of course you could flip the logic around if you wanted it to default to masked unless a specific contextual serializer is given

[janvladimirmostert]

these are both pretty cool solutions, thanks!!

[janvladimirmostert]

I'm already using a

object RequestSerializer : JsonContentPolymorphicSerializer

which doesn't support overriding anything other than the deserializer
seems like I'll need to write my own from scratch to support both polymorphic + masking serializing + deserializing

[janvladimirmostert]

the source of JsonContentPolymorphicSerializer is extending KSerializer, so that should be a good place to dig

[janvladimirmostert]

this is a little boilerplate-heavy, but it's self-contained and since we only have a few such fields, the "is masked encoder" hack works brilliantly

@JvmInline
@Serializable(with = Email.SensitiveSerializer::class)
value class Email(override val value: String) : Sensitive {
    override fun mask(): String {
        return value.split("@").let {
            "_".repeat(it.first().length) + "@" + it.last()
        }
    }

    object SensitiveSerializer : KSerializer<Email> {
        override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Email", PrimitiveKind.STRING)
        override fun deserialize(decoder: Decoder): Email = Email(decoder.decodeString())
        override fun serialize(encoder: Encoder, email: Email) {
            if (encoder is JsonEncoder && encoder.json == sensitiveMapper) {
                encoder.encodeString(email.mask())
            } else {
                encoder.encodeString(email.value)
            }
        }
    }
}

[ephemient]

I think that's a bad idea - you would expect Json(from = json) to behave like json, but it won't with that

[janvladimirmostert]

you mean I should swop it that it defaults to masking unless I use a special mapper that doens't mask?

[janvladimirmostert]

override fun serialize(encoder: Encoder, value: Email) {
            if (encoder is JsonEncoder && encoder.json != wireMapper) {
                encoder.encodeString(value.mask())
            } else {
                encoder.encodeString(value.value)
            }
        }

[janvladimirmostert]

Json.encodeToString(myTestData).also(::println)

now masks as well which is perfect

I'll build the correct serializer into the HTTP function so that it doesn't mask there

[ephemient]

no. I mean that .json == is bad

[ephemient]

there's already a mechanism for customizing behavior, and it works even when the format instances are extended.

[janvladimirmostert]

agreed
your solution addresses that, but I need to experiment with your solution first to fully understand what it does

[janvladimirmostert]

ah, ok, I see what yours is doing differently
you pass in which contexts are available in the Json builder and then based on that figure out if it should mask or not

Since I have multiple classes, I can't just have one SensitiveSerializer, but I can still pass that in as a context and use that as a flag.

Sounds like a nice avenue for further experimentation

[janvladimirmostert]

I've gone for a hybrid based on your idea

@JvmInline
@Serializable(with = Email.SensitiveSerializer::class)
value class Email(override val value: String) : Sensitive {

    override fun mask(): String {
        return value.split("@").let {
            value.first() + "_".repeat(max(0, it.first().length - 1)) + "@" + it.last()
        }
    }

    @ExperimentalSerializationApi
    object SensitiveSerializer : KSerializer<Email> {
        override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Email", PrimitiveKind.STRING)
        override fun deserialize(decoder: Decoder): Email = Email(decoder.decodeString())
        override fun serialize(encoder: Encoder, value: Email) {
            encoder.serializersModule.getContextual(Sensitive::class)?.let {
                encoder.encodeString(value.mask())
            } ?: encoder.encodeString(value.value)
        }
    }

    override fun toString(): String {
        return mask()
    }

}

[janvladimirmostert]

val wireMapper = Json {
    prettyPrint = true
    ignoreUnknownKeys = false

}

val sensitiveMapper = Json {
    prettyPrint = true
    ignoreUnknownKeys = false
    serializersModule = SerializersModule {
        this.contextual(MaskedSerializer)
    }
}

[janvladimirmostert]

thanks for the ideas @ephemient and @Adam S

[pdvrieze]

A different approach (esp. if you have a deep hierarchy) is to have the following:
• Have a custom format that wraps the original but adds a property that specifies the needed status. It forwards to the base format, but will wrap return values as needed
• Have a custom serializer that uses is to determine whether the format is this special format (the same case you can special case the Jon format). It can then read the property and do what is appropriate.
This technique could be used for any case that needs dynamic serialization. The advantage is that you don't need to rely on the right serializer being added to the context.