https://kotlinlang.org logo
Join Slack
Powered by
I'm currently working with the nextcloud api and h...
# serialization
m
I'm currently working with the nextcloud api and have some problems deserializing it. For example I get the following answer (stripped all unneeded stuff). So the problem is, that I get the 
ocs.data[*].groups
property as 
Map<String, Long>
, if there are some groups and permissions (The long) available or as empty Array, if nothing is set there. This problem I also have on the 
ocs.data
level on some other api endpoints. Kotlin: 1.8.20 kotlinx.serialization Plugin: 1.8.20 kotlinx.serialization: 1.5.0
Copy code
{
  "ocs": {
    "meta": { ... },
    "data": {
      "1": {
        "groups": [],
      },
      "2": {
        "groups": {
          "admin": 7
        },
      }
    }
  }
}
I tried implementing a 
TransformEmptyArrayAsObjectSerializer
like so:
Copy code
internal open class TransformEmptyArrayAsObjectSerializer<T: Any>(tSerializer: KSerializer<T>) : JsonTransformingSerializer<T>(tSerializer) {
    override fun transformDeserialize(element: JsonElement): JsonElement =
        if (element is JsonArray) buildJsonObject {  }
        else element
}
But after annotating the object with 
@Serializable(with = TransformEmptyArrayAsObjectSerializer::class)
there is a weird compiler error and IntelliJ is highlighting the line with 
Class 'TransformEmptyArrayAsObjectSerializer<*>', which is serializer for type 'Any', is applied here to type 'Map<String, Set<PermissionType>>?'. This may lead to errors or incorrect behavior
. Which I do not undestand correctly, but I think it's about type erasure? The data class definitions and the weird compile message can be found in the thread. Question: How can this be solved?
Copy code
@Serializable
internal data class Ocs<T: Any>(
    val ocs: OcsEntity<T>,
)

@Serializable
internal data class OcsEntity<T: Any>(
    val meta: MetaEntity,
    @Serializable(with = TransformEmptyArrayAsObjectSerializer::class)
    val data: T,
)

@Serializable
internal data class MetaEntity(
    @SerialName("status")
    val status: String,
    @SerialName("statuscode")
    val statusCode: Int,
    @SerialName("message")
    val message: String,
)

@Serializable
public data class GroupFolderDetails(
    @SerialName("id")
    val id: Int,
    @SerialName("mount_point")
    val mountPoint: String,
    // empty array "[]" or a map of groups with their permissions
    @SerialName("groups")
    @Serializable(with = TransformEmptyArrayAsObjectSerializer::class)
    val groups: Map<String, Set<PermissionType>>? = null,
    @SerialName("quota")
    val quota: Long,
    @SerialName("size")
    val size: Long, // needs 64-bit integer
    @SerialName("acl")
    val acl: Boolean,
    @SerialName("manage")
    val manage: List<GroupFolderManageDetails>? = null,
)
Tried to deserialize it as 
Ocs<Map<String, GroupFolderDetails>>
Copy code
e: org.jetbrains.kotlin.backend.common.BackendException: Backend Internal error: Exception during IR lowering
File being compiled: /home/maximilian/Development/iog/feather/nextcloud/src/main/kotlin/dev/maximilian/feather/nextcloud/groupfolders/entities/GroupFolderDetails.kt
The root cause java.lang.RuntimeException was thrown at: org.jetbrains.kotlin.backend.jvm.codegen.FunctionCodegen.generate(FunctionCodegen.kt:51)
...
Caused by: java.lang.RuntimeException: Exception while generating code for:
FUN CLASS_STATIC_INITIALIZER name:<clinit> visibility:public/*package*/ modality:FINAL <> () returnType:kotlin.Unit
  BLOCK_BODY
    SET_FIELD 'FIELD FIELD_FOR_OBJECT_INSTANCE name:Companion type:dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.Companion visibility:public [final,static]' type=kotlin.Unit origin=INITIALIZE_FIELD
      value: CONSTRUCTOR_CALL 'public constructor <init> ($constructor_marker: kotlin.jvm.internal.DefaultConstructorMarker?) declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.Companion' type=dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.Companion origin=null
        $constructor_marker: CONST Null type=kotlin.jvm.internal.DefaultConstructorMarker? value=null
    BLOCK type=kotlin.Unit origin=SYNTHESIZED_INIT_BLOCK
      SET_FIELD 'FIELD COMPANION_PROPERTY_BACKING_FIELD name:$childSerializers type:kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> visibility:private [final,static]' type=kotlin.Unit origin=null
        value: BLOCK type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
          VAR IR_TEMPORARY_VARIABLE name:tmp0 type:kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val]
            CALL 'public final fun arrayOfNulls <T> (size: <http://kotlin.Int|kotlin.Int>): kotlin.Array<T of kotlin.arrayOfNulls?> declared in kotlin' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
              <T>: kotlinx.serialization.KSerializer<kotlin.Any>
              size: CONST Int type=<http://kotlin.Int|kotlin.Int> value=7
          CALL 'public final fun set (index: <http://kotlin.Int|kotlin.Int>, value: T of kotlin.Array): kotlin.Unit [operator] declared in kotlin.Array' type=kotlin.Unit origin=null
            $this: GET_VAR 'val tmp0: kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val] declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.<clinit>' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
            index: CONST Int type=<http://kotlin.Int|kotlin.Int> value=0
            value: CONST Null type=kotlin.Nothing? value=null
          CALL 'public final fun set (index: <http://kotlin.Int|kotlin.Int>, value: T of kotlin.Array): kotlin.Unit [operator] declared in kotlin.Array' type=kotlin.Unit origin=null
            $this: GET_VAR 'val tmp0: kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val] declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.<clinit>' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
            index: CONST Int type=<http://kotlin.Int|kotlin.Int> value=1
            value: CONST Null type=kotlin.Nothing? value=null
          CALL 'public final fun set (index: <http://kotlin.Int|kotlin.Int>, value: T of kotlin.Array): kotlin.Unit [operator] declared in kotlin.Array' type=kotlin.Unit origin=null
            $this: GET_VAR 'val tmp0: kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val] declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.<clinit>' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
            index: CONST Int type=<http://kotlin.Int|kotlin.Int> value=2
            value: CONSTRUCTOR_CALL 'public constructor <init> (tSerializer: kotlinx.serialization.KSerializer<T of dev.maximilian.feather.nextcloud.TransformEmptyArrayAsObjectSerializer>) [primary] declared in dev.maximilian.feather.nextcloud.TransformEmptyArrayAsObjectSerializer' type=dev.maximilian.feather.nextcloud.TransformEmptyArrayAsObjectSerializer<kotlin.String> origin=null
              <class: T>: <none>
          CALL 'public final fun set (index: <http://kotlin.Int|kotlin.Int>, value: T of kotlin.Array): kotlin.Unit [operator] declared in kotlin.Array' type=kotlin.Unit origin=null
            $this: GET_VAR 'val tmp0: kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val] declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.<clinit>' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
            index: CONST Int type=<http://kotlin.Int|kotlin.Int> value=3
            value: CONST Null type=kotlin.Nothing? value=null
          CALL 'public final fun set (index: <http://kotlin.Int|kotlin.Int>, value: T of kotlin.Array): kotlin.Unit [operator] declared in kotlin.Array' type=kotlin.Unit origin=null
            $this: GET_VAR 'val tmp0: kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val] declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.<clinit>' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
            index: CONST Int type=<http://kotlin.Int|kotlin.Int> value=4
            value: CONST Null type=kotlin.Nothing? value=null
          CALL 'public final fun set (index: <http://kotlin.Int|kotlin.Int>, value: T of kotlin.Array): kotlin.Unit [operator] declared in kotlin.Array' type=kotlin.Unit origin=null
            $this: GET_VAR 'val tmp0: kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val] declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.<clinit>' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
            index: CONST Int type=<http://kotlin.Int|kotlin.Int> value=5
            value: CONST Null type=kotlin.Nothing? value=null
          CALL 'public final fun set (index: <http://kotlin.Int|kotlin.Int>, value: T of kotlin.Array): kotlin.Unit [operator] declared in kotlin.Array' type=kotlin.Unit origin=null
            $this: GET_VAR 'val tmp0: kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val] declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.<clinit>' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
            index: CONST Int type=<http://kotlin.Int|kotlin.Int> value=6
            value: CONSTRUCTOR_CALL 'public constructor <init> (element: kotlinx.serialization.KSerializer<E of kotlinx.serialization.internal.ArrayListSerializer>) [primary] declared in kotlinx.serialization.internal.ArrayListSerializer' type=kotlinx.serialization.internal.ArrayListSerializer<dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderManageDetails> origin=null
              <class: E>: <none>
              element: GET_FIELD 'FIELD FIELD_FOR_OBJECT_INSTANCE name:INSTANCE type:dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderManageDetails.$serializer visibility:public [final,static]' type=dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderManageDetails.$serializer origin=null
          GET_VAR 'val tmp0: kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> [val] declared in dev.maximilian.feather.nextcloud.groupfolders.entities.GroupFolderDetails.<clinit>' type=kotlin.Array<kotlinx.serialization.KSerializer<kotlin.Any>> origin=null
	at org.jetbrains.kotlin.backend.jvm.codegen.FunctionCodegen.generate(FunctionCodegen.kt:51)
a
in the example JSON you gave, 
ocs.data
is always an object, but in the 
OcsEntity
class you’ve set the 
data
property to use your custom serializer - that doesn’t seem right to me
m
This case also happens (always an object or an empty array for 
ocs.data
or 
ocs.data[*].groups
)
Copy code
{
  "ocs": {
    "meta": { ... },
    "data": []
  }
}
a
if you want to get something going quickly you could just use 
val data: JsonObject
, and then manually transform the data. Okay, sure, it’s nice to use a serializer and hide away the JSON logic, but sometimes it’s easier to decode it with Kotlin code.
ah okay, gotcha. Hmm…
m
Setting the 
groups
attribute from the 
GroupFolderDetails
a specific inherited serializer works. But this does not work on the 
ocs.data
level
Copy code
internal object GroupFolderGroupDetailsSerializer :
    TransformEmptyArrayAsObjectSerializer<Map<String, Set<PermissionType>>>(
        MapSerializer(
            String.serializer(),
            PermissionTypeSetSerializer
        )
    )

@Serializable
public data class GroupFolderDetails(
    ...,
    @SerialName("groups")
    @Serializable(with = GroupFolderGroupDetailsSerializer::class)
    val groups: Map<String, Set<PermissionType>>? = null,
    ...
)
Also found a workaround for the other case:
Copy code
internal object GroupFoldersAnswerEntitySerializer :
    TransformEmptyArrayAsObjectSerializer<Map<String, GroupFolderDetails>>(
        MapSerializer(
            String.serializer(),
            GroupFolderDetails.serializer(),
        ),
    )

@Serializable(with = GroupFoldersAnswerEntitySerializer::class)
internal sealed interface GroupFoldersAnswerEntity : Map<String, GroupFolderDetails>
and then deserializing it as 
Ocs<GroupFoldersAnswerEntity>
. But I would love to get a workaround, that is not that specific and generically useable.
a
I had to handle a similar problem where an empty array would be encoded to an empty JSON object. I solved it by using a value class with a custom serializer, and using that 
MyValueClass<T>
instead of 
List<T>
- I’ll se if I can come up with an example that works for your case
try this - I defined a value class 
NextCloudData<T>
that has a custom serializer 
NextCloudDataSerializer<T : Any>
. Value classes are nice in KxS because they can be equivalent to an existing type (in this instance a 
Map<String, T>
), but you can attach specific serialization logic to them. Instead of using the JSON Transform serializer I like implementing a KSerializer, because it gives a little more control over the logic. If 
NextCloudDataSerializer
is decoding JSON then it can first decode to the polymorphic 
JsonElement
type, and if it matches your requirements, then it will use a ‘delegate’ serializer (of type 
Map<String, T>
) to decode the value properly. In code you can use 
NextCloudData<T>
in any place where you would normally use a 
Map<String, T>
json.kt
m
My time ran out today. I will look into this later. Thank you for the ideas
This case also is possible:
Copy code
{
  "ocs": {
    "meta": { ... },
    "data": {
      "success": true     
    }
  }
}
Which correctly gets deserialized with
Copy code
@Serializable
internal data class SuccessResponse(
    val success: Boolean,
)
as 
Ocs<SuccessResponse>
So only if I expect a 
Map<String, T>
, then there is this weird behavior of 
[]
or a normal encoded Map as json object. (Possibly there are some other weird nextcloud edge cases, but I currently discovered them not until yet).
e
I don't understand why they would do that…
I've seen APIs that return 
""
for empty, which is equally annoying to deal with in a strictly typed language but kinda makes some sense in JS (
Boolean(null) == Boolean("") == false
, 
Boolean([]) == Boolean({}) == true
)
in any case, if that's something they always do, I think it would be better to handle it in the format rather than in individual serializers
unfortunately it's not really something easily customized in kotlinx.serialization now
but I have made a few things like https://kotlinlang.slack.com/archives/C7A1U5PTM/p1653151131184099?thread_ts=1653099160.150059&amp;cid=C7A1U5PTM which show that it's possible, with effort
4 Views
Open in Slack
PreviousNext
Powered by