Can anyone here tell me how significant the binary...
# arrow
s
Can anyone here tell me how significant the binary compatibility issues are for
NonEmptyList
(using
NonEmptyListSerializer
) between Arrow v1 and v2? My company (Cash App/Block) has some data serialized with the
NonEmptyListSerializer
of Arrow v1, but we need to upgrade to Arrow v2. Is it possible to define a serializer that can deserialize both the v1 and v2 binary formats? 🤔
From the Arrow 2.0 Release blog:
we had to break binary compatibility in several places to implement improvements, such as in
NonEmptyList
y
I don't have the source in front of me right now, but I'd guess it's likely fine? If not though, you could write a program to deserialize it, convert it to
List
, serialize it again, deserialize in 2.0, and finally convert it to
NonEmptyList
and serialize
p
we had to break binary compatibility in several places to implement improvements, such as in
NonEmptyList
this is referencing the binary compatibility of the arrow-core library itself, not the kotlinx-serialization serialized content (assuming that's what you mean by serialized) The incompatibility would arise should you try and execute a binary/jar compiled against arrow v1 with arrow v2 on the classpath
FYI
NonEmptyListSerializer
is a simple wrapper around a delegated
ListSerializer
with the additional conversion to
NonEmptyList
using
.toNonEmptyListOrNull() ?: throw SerializationException("expected non-empty list")
when deserializing
âž• 1
u
We did run into some serious issues, especially in combination with Spring "magic" – e.g., when using a
NonEmptyList
for API parameters or similar data structures where Spring attempts to create proxies.
p
Yes, this is likely due to the change to using a value class, and a kotlin compiler bug that leaves type-args in when it shouldn't (which impacts proxies etc)
a
kodee happy 2
in this case I think that: • binary compatibility is definitely not there, since the underlying format of
NonEmptyList
has changed • on the other hand, serializing in v1 and deserializing in v2 should work fine (or even the other way around), since both turn into the same thing