Is there any support for creating `proto3` formatted Schemas through kotlinx-serialization-protobuf?...

# serialization

[jmillner_]

Is there any support for creating proto3 formatted Schemas through kotlinx-serialization-protobuf? I have been able to get a proto2 version via the experimental ProtoBufSchemaGenerator

[jmillner_]

@glureau I spotted your post above 😄

[jmillner_]

Ah if I could spiderman gif I would

[glureau]

I'm working on the same problem, did some changes on this file, so you may be interested to have a look (gist) :
• syntax line is updated
• value class is supported (afaik the serialization consider the internal type directly, so I used the same approach on the schema, not sure about that)
• the "required"/"optional" strings are not printed in proto3
Things I'm missing right now:
• nested message support (not proto3 specific but required for my project)
• checking enum has a '0' entry (if all entries have a SerialNumber, we'll need a warning at least)
• same with a check on 'packed' for all lists
This is pretty recent, if something better exists I'm all in, if not I may push a few PR with those changes when the work is ready for production.

[jmillner_]

Definitely, thanks @glureau - I'll take a look!

[jmillner_]

I'm not very familiar with Protobuf at all, but great to see it in KotlinX serialization already.

Doesnt seem to be many people shouting about it, so probably not too burning - but as far as I'm aware proto 3 has been out for some time?

[jmillner_]

@glureau Just in my limited playing I was able to validate that your implementation seemingly was able to successfully create a proto 3 with a data class I'm fumbling around with in my own project: https://gist.github.com/james-millner/f6ce9084c639f2e73cea6985099de00c

The aspect you mentioned though RE nested message support, can you elaborate abit further?

Do you mean nested seperate @Serializable objects?

[jmillner_]

I have some child objects on this gist, equally serializable and it seems to be happy - against with my limited understanding.

I'm going to have a play with protoc to generate some other code, and compare the outcome just to see if it's compatible. On the surface it seems like it is

[glureau]

data class Foo(foo: Int) { data class Bar(bar:Int) }

In this configuration, the protobuf file will contain

message Foo { int32 foo = 1; }

message Bar { int32 bar = 1; }

Now let's say there is another kotlin file:

data class Foo2(foo: Int) { data class Bar(baz:Int) }

Here in Kotlin we will have 2 different nested classes: Foo.Bar and Foo2.Bar, the qualified names are different, but on the protobuf, both will be named "Bar". And since there's no namespacing for each message, I think it'll create a conflict at some point. What I'd like is to get a nested message on the protobuf file, to avoid such conflicts.

message Foo { int32 foo = 1; message Bar { int32 bar = 1; } }

[glureau]

In my previous experimentation I used the generation for 80 different classes that contains shared classes but also sometimes define nested class that can have the same name than another nested class. I was doing the schema generation for each class independently, then trying to reorganize those files with a script, and there I noticed that if I cut original files in multiple proto files (one by message) then I'll have to make a conflict resolution on my own, that's not what I expected. I'll try to use the generator of the 80 class at the same time just to check the (1-file) output, if ever it handles such cases.

[glureau]

I've just confirmed, if I use all descriptors, then only the last nested class is used, others are just skipped. Working on a new version right now.

[glureau]

Hi James, I failed my first attempt due to not be able to detect the nesting, but eventually AdamS mentionned the capturedKClass could help detect that. I'm really not a fan of the architecture of the generator rn since it has not been designed for nesting from my point of view, so it may take some time to do it right. Unfortunately for you, I'm going in holidays right now and for 2 weeks, but maybe I'll give another shot in 3 weeks to address that, we'll see. Don't hesitate to give it a try if ever you need it 😉

[jmillner_]

Not to worry @glureau thats fine, have a great holiday!

I'll give your implementation a try in my library project if thats OK, as I think as it stands it generates valid proto3 syntax for my simple data type - at least in its current form. I dont have any nested data classes, just one data class that has relationships with other data classes.