Do compiler plugins have to create code/implementa...

# compiler

[Marc Knaup]

Do compiler plugins have to create code/implementations “in-memory” (like Kotlin Serialization does) or can they also generate .kt files as you would with annotation processing?

[tynn]

Technically KAPT is a compiler plugin. Thus it should be possible to also create sources. https://kotlinlang.org/docs/reference/kapt.html#annotation-processing-with-kotlin

[Marc Knaup]

Yeah, but kapt always was some kind of hack and only works with Kotlin/JVM w/ Java sources. I wonder if there is a “legal” and supported way of generating source files as part of the compilation process. That should also work multiplatform.

[gildor]

See KSP, it's a compiler plugin which provides api to generate Kotlin sources

[Marc Knaup]

“IDE integration: Currently IDEs know nothing about the generated code.”
Doesn’t seem to be too standard 😅
But I guess that could be worked around with a Gradle plugin that adds a generated sources folder somehow 🤔

[gildor]

There is no standard

[gildor]

But KSP will become standard de facto, just because it's developed by Google and planned to be used for their own annotation processors

[Marc Knaup]

Sounds good though :)

[Marc Knaup]

I write so much repetitive code it’s annoying 😮

[gildor]

Generate it and add to a project as simple source files, it's probably the easiest solution for now

[Marc Knaup]

It’s gonna be a framework and other framework users will face the same issue. So I’d like to make it part of the framework.

[Marc Knaup]

Similar to JSON codec generation but for BSON and GraphQL.

[gildor]

Json codec? You mean kotlinx serialization?

[Marc Knaup]

Yeah, although my JSON library still uses kapt 😮

[gildor]

Graphql looks as a case for simple code generator, without annotation processing

[gildor]

And yes, Gradle plugin can help with it, for example this what SQLDelight is doing

[Marc Knaup]

GraphQL is anything but simple 😅

[gildor]

I didn't say that it simple, I said that code generation is probably enough, you don't need annotation processing for this

[Marc Knaup]

I do need some additional input. For example field descriptions and directives.

[gildor]

Because you can generate it from some scheme, not from existing code

[Marc Knaup]

No, I want it the other way round. GraphQL schema from code.

[gildor]

For documentation generation? I would rather generate code from scheme, but probably you have cases for this.

Yeah, for this you need full blown annotation processing (or some ast reader for Kotlin code)

[Marc Knaup]

The data model & extensions in Kotlin define the shape of the GraphQL schema. And that’s in a modular way. 20 different feature modules add different types and properties that may or may not be represented in the overall GraphQL schema.

[gildor]

But why ksp is no t working for you? What kind code should be available in ide? Isn't scheme is result of generation?

[Marc Knaup]

The GraphQL execution phase needs all information about the relation of types, properties, how to resolve abstract types and how to resolve field values.
That’s a lot of Kotlin DSL right now.

[gildor]

Yes, I understand, but this what ksp provides you, abstraction to register code visitors and emit some sorce from it

[gildor]

Or alternatively use compiler plugins api directly

[Marc Knaup]

Yup. I’ll probably give it a go, starting with BSON codecs. They are simpler and needed for easy MongoDB usage 🙂

[gildor]

Why do not use kotlinx.serialization for bson?

[Marc Knaup]

I don’t like it. It doesn’t generate code but just provides a non-debuggable, non-copyable, non-inspectable implementation.
Also it has limitations like not supporting inline classes.

[Marc Knaup]

I also need to stay compatible with MongoDB’s “CodecRegistry” approach which is kinda annoying. No serialization library knows about these registries.

[gildor]

I would say it's good that it doesn't generate code, less things to compile.
Same way as I don't see source code of big part of compiler and I'm fine with it, I have byte code if I need it

[gildor]

I think it's waste of time and effort to write custom serialization for this case

[Marc Knaup]

I’m working much more efficiently with generated JSON code and don’t mind the compile time. That’s what caching/incremental is for.

I had lots of issues debugging kotlinx-serialization issues as I cannot step through code to understand runtime exceptions. I cannot read and understand the code the plugin generates. I cannot easily create a custom serializer by copying a generated serializer and just modifying it slightly - I have to start from zero every time. It’s very difficult to comprehend what happens in general.

The compiler code itself isn’t a good comparison. A good comparison is a closed-source library. It’s difficult to work with these as they’re difficult to understand, debug and modify.

[gildor]

Copy serializer and modify doesn't look as something what I would expect from serialization library

[gildor]

I don't see why compiler code is not a good comparison, it's exactly what compiler does, better error reporting may be a solution

[Marc Knaup]

If I have a serialization library and have a large class to serialize I want to automate part of it (or everything at first) and be able to modify behavior as needed, for example for needed migration.
In case of kotlinx-serialization I have to write a new serializer from scratch, which is a lot of work.

The compiler compiles code. It doesn’t provide implementations for anything except some intrinsics. Even almost the entire stdlib is available in source. You can always see what happens behind the declaration or interface.