Is anyone, community or JB, working on concerted, focused efforts to make reflectionless metaprogram...

# kapt

[Derek Berner]

Is anyone, community or JB, working on concerted, focused efforts to make reflectionless metaprogramming in Kotlin more accessible to the masses?

[simon.vergauwen]

We’re doing a lot of effort in #arrow-meta where we’ve recently changed our focus on compiler plugins instead of kapt.

[Derek Berner]

I have this extremely strong hunch that full-featured reflectionless metaprogramming would be a huge win for Kotlin.

[Derek Berner]

Right now I'm just relying on string builders knowing that the generated code will be compile-time checked.

[simon.vergauwen]

That’s what we currently do in Arrow in most places. Compiler plugins is the true solution here and we’ve made a lot of progress using them. Only the API is unstable which could be a huge pain but we’re trying to solve that with a nice decoupled wrapper, and there is currently 0 docs which is also painful but only in the wrapper code.

[Derek Berner]

Are compiler plugins new?

[Derek Berner]

Sorry, should read to the end of your comment first! 😅

[simon.vergauwen]

Not new, not official supported yet. Google is using it for Jetpack Compose. Jetbrains uses it for kotlinx.serialization, Android extensions, etc

[simon.vergauwen]

So there’re stable libraries built on top of it already

[Derek Berner]

Yeah, the nice thing is the generated code is all that matters, so the tech itself can be unstable.

[Derek Berner]

Well, not exactly that but you see the point I'm getting at.

[Derek Berner]

Are there resources that talk about this?

[simon.vergauwen]

That’s what we figured 2 😄 I was once more amazed by the progress we’ve made so definitely interested to check it out.

[Derek Berner]

I'll lurk #arrow-meta and start googling for kotlin compiler plugins

[simon.vergauwen]

We’ve tried to collect resources here. https://github.com/47deg/arrow-meta-prototype

[Derek Berner]

👍

[simon.vergauwen]

Most of what you can find with Google is linked in the main README.MD. I’ve already spent a couple of nights Google resources 😄

[Derek Berner]

I've wasted like 2 days on trying to kapt my string-builder metaprogramming, and it's wonky to say the least

[Derek Berner]

I'm glad Kotlin has a viable alternative, and sad to hear that it's not well known.

[Derek Berner]

Anyway doing a CD deploy and I think it's finishing up so gotta go verify.

[simon.vergauwen]

It’s not officially supported but they “”"promised”“” a official solution 2 KotlinConfs in a row. So 🤞that we can call it the official solution soon. It’s the only way to do metaprogramming with MPP support anyway.

[gildor]

What is your exact problem with Kapt?
For author of annotation processor it's not different from using Java APT, which is pretty well documented. Kapt just allows to use APT API for Kotlin code, there are also libraries like KotlinPoet which simplifies Kotlin code generation itself

[simon.vergauwen]

APT API for Kotlin means that you cannot inspect Kotlin specific things. Like is it a sealed or data class?
Figuring this out requires you to go through @Metadata, which has a horrendous and undocumented API.
Additionally it doesn’t work for MPP, so it’s only an option if you do JVM only development with Kotlin and have no plans of changing that in the future.

[simon.vergauwen]

Also the information you can get from apt is quite limited, and I’ve had to rely on string manipulation a lot to access simple info like generics.

[simon.vergauwen]

To figuring out some certain information for Arrow we had to rely on using reflection etc and that is probably what Derek meant with “reflectionless metaprogramming”.

[simon.vergauwen]

APT codegen is also very limited in what you can do compared to meta solutions in other languages. i.e. code expansion to add companions like Parcelize and kotlinx.serialization do.

[simon.vergauwen]

Sorry, kapt has given me some painful times 😅

[Derek Berner]

It bums me out that the de facto metaprogramming facility in Kotlin is reflection

[simon.vergauwen]

Huh?

[Derek Berner]

Basically you have two options when doing metaprogramming, codegen or reflection

[Derek Berner]

Reflection occurs at runtime, is expensive, and can't be compile-time checked.

[Derek Berner]

But because Kotlin is a JVM language, and therefore gets access to the Java Reflection library for free, it is the de facto tool for doing metaprogramming. And I just think Kotlin can and should do better.

[simon.vergauwen]

Yes, but kapt is not reflection.

[Derek Berner]

Right. I'm saying reflection is currently the de facto solution

[simon.vergauwen]

Given that no MPP Kotlin solution is available?

[Derek Berner]

Well that and kapt is poorly documented, and I hadn't even heard of compiler plugins until today. The docs are all "kapt is what makes Dagger work" and not "this is how you use kapt to write your own processor"

[Derek Berner]

But as long as the Kotlin ecosystem is piggybacking on reflection-heavy utilities it's going to be limiting itself.

[simon.vergauwen]

Ah yes, similar issue in Java with APT. Or at least it felt like that when I started out with it.

[simon.vergauwen]

Yes they're but I don't think they will. It's too limiting, especially in the world of all the hot new modern langs.

[Derek Berner]

Java reflection is deep, but it's also easy to get started with. And so you have "clever" junior devs writing ten thousand line reflection tools that wind up buggy and hard to support. There should be more push to provide that same ease of entry for static metaprogramming solutions.

[jw]

Most of those things you mention can't be done in annotation processing are available with the kotlinx-metadata-jvm library

[simon.vergauwen]

How is kotlinx-metadata different from annotation processing? It's still annotation processing but it just provides an api to read Kotlin metadata as flags instead of deserializing the protobuf myself.

[simon.vergauwen]

Nevermind, just read the whole README. Wasn't aware of this somehow. How come this not promoted more? I completely missed this and apparently all Arrow maintainers 2.

[simon.vergauwen]

Still doesn't support the use cases we need for Arrow and we can achieve with plugins with MPP support.

[jw]

Right so it's an API for reading the compiler metadata in a more user-friendly way. It's not specific to annotation processing, but annotation processing is one context in which you can gain access to the annotations to read that information.

[Derek Berner]

kotlinx-metadata-jvm isn't super well documented. I had to really focus my google-fu to find it, and it's pretty alpha ATM. It makes doing kapt with Kotlin language features possible with effort, which is a step in the right direction but not a complete solution yet

[Derek Berner]

And as @simon.vergauwen pointed out still isn't a MPP solution

[Derek Berner]

And I get not wanting to call attention to the dragons, it hurts adoption. But some of us really want to slay dragons to help make this language better

[simon.vergauwen]

In Arrow we’ve used this lib https://github.com/Takhion/kotlin-metadata to read Kotlin metadata since before kotlinx-metadata-jvm existed I think.

It comes with all issues I mentioned above but cannot confirm if kotlinx-metadata-jvm solves some of the pains we’ve faced…

[simon.vergauwen]

It makes doing kapt with Kotlin language features possible with effort

But it’s still far from a proper meta solution IMO. APT/KAPT are limited in what they can do since you cannot influence any existing code.

[Derek Berner]

Can't get back to the .kt file in kapt even using kotlinx-metadata-jvm

[jw]

Well sure, because it's not guaranteed to be written in Kotlin or even compiled from a source language.