Hey all! I'm really excited about the prospect of ...

# arrow-meta

[Nathan Bedell]

Hey all! I'm really excited about the prospect of using Arrow meta to write compiler plugins for Kotlin. To get started learning arrow-meta, I tried writing a simple compiler plugin recently to try to add a simple Haskell-like "deriving" mechanism that adds statements to an existing class body to automatically implement a particular interface for data classes.

I'm using kotlin-poet to generate the new statements, and the code I'm using looks like (some sections elided for brevity):

/** Kotlin compiler plugin to automatically derive a Buildable instance for any class
 * annotated Buildable. */
val Meta.genBuildable: CliPlugin get() =
    "GenBuildable" {
        meta(
            classDeclaration(
                ctx = this,
                match = {
                    // ...
                },
                map = { (c, d) ->
                    val dataClass = classAsDataClass(c)!!
                    Transform.replace(
                        replacing = c,
                        newDeclaration = c
                            .text.`class`.syntheticScope.value!!.also { it ->
                                it.addDeclaration(
                                    generatePartialClass(dataClass)
                                        .toString()
                                        .declaration<KtDeclaration>()
                                        .value!!
                                )
                                it.addDeclaration(
                                    generateCtx(dataClass)
                                        .toString()
                                        .declaration<KtDeclaration>()
                                        .value!!
                                )
                                it
                            }.text
                            .`class`
                    )
                }
            )
        )
    }

I've tried this, as well as a few similar things. However, every time I try to run my plugin, I get the error:

java.lang.IllegalArgumentException: Missing extension point: org.jetbrains.kotlin.com.intellij.treeCopyHandler in container org.jetbrains.kotlin.com.intellij.core.CoreApplicationEnvironment$1@757c71a1
	at org.jetbrains.kotlin.com.intellij.openapi.extensions.impl.ExtensionsAreaImpl.getExtensionPoint(ExtensionsAreaImpl.java:260)

I didn't see any examples like what I'm trying to do in the arrow-meta-examples, so I was wondering if someone could point me in the right direction with this. I'm planning on making this an open-source project when finished, so I'd be happy to provide the full source code if someone wants to take a closer look -- but any help would be much appreciated!

[raulraja]

Hi Nathan, that error usually shows up when you use an underlying compiler API that is ment to be just for the IDE. Additionally the entire frontend in the Kotlin compiler is currently changing to FIR and at some point this API with quotes and transforms may not be available which means this whole thing would need to be redone to adapt to FIR objects including the underlying Arrow Meta quote system.

If you want to describe what your plugin does in more detail I can probably recommend an approach in how I’d implement it today given the current state of Compiler APIs and Meta.

Using Transform.replace has the disadvantage that IDEA won’t see your generated members and all the code would be redlined in the IDE but still works once its finished, not a good experience to users. This hopefully changes when FIR is introduced.

If you want to provide ad-hoc implementations of interfaces it’s usually easier to provide the implementations and factories that yield those implementations in the Backend IR phase.

The challenge with something like deriving in Kotlin data classes is that most of the derivable type classes like Show, Eq etc already have corresponding members that will get called as special operators like (EQEQ) that are already emitted based on the structure by data classes.

[Nathan Bedell]

Hi Raul, thanks for your response.

I actually have a gist here that shows exactly the interfaces that I want to be able to "derive" for this particular project: https://gist.github.com/Sintrastes/83d2fa5a918e4e4923dc8fa126346fdf

So, basically the goal is, given a declaration like:

@DeriveBuildable
data class MyData(
    val arg1: String
    val arg2: Int?
)

I want to transform the AST (or, as you say, it may be better to do this in the backend IR phase) so that MyData has a class body implementing the Buildable interface, as well as the two inner interfaces of Buildable.

The motivation for this is to automatically derive a "Partial" version of a data type, where the fields may be null, with a monoid operation for combining these partial data types, together with a build operation that attempts to turn a partial data type into a non-partial version, if all of the relevant fields are non-null.

[raulraja]

Depends on how you want to implement this you can do it with Transform.newSources or with an IR transformation.
If we follow the model in your gist then you are just generating a type class based on the class structure. You could just source code generation in this case if you are gonna consume it ad-hoc like that and what you declare is also seen in the same module in resolution. You’d be able to refer to the symbols just like in your gist.
If instead you want MyData to directly implement Buildable<MyData> to be consumed in a different module from where its compiled you can use irClass and mutate the tree before codegen.

Once you decide how to proceed if you want to share a repo where this is happening we can look into it and help you with any apis for codegen or IR 🙂

[raulraja]

Additionally we have the Given context plugin in meta that is not yet advertised because still has a couple of rough edges. That plugin will include implicit derivation for data and sealed classes. Currently it support semi inductive derivations of the parts and injection for type classes (DI) which may cover partially what you are doing here. More info https://github.com/arrow-kt/arrow-meta/tree/bafc829be3/proofs-plugin/src/test/kotlin/arrow/meta/plugins/proofs

[Nathan Bedell]

Wow! Thanks for the info regarding the Given plugin. That's going to be a game-changer one you all are able to polish it up. I'm glad the arrow team is still working towards the goal of true typeclasses in Kotlin even after KEEP-87 was closed.

While I originally envisioned MyData implementing Buildable<MyData> directly, I think now that I think about it, generating a typeclass for it would be just as work-able. So I think for now I will try the Transform.newSources approach instead, and see how that goes.

[raulraja]

cool, let us know how it goes. We’ve been hesitant regarding the implementations of things that are derivable because presumably tuples may come back to Kotlin. If they do there may be a chance that data classes and other tuple like structures have a common parent and in that case, if you can go from you data class to TupleN you can probably derive all this automatically. You can potentially already do what you are doing or similar with the Serialization plugin since it gives you a descriptor with structures about the type and values

[Nathan Bedell]

Interesting, I definitely will. In another project (actually, one of the things I'm hoping of using this Buildable interface to make a bit nicer) I had to write my own DataN interfaces (with operator fun componentN() methods), and a lot of boilerplate duplicated to make functions working on Data2, Data3 ... DataN.

So if Kotlin brought back tuples, that'd certainly be helpful as well!

[Nathan Bedell]

@raulraja -- You mentioned before that Transform.replace has the disadvantage of showing up red-lined in the IDE. I moved to using Transform.newSources in my plugin as suggested -- and I now have the plugin working the way that I want, but the generated extension methods are still showing up as red-lined in the IDE.

Is this a known issue, or am I doing something wrong here?

https://github.com/Sintrastes/buildable-kt

[raulraja]

Hi @Nathan Bedell currently sources are placed in the kapt like folder. If you are not using the kapt plugin its probably not adding that folder automatically as source root. We plan on changing this in meta so instead it places the sources in a meta folder and our gradle plugin would automatically add to the source set. If you want to add the source sets manually for the time being https://youtrack.jetbrains.com/issue/KT-16874 or similar allow you through gradle add those as generate sources.