I have implemented my own extensions for `PackageF...

# compiler

[raulraja]

I have implemented my own extensions for PackageFragmentProviderExtension and now I’m verifying that all the code is generated properly and I can even import classes and they compile fine from the command line. For example this compiles:

import sample.GeneratedClass

And also this:

Class.forName("sample.GeneratedClass")

I’m still seeing these issues:
1. This compiles fine only in the command line. IDEA still shows redlines for all the types it can’t find. Does IDEA use automatically the PackageFramentDescriptor and SyntheticResolver info or am I missing additional setup to get this to work on IDEA?
2. It only compiles fine with import sample.GeneratedClass if its in the same module. If I add the compilation result with codegen as dependency in a different module it does not compile but shows instead:

sample/test.kt: (3, 21): Unresolved reference: GeneratedClass

Any help is appreciated, thanks!

[raulraja]

Am I suppose to use this class in mi compiler plugin for the Synthetic descriptors I’m generating?

class SyntheticClassOrObjectDescriptor(
    c: LazyClassContext,
    parentClassOrObject: KtPureClassOrObject,
    containingDeclaration: DeclarationDescriptor,
    name: Name,
    source: SourceElement,
    outerScope: LexicalScope,
    private val modality: Modality,
    private val visibility: Visibility,
    override val annotations: Annotations,
    constructorVisibility: Visibility,
    private val kind: ClassKind,
    private val isCompanionObject: Boolean
)

If that is the case how can I obtain a KtPureClassOrObject that actually is the package (not a class)?. The GeneratedClass I’m adding goes top level into the package and it’s not a synthetic nested class.

[kralli]

Well, IDEA is where the fun starts. In order for your plugin to be displayed correctly in IDEA, you have to write an IDEA plugin. You can reuse the code you've wrote for the command line, in fact you have to register exactly those extension in the IDEA plugin. Thing is, the signatures of the ComponentRegistrar and some other extensions are different for these two. IDEA directly utilizes the com.intellij package, whereas the command line uses the those same classes but shadowed under the org.jetbrains.kotlin.com.intellij package. That means that you have to build two separate versions of your compiler plugin using the gradle shadow plugin, which is really inconvenient.

[raulraja]

Thanks @kralli, after I follow this process do I need to publish the plugin for it to be manually installed by a user? or does IDEA get that automatically when the user adds my plugin to it's Gradle or Maven config?

[kralli]

I'm not sure. I think they have to add it manually.

[kralli]

As for the SyntheticResolveExtension KtPureClassOrObject is only implemented by these classes. I don't think it is designed to create top-level classes.

[raulraja]

@kralli thanks for all your help

[raulraja]

I'll try to see if I can create a similar class from scratch with those features

[raulraja]

That works for package level classes

[raulraja]

I was actually expecting the synthetic resolver to inject metadata in the bytecode to help IDEA see the synthetic declarations

[kralli]

If nothing works, take a look at the ExpressionCodegenExtension. Within that extension you can get hold of the ExpressionCodegenExtension.Context. It contains the ExpressionCodegen and more importantly the GenerationState. That class is managing the classes that are getting generated via the ClassFileFactory. There should be a way of telling it to generate your class. Might not be the prettiest way, but it probably works.

[raulraja]

I'm using IR generation and the classes are generated and I can even instantiate them, the issue is that in different modules that are not where they are declared they don't compile

[raulraja]

When referenced via imports

[raulraja]

Additionally the show red lines in IDEA even in the same module where they are expanded

[raulraja]

Is the ExpressionCodegenExtension ment to be used in addition to the IR generation or the alternative if I don't use IR?

[kralli]

Yeah right, I forgot that you were aiming for IR. The ExpressionCodegenExpression is weirdly enough not declared under the java/jvm part of the compiler, but it definitely relies on jvm stuff and will not work with the IR backend.

[kralli]

As for the issue with the different module I can't help. I haven't tried that one out yet, as I'm currently not generating any additional classes.

[dmitriy.novozhilov]

Can someone confirm if this is case? What are other plugins like the serialization plugin or jetpack compose do in these cases?

Yes, it's true that you need install separate IDEA plugin to support your compiler plugin in IDE.
Serialization plugin for IDE is bundled into kotlin plugin. Plugin for Jetpack Compose most likely bundled into Android Studio

[kralli]

But I don’t think that it is going to stop really anyone from using the framework. It’s kinda like ProjektLombok. You must install a plugin manually for that too.

[raulraja]

Shouldn’t this be much easier?. If I create a Class the class is in the jar. Idea already parses .class content to determine structure. Isn’t the code I’m adding contributing to such structure? Where is the actual limitation in which we are forced to have users install plugins manually vs just letting IDEA read the generated synthetic methods? Same deal with the scala plugin back then, Libraries had to add themselves to the Jetbrains scala plugin for community support.

[dsavvinov]

Answering your question @raulraja:

If I create a Class the class is in the jar. Idea already parses .class content to determine structure. Isn’t the code I’m adding contributing to such structure?

That’s true only for binary dependencies, in which case it is reasonable, because binary .jars rarely change.

For source-code, IDEA doesn’t read produced jars (that would mean that after you’ve added several lines of code in your project, you have to recompile application completely), instead, it analyzes sources “on-the-fly”. Therefore, if some extensions (say, compiler plugins) add some declarations “on-the-fly” too, IDEA should be aware of it, and you’ll have to ship your own IDE plugin.

Yeah, that’s not an ideal world, and theoretically, nothing prevents from improving it, but it’s just another non-trivial slab of work we have to do 🙂

[dsavvinov]

And yes, small correction:

Are we expected to try to bundle our plugins inside the Kotlin plugin to take advantage of seamless adoption?

No, you can ship separate IDEA plugin, which should be installed alongside with Kotlin Plugin. That’s how some large frameworks which rely on code generation heavily do, AFAIK

[raulraja]

But the Kotlin plugin is bundled with IDEA. Having a user to go search and manually install a plugin is horrible in my opinion. This so that they can just get valid syntax highlighting of terms referencing valid generated bytecode. This state of things puts community libraries in disadvantage compare to big corporations that can just put a team of people to work on a compiler fork. I don’t mean to be upset at this but Kotlin metaprogramming is important for the community, developers should have equal opportunity in this field of compiler plugins in contrast with big players like Google if we are talking about an open source project like Kotlin.

[dsavvinov]

There’s some critical miunderstanding here. It’s not a compiler fork. It’s a usual IDEA Plugin, and people wrote a dozens of them long before Kotlin, and will write more for sure. The only difference is that it has to work in conjunction with Kotlin IDE Plugin, but you don’t have to maintain fork by whatever means

[raulraja]

I'd like to fix IDEA to automatically recognize generated members if that is something JB is open to so people developing compiler plugins don't need to force their users to install manually

[dsavvinov]

JetPack Compose uses compiler fork only for the sake of prototype. I’m pretty sure they are not going to maintain it for production, because yes, that introduced a lot of issues. In this sense, it’s just yet another compiler plugin, which has to somehow ship IDE support (it is true though that if you happen to have your own IDE, shipping your plugins is a bit easier), and it doesn’t have any exclusive rights here.

[dsavvinov]

I’d like to fix IDEA to automatically recognize generated members

Yes, that’s a very nice feature, and if it were easy, we would have implemented it long ago.

[Ryan Mentley]

Yes, we (Compose) really don't want to fork the compiler...it causes a ton of annoying issues for us, too.

[Ryan Mentley]

I don't have the link(s) handy but there are pull request(s) working on upstreaming our changes