Hey, I’m a little new with compiler plugins, but w...

# k2-adopters

[Oleg Yukhnevich]

Hey, I’m a little new with compiler plugins, but will it be possible with K2 compiler plugin to transform:

@Ann
external fun doSomething() // in common

into

expect fun doSomething() // in common
actual fun doSomething() {...} // in jvm
actual fun doSomething() {...} // in native

Overall how it will be done under the hood doesn’t matter, the idea is to generate different implementations for different targets, while still having common function
is it possible ? 🙂
as for use case - “multiplatform cinterop” - https://github.com/whyoleg/ffi-kotlin/tree/gradle-plugin#compiler-plugin

[Oleg Yukhnevich]

I found, that it’s possible to change status (expect/actual, external) and generate declarations
Question is mostly about: is it possible to generate different platform declarations?

[dmitriy.novozhilov]

Potentially it's possible, but I never tested such scenario

[Oleg Yukhnevich]

Ok, thx!
At least it’s feasible to try!
Will post here with results in coming days

[Oleg Yukhnevich]

and here are some first results (code: https://github.com/whyoleg/ffi-kotlin/commit/17408ae927c9330021bfb026fd887001888f1f9b)
• I was able to compile function with annotation and without body by making it expect in common and generation of actual in platform (source code of FIR extensions)
• It’s super hacky now, just to try, and Im not sure, that it’s ok to access common session like this: session._moduleData_.dependsOnDependencies._firstOrNull_()?.session . Of course if there will be more shared sourcesets I will need to iterate over all of them, but may be somewhere exists other way to access declarations from other sessions? In my case I need to get declaration annotated in common session, but use it in platform session to generate declaration based on it. I haven’t found any example of something similar in kotlin repository 🙂 All plugins there looks like just use single session, and it’s ok for them, but not in my case.
• I was even able to then generate simple platform dependent IR body for this function . (the simplest part here, hehe)
• But… It runs only on JVM 😞 for JS and Native it fails trying to serialize IR output to klib. F.e. JS fails on this line. Native fails on line with same semantic in different file. Any thoughts, on where I should look to understand, what I’m doing wrong?
And BTW, looking at fir-plugin-prototype module is super helpful to understand how to do something (both FIR and IR). Thx!

[dmitriy.novozhilov]

It’s super hacky now, just to try, and Im not sure, that it’s ok to access common session like this

Actually, it's quite correct, I don't see any issues related to usages of compiler structures

F.e. JS fails on this line.

Could you please report an issue about it? Looks like we never tested compiler plugins which generate new top-level declarations on platforms which differ from JVM

[dmitriy.novozhilov]

And BTW, looking at fir-plugin-prototype module is super helpful to understand how to do something (both FIR and IR)

The original purpose of this module is having some sandbox for testing all FIR extension points. Good to know that it also helps to understand compiler API
IMO this module is quite a mess, but actually it can be transformed into some sample compiler plugin project

[Oleg Yukhnevich]

Actually, it’s quite correct

Good to know this!

Could you please report an issue about it

Yes, will create it. Will it be enough to create 1 issue for K/JS and K/N? (stacktrace is different, but check is the same)

Good to know that it also helps to understand compiler API

It’s super helpful, as all other compiler plugins in the repository or super simple (noarg, allopen) or super complex (serialization). But this module provides short intro into most of the available extensions with simple code samples for both FIR and IR! It’s really great to understand the basics. For me it’s second time Im trying to do IR plugin and first time with FIR, and it was smooth!

While we are here, I have 2 small questions:
• I saw in fir-plugins doc, that there is no yet possibility to generate bodies for existing function. But may be there is some possibility now to just add some stub to body in FIR?
◦ When generating new actual declarations I also don’t add body, but it works ok
◦ Now, Im not able to expand @ForeignCCall fun doSmth(): String into full blown declaration in platform code but only create expect/actual. So strange, but it’s possible now only to declare it in common and actualize in platform, but not just declare in platform 😞
• Is there some deterministic possibility to understand is session is from shared sourceset or from leaf platform sourceset in FIR?
◦ It’s needed for same thing as upper point. while now I can just check, that dependsOnDependencies is empty (common) but if there will be hierarchy it will be not possible, as intermidiate sourceset (like native) will have dependsOn relations. I can’t also use TargetPlatform from FirModuleData as looks like now (or forever) it is the same for all sessions created in single compilation.

[dmitriy.novozhilov]

Will it be enough to create 1 issue for K/JS and K/N?

Yeah, it will be
It would be quite useful if you point out both problematic places in the description

I saw in fir-plugins doc, that there is no yet possibility to generate bodies for existing function. But may be there is some possibility now to just add some stub to body in FIR?

There is no difference in stub body and lack of body for generated declarations
If declaration has no body then IrFunction.body just will be empty, and you will be able to fill it like you want
If it will have some stub body, then fir2ir generate some stub ir body, which you'll also replace with something meaningful
So there is no sense in stub bodies

Now, Im not able to expand @ForeignCCall fun doSmth(): String into full blown declaration in platform code but only create expect/actual. So strange, but it’s possible now only to declare it in common and actualize in platform, but not just declare in platform

Could you please elaborate? I didn't fully understand the case

Is there some deterministic possibility to understand is session is from shared sourceset or from leaf platform sourceset in FIR?

There is no simple way to do it, because of following reasons:
1. there is no need in such API in compiler right now
2. scheme of compilation is subject to change (we plan to separately compile common modules to klib once and them use them as regular binary dependencies, but ETA of this is unknown)
3. layout of module data may be quite different in IDE (because it contains information about all modules in project), so there might be some inconsistencies (I'm not sure)

I can’t also use TargetPlatform from FirModuleData as looks like now (or forever) it is the same for all sessions created in single compilation.

This is by design of how MPP modules are compiled right now

[dmitriy.novozhilov]

There is one way how you can access the whole modules structure: there is a HmppModuleStucture class which contains layout of modules passed from build system to cli compiler
It is stored inside CompilerConfiguration and you can access it from CompilerPluginRegistrar
Names for module datas are matched 1 to 1 to names from HmppModuleStucture, so you can use it

But note that it won't work in IDE, because CompilerPluginRegistrar is not called during setup of plugins inside it

[Oleg Yukhnevich]

It would be quite useful if you point out both problematic places in the description

got it, will do later today

Could you please elaborate? I didn’t fully understand the case

Now there is checker for top level functions, that require bodies for nonExpect functions with existing source. And as there is now no possibility to add body in FIR via extension, I can’t generate body for @ForeignCCall fun doSmth(): String in IR plugin, because it fails by this checker, because function has no body.
In case of generating platform declaration in FIR plugin, I mark existing declaration with isExpect=true (so checker don’t fail) and generating new declaration with source == null (so checker don’t fail)
But in case I want to just modify body of existing declaration checker fails, as it should not be expect and have source != null.
I see now, that there is FirPlatformDiagnosticSuppressor and I believe I can use it now to ignore body check and generate IR body, though I haven’t tried it, as single usage of this suppressor is for JS 🙂

There is no simple way to do it, because of following reasons:

Got it, sound logical, I will investigate other ways of doing what I want

But note that it won’t work in IDE, because CompilerPluginRegistrar is not called during setup of plugins inside it

Ok, from IDE side for my use case the only thing, that I need is to allow to compile functions (and may be classes) with annotations without body, similar to how external modifier works.

Will continue investigation of K2 possibilities for my use cases and report here (or in new threads if there will be different theme)
Big thx for helping and providing a lot of useful information!

[dmitriy.novozhilov]

Now there is checker for top level functions, that require bodies for nonExpect functions with existing source.

Oh, it's indeed a problem. Could you please file an issue about it too?
As a workaround you actually can generate a body for function by calling replaceBody on function returned from createTopLevelFunction. As a stub body I recommend to use return null!! or return TODO()

[Oleg Yukhnevich]

Note: when generating NEW declarations with createTopLevelFunction it works as it have source==null and checker works fine
It doesn’t work for EXISTING declarations without body, as there is no extension to change it
or may be it’s possible to replace it while generating in FirDeclarationGenerationExtension?

[Oleg Yukhnevich]

Created:
• Assertion error: https://youtrack.jetbrains.com/issue/KT-58886/K2-compiler-plugin-generated-top-level-declarations-cause-AssertionError-on-K-JS-and-K-Native
• Modifying declaration: https://youtrack.jetbrains.com/issue/KT-58888/K2-compiler-plugin-top-level-declaration-without-body (though, I think it’s more like a feature request…)

[dmitriy.novozhilov]

Great, thank you!