has anyone written an IR plugin that creates new IrFiles? The examples I’ve seen seem to focus on ma...

# compiler

[Zac Sweers]

has anyone written an IR plugin that creates new IrFiles? The examples I’ve seen seem to focus on manipulating existing code but not necessarily generating new top-level classes. Curious how others are approaching this

[Youssef Shoaib [MOD]]

I tried to do something like that but it quickly got out-of-hand because the compiler just doesn't expect new files to be created in the IR stage. However, there's the CollectAdditionalSources (if memory serves) extension point where you can place in new files. So, your plugin can just create dummy files, and IR files will be generated for them accordingly.

[Zac Sweers]

yeah I’m finding out the convention seems to be to just add to existing files or classes

[Youssef Shoaib [MOD]]

Yeah and IIRC it shouldn't change much since worst-case scenario you can do (most) things by just adding whatever number of classes you want into one singular file

[Zac Sweers]

to a singular file or to a class? I found that adding to the file resulted in a CME, but didn’t see an obvious way to defer it to later

[Youssef Shoaib [MOD]]

I thought adding to a file worked? One second. I do have a plugin where I added functions to a file.

[Youssef Shoaib [MOD]]

Found it! I remember now. Basically, you can't add declarations to a file while you're transforming it (that's why you're getting a Concurrent Modification Exception), and so what I do is I collect all the new declarations that I'd like to add to an arbitrary IrFile in a map with that IrFile, and then after the transformer is completely done, I do file.declarations.add() and I set that specific declaration's parent to be the file. Like this simply:

val deferredAddedFunctions: MutableMap<IrDeclarationWithParent, IrFile> = mutableMapOf()
MyTransformer(deferredAddedFunctions).lower(moduleFragment) //Add any created declaration to that mutable list
for ((deferredAddedFunction, file) in deferredAddedFunctions) {
  file.declarations.add(deferredAddedFunction)
  deferredAddedFunction.parent = file
}

IIRC this avoids a CME and works properly. Creating those declarations, btw, can be done using IrDeclaration.factory.buildXX. In the linked repository, you can see an example of this in practice, but keep in mind that the code isn't the most comprehensible in the world right now since it's just a side project.

[Youssef Shoaib [MOD]]

Any questions just ask me! 😄

[Zac Sweers]

nice! Thanks so much

[Zac Sweers]

worked like a charm, thanks again

[Zac Sweers]

Followup question for ya - ever tried to generate other files? For example - resources

[Youssef Shoaib [MOD]]

Not quite, but I have generated files by placing them in buildDir/generated/sources/${pluginName}/${sourceSetName}. I believe for resources they're usually placed in buildDir/resources/${sourceSetName} (based on the behavior of the Maven plexus plugin I believe). BTW, it's absolutely okay to generate those files as blanks and then populate them in the IR phase since resources are merged in the JVM backend itself (and for JS their equivalent for resources is sent to whatever build system is used to pack all the files together).

[Youssef Shoaib [MOD]]

But yes there shouldn't be any issues like CME or anything like that if you add resources, but I think you'll have to create blank files early on because I'm not sure if the compiler will pick up any new files in the middle like that.

Btw congrats on your appearance on Talking Kotlin. Looking forward to that episode!

[Zac Sweers]

yeah I know I could just generate them manually (pass in an output path arg, patch them to the resources task, etc), but worried about how that would play with incremental compilation since it’s not exactly cooperating with the rest of the compiler

[Zac Sweers]

thanks! How did you know? 👀

[Youssef Shoaib [MOD]]

Regarding incremental compilation, I believe plugins currently can't deal with proper incremental compilation current, at least AFAIK some plugins like Anvil just disable it. I think creating resources should be fine though since it's similar to how annotation processors can create resources

[Youssef Shoaib [MOD]]

There's a premiere on the Kotlin by Jetbrains Youtube channel scheduled for the 27th I believe, with the title "Slacking with Zac Sweers" for 5pm UTC

[Zac Sweers]

Yeah I'm more thinking about cleaning up stale files if an originating file is deleted