There's been lots of chat about FIR in the context of plugins and the within the compiler, but are t...

# compiler

[jw]

There's been lots of chat about FIR in the context of plugins and the within the compiler, but are there any examples of setting up FIR in a standalone project to interpret parsed Kotlin? I'm currently doing this with PSI and it's a huge pain because I don't actually care about the precise syntax tree but just want to parse a semantic tree (similar to what runtime reflection would provide).

[dmitriy.novozhilov]

There is no such standalone tool yet
But you can create FIR plugin with additional file checker and introspect resolved files in this checker

[dmitriy.novozhilov]

Here is the sample project with FIR plugin setup
https://github.com/demiurg906/kotlin-compiler-plugin-template

[jw]

I found compileModuleToAnalyzedFir which might be exaclty what I want. So I think I can walk myself backwards towards the KotlinCoreEnvironment / CompilerConfiguration / VirtualFileManager / etc. that I already have setup for PSI.

[dmitriy.novozhilov]

Yeah, this approach will work too

[jw]

it's working! SO much nicer than dealing with PSI. It's like night and day.

[Strum355]

Curious what youre working on here that you want to interpret FIR 👀

[jw]

We use Kotlin as a schema language to define sets of types and properties within those types to perform code generation of other things. For the past two years I've been using reflection, but we lose two very important things: comments and default expressions. For defaults I've just been forcing people to put a string in an annotation that I read. Haven't had a solution for comments.

So I always knew that I would be switching to UAST/PSI, but it is so unpleasant to parse at that layer because it's basically just a lexer/parser. Getting out something usable (especially in comparison to reflection) involves so much error/unexpected handling.

So FIR gives me higher level constructs closer to reflection-level and also handles linking of types.

[eygraber]

Is it worth it for projects like detekt (where a compiler plugin seems to be the future) and ktlint (where there don't seem to be any plans to use a compiler plugin) to switch over to using FIR? Using UAST/PSI has always been a drag for me when reading/contributing rules, but I'm assuming it would be a pretty big migration?

[jw]

Well ktlint cares about formatting so they'd need to stick to PSI

[eygraber]

Would it be a bad idea to try and write out the formatted code by mapping FIR to source code instead of modifying the existing code?

[dmitriy.novozhilov]

It's not easy to map FIR back to source code
FIR is a semantic tree, so it doesn't store any information about syntax aspects, like parenthesis or linebreaks
Also it performs some desugarings, so you will be forced to undo them back (e.g. there is no if in FIR tree, only when)

The only thing FIR knows about the syntax is the links to original PSI elements (to properly report diagnostics), but the mapping is not full (some PSI nodes are missing, and some were duplicated)

[dmitriy.novozhilov]

I also didn't fully understand how do you want to use resolve information for pure syntax style diagnostics

[eygraber]

Ok so wouldn't be useful for tools like ktlint, but for something like detekt rules that check behavior instead of syntax it could be useful.

[dmitriy.novozhilov]

Yeah, probably
But there is a cumbersome question about API stability. At this moment we are struggling to make stable API for just declarations. And API of body content much larger and has more chances to change with bugfixing, performance improvement of new language features
So tools which rely on it will suffer a lot (like all IR plugins right now)

[eygraber]

Stability will come with 2.0 or after it?

[dmitriy.novozhilov]

After
Main goal for 2.0 is enabling FIR by default in all scenarios

[hfhbd]

Just curious, are comments even reference-able by using psi or are they dropped during parsing (ParserDefinition.getCommentTokens)? I had the same problem with my own compiler (a COBOL to Kotlin converter), and decided to add the comments to the actual language to keep the comments used by the Kotlin code generator.

[dmitriy.novozhilov]

They should be referencable, because compiler looks for kdoc comments on IR stage in some cases

[hfhbd]

But not inline comments, should it?

[dmitriy.novozhilov]

I never checked, actually
But most likely they also should be saved

[mikehearn]

How does that compare to stability of PSI? I have a side project that involves something like PsiMiner

[mikehearn]

I'm mostly interested in just going from declaration to the character/byte ranges in the defining file, but also doing reference resolution.

[mikehearn]

It's a bit unclear from the above discussion what's better. If FIR is a semantic tree meant for a compiler presumably it also drops comments?

[mikehearn]

(however ideally this tool is not kotlin specific, hence the appeal of psi)

[jw]

FIR doesn't have comments but it has originating PSI elements where you can drop down and read the precise syntax tree

[handstandsam]

Thanks for this thread. It led me to https://github.com/cashapp/zipline/blob/6f40aaef8ccfbc02b6cb575d72888eedd65cf9b5/z[…]in/kotlin/app/cash/zipline/api/validator/fir/KotlinFirLoader.kt which is super helpful (and works).