The Kotlin compilation process works in several st...

# compiler

[tschuchort]

The Kotlin compilation process works in several stages:
- First the compiler is called with the kapt plugin which generates Java stub files from the Kotlin files and then calls apt with that. AFAIK the kapt plugin "shortwires" the rest of the compilation, so that no actual compilation of Kotlin sources takes place in this stage
- Then the Kotlin compiler is called regularly to compile the Kotlin sources
- Finally the Java compiler is called to compile all Java sources, with the compiled Kotlin class-files from the previous step on the classpath.

My question is: Do third-party compiler plugins get passed to both the first 2 stages or just in the 2nd stage? Does the kapt plugin prevent the execution of further compiler plugins as well, like it shortwires the regular compilation?

[raulraja]

Compiler plugins get called when kt files are ready

[raulraja]

On the analysis phases though you can intercept resolution and codegen as well

[tschuchort]

I think we are talking about different phases. The "phases" I mean are decided by the Kotlin Gradle and Kapt Gradle plugins and make distinct calls to the actual Kotlin compiler (afaik).

[raulraja]

Yes but those phases are not calling the ComponentRegistrar ,the Kotlin compiler calls registerProjectComponents which plugins override.

[raulraja]

Then the plugin register to the Kotlin compiler phases unlreated to kapt and apt afaik

[raulraja]

Kotlin compiler plugin do not need to use annotation processors or depend on annotations

[raulraja]

For example a plugin may just be fiddling with IR codegen or operate at the psi ktelements level before resolution

[Imran/Malic]

An AdditionalTypeChecker would be an example for the latter, right @raulraja?

[raulraja]

Yes, though typechecking is not a compiler extension so more of a hack in which you use an early phase to change a compiler service.

[raulraja]

@tschuchort if this is related to your lib it probably comes from us trying to use it to inject the compiler plugin in tests?

[raulraja]

I think in our case we were failing to see a way to modify the compiler arguments in which we could just tell it where our plugin jar should be loaded from

[raulraja]

We plan on using compile testing in arrow meta and thanks for doing an awesome lib like that 😄

[Imran/Malic]

Thanks @tschuchort for the lib💪🏽👏🏽

[tschuchort]

The gradle plugin calls the Kotlin compiler twice, once with the kapt plugin and once without. When the compiler gets to KaptComponentRegistrar::registerProjectComponents it does its thing and registers the AbortAnalysisHandlerExtension which just says "yes we're finished, do nothing else", effectively preventing all the following extensions like code generation to be run. Then the compiler terminates and the Gradle plugin calls the compiler again, this time without the kapt plugin.

Now what I'd like to know is if the Gradle plugin also passes all other third-party plugins via the pluginClasspath in both invocations of the compiler. Because this could lead to some third-party plugins being executed twice. Or perhaps it makes no difference if they are executed twice if they only register extensions for the later extensions that don't get executed due to kapt's short-circuiting.

[tschuchort]

Yes, this is related to the PR by rachel carmena, but we also need to know for the PR that @Foso is working on.

[raulraja]

The compiler may be in daemon state

[raulraja]

when used by the gradle process in whichever case it would always be invoked with the plugin directive and plugin classpath

[raulraja]

This should be easy to test with our plugin because when it loads it prints to std out

[raulraja]

which subplugins of meta is registering like higherkinds, type classes etc…

[raulraja]

Since the lib is instantiating the compiler args I think it’d be enough to provide the user raw access to the Kotlin Compiler Arguments

[raulraja]

other users may want to modify these to do other things such as enable IR, etc.

[tschuchort]

in whichever case it would always be invoked with the plugin directive and plugin classpath

How can the pluginClasspath be the same when the kapt plugin is only used in the first call? If it were used always, it would never compile anything because kapt always aborts the compilation before the codegen phase

[tschuchort]

I didn't give access to the K2JVMCompilerArguments directly because I want to hide the compiler dependency. Although most command line arguments (like the pluginClasspath) are parameters of KotlinCompilation that you can change. Though that doesn't help you with this problem because you as a user can't decide to which stage those command line arguments apply.

[tschuchort]

As you can see here https://github.com/tschuchortdev/kotlin-compile-testing/blob/0f6519fdddb00f89763eceafa8cc7e1fa78e20bb/src/main/kotlin/com/tschuchort/compiletesting/KotlinCompilation.kt#L476 and here https://github.com/tschuchortdev/kotlin-compile-testing/blob/0f6519fdddb00f89763eceafa8cc7e1fa78e20bb/src/main/kotlin/com/tschuchort/compiletesting/KotlinCompilation.kt#L520 different cli arguments are given to the K2JVMCompiler in different stages.

[raulraja]

You could potentially append in two places user overrides in the test? I guess I don't understand the issue because in both cases we could load env bars and system properties which is also how we pass them today to the compiler

[raulraja]

A better type safe way is to expose the compiler config directly

[Rachel]

Hi @tschuchort, thanks again for your time and your kind attention with this issue

[Rachel]

I made this test:

[Rachel]

- Comment the code in which the existence of annotationProcessors is checked in stubsAndApt

[Rachel]

- Create a new local lib with that change

[Rachel]

- Run a test with Arrow plugin compiler and that new lib

[Rachel]

Result: there is no problem with the 2 executions. Files are overwritten in the same tmp directory

[Rachel]

I attach the log here:

[Rachel]

two-executions.log

[tschuchort]

I'm not convinced that this is the correct way to do it, but if you say that it works with arrow at least, I will merge the PR and we can fix it later if problems arise.

[Rachel]

Thanks @tschuchort!! I can promise that I will be available to fix those problems

[Rachel]

I feel responsible about it

[Rachel]

Thanks again Thilo!!

[tschuchort]

Well, it's not really a bug with your code. We simply don't know what the desired behaviour should be. But if you find out any more information about this, feel free to make another PR.

[Rachel]

Ok @tschuchort, let's keep in contact!

[Rachel]

Thanks!