With Kotlin 1.6.21 and above, what is the current ...

# compiler

[Oliver.O]

With Kotlin 1.6.21 and above, what is the current purpose of the task compileCommonMainKotlinMetadata?

Second question in 🧵.

[Oliver.O]

Given this configuration using the (current) multiplatform Gradle plugin:

kotlin {
    jvm {
        withJava()
    }
    js(IR) {
        browser()
    }
    sourceSets {
        val commonMain by getting {
        }
        val clientMain by creating {
            dependsOn(commonMain)
        }
        val jvmMain by getting {
            dependsOn(clientMain)
        }
    }
}

Why is there an extra metadata compilation for commonMain but not for the custom intermediate clientMain source set?

[Oliver.O]

Maybe I should add that I need the information to complete my work on addressing KSP multiplatform issues. It seems like I'm almost done but I need a better understanding of why there is an extra common compilation and not just one compilation per target. @dmitriy.novozhilov

[dmitriy.novozhilov]

IDK actually
@dsavvinov maybe you know?

[dsavvinov]

Yes, I do.

First of all, there's an additional internal target in each multiplatform project - metadata target. It's purpose is to publish a representation of API of the common code (one of such representations is .kotlin_metadata, hence the name). It is needed so that consumers of multiplatform code could resolve their common code against yours common code, and saw only what you, as a library author, decided to share among all targets.

This target contains at least one, or potentially multiple, "metadata compilations" - roughly one per common* source set. Specifically, you've seen the metadata compilation for commonMain source set.

Finally, on Gradle level, there's no compilations or targets. Instead Gradle provides us entities like Task or Configuration. In order to tell Gradle to resolve and download some dependencies, we create a bunch of tasks and Configurations, based on Compilations. I won't get into details of that mapping (is frustratingly non-trivial), but 'compileCommonMainKotlinMetadata' is one of such Configurations of metadata-compilation for commonMain source set (if I'm not mistaken, it contains classpath for that compilation).

*Now, you've correctly spotted inconsistency: there's no metadata compilation for clientMain source set.
This is because it is a weird beast: on the one hand, it doesn't share code between two or more targets, but on the other hand it is not a default platform-specific source set like jvmMain. We have some troubles deciding how to handle those source sets, and therefore we dont create a metadata-compilation for them.

If you include this source set into JS/main compilation by adding a jsMain.dependsOn(clientMain), it will become a usual shared source set (similar to commonMain in that regard), and the metadata should be created for it

[Oliver.O]

Thank you so much. That helps a lot. Currently, there is a fundamental problem with KSP and compileCommonMainKotlinMetadata. KSP processors are currently not source set-aware. A KSP processor generating code for commonMain sees the same code again and re-generates identical code for a target compilation, leading to Redeclaration errors. To resolve this, we have to provide the processor with source set information to make informed decisions. I just wanted to make sure that this multiple-compilation strategy is still needed and we can not just skip the common metadata compilation.

I did already guess that the whole metadata thing is for library sharing, but was not sure whether this was superseded by current Klibs and all that. I still wonder why the compiler cannot just generate common code in a target compliation when it detects that it is not yet there (or non-current). I'll experiment with your Js suggestion to figure out how to handle cases involving multiple common code sets. Thanks again!

[dsavvinov]

It will be changed somehow as klibs will get more and more into the ecosystem, but I don't believe it will induce some complete rewriting of the related code. See, it is convenient to separate concerns of a single "compile this code"-task and of the complex multi-step process of generating binary representation of a multiplatform library. We could roll this into compilation maybe, but you'd still have to deal with some other entities then, which would represent a single atomic invocation of Kotlin Compiler, so inherently you will need corresponding handling somewhere

[Oliver.O]

Sure – the compiler is already a huge beast and simplifying things there requires lots of deliberation and, of course, time. For now, I'll just try to figure out what might be the most convenient API for KSP processor authors in the presence of multiple invocations. (Unlike the compiler, a KSP processor should probably not be required to maintain its own database cache with information on which code it has seen or generated in previous invocations.)