How can I fetch the Kotlin Project Extension? I'm trying to fix a bug in Dokkatoo <https://github.co...

# gradle

[Adam S]

How can I fetch the Kotlin Project Extension? I'm trying to fix a bug in Dokkatoo https://github.com/adamko-dev/dokkatoo/issues/123

Given a multi-module Gradle project

.
└── foo-project/
    ├── submodule1 (Kotlin JVM)
    └── submodule2 (Kotlin JVM)

When I try and get KotlinProjectExtension I get an exception

val ke = project.extensions.findByType<org.jetbrains.kotlin.gradle.dsl.KotlinProjectExtension>()
// Type org.jetbrains.kotlin.gradle.dsl.KotlinProjectExtension not present

I find I need to have kotlin("jvm") version "1.8.22" apply false on the root project, otherwise Dokkatoo can't fetch the Kotlin org.jetbrains.kotlin.gradle.dsl.KotlinProjectExtension in the subprojects, even though they have applied the Kotlin JVM plugin.

I tried falling back to fetching org.jetbrains.kotlin.gradle.dsl.KotlinJvmProjectExtension, but that also fails with the same error, "Type org.jetbrains.kotlin.gradle.dsl.KotlinJvmProjectExtension not present"

If I debug print the available extensions, I can see that KotlinJvmProjectExtension is present

logger.warn("extensions: ${project.extensions.extensionsSchema.elements.joinToString { "${it.name} ${it.publicType}" }}")
// extensions: kotlin org.jetbrains.kotlin.gradle.dsl.KotlinJvmProjectExtension (...)

[Adam S]

the code that's trying to fetch the extension is here: https://github.com/adamko-dev/dokkatoo/blob/v2.0.0/modules/dokkatoo-plugin/src/main/kotlin/adapters/DokkatooKotlinAdapter.kt#L71-L73

[Javier]

as you are not applying the KGP in the root, the extension will not be there

[Adam S]

sorry, I wasn't clear. The extensions also aren't present in the subprojects, and kotlin("jvm") is applied there

[Javier]

that is really weird, but you should wrap your calls using the extension with withPlugin(…) { }

[Adam S]

I have a test that generates the structure - without kotlin("jvm") version "1.8.22" apply false it fails https://github.com/adamko-dev/dokkatoo/blob/v2.0.0/modules/dokkatoo-plugin/src/testFunctional/kotlin/MultiModuleFunctionalTest.kt#L397-L399

[mbonnin]

KGP needs to be applied next to dokkatoo I'd say?

[mbonnin]

If not they might end up in different classloaders

[Adam S]

yes, I've used withPlugin(...) {} https://github.com/adamko-dev/dokkatoo/blob/v2.0.0/modules/dokkatoo-plugin/src/main/kotlin/adapters/DokkatooKotlinAdapter.kt#L60-L67

[mbonnin]

That's applying the plugin (calling Plugin.apply() ) but not loading it (making .jar known to the JVM)

[Javier]

I don’t think you can avoid applying the Kotlin plugin

[Adam S]

which line is calling Plugin.apply()?

[Javier]

@mbonnin I don’t think they should apply the plugin inside

[Javier]

only react to its presence

[Javier]

and as kgp is not being applied, you can’t react to its presence

[mbonnin]

Let me dig the details but what I meant is KGP needs to be loaded next to dokkatoo if dokkatoo wants a chance to see the symbols in its classloader

[Javier]

but dokkatoo should only add kgp as compileOnly in its dependencies 🤔

[mbonnin]

Agreed

[mbonnin]

My understanding is:
• root classloader: contains dokkatoo jar, not KGP
• module classloader: contains KGP jar, can reference dokkatoo symbols because root classloader is a parent of the module classloader

[mbonnin]

But root classloader doesn't know anything about children classloaders

[mbonnin]

It's a huge pain but 🤷

[mbonnin]

See also https://dev.to/autonomousapps/build-compile-run-a-crash-course-in-classpaths-f4g

[Adam S]

the root project has Dokkatoo applied, but no KGP

subproject-hello has both Dokkatoo and KGP applied - so am I right in thinking they should both be in the classloader?

In subproject-hello when Dokkatoo tries to react to KGP I can see that KotlinJvmProjectExtension is present and applied via debug printing, but Dokkatoo fails to fetch it

[Adam S]

it makes sense that the root project can't fetch KotlinJvmProjectExtension, but it's not trying to

[mbonnin]

am I right in thinking they should both be in the classloader?

I'm not so sure they are in the "same" classloader. My current thinking is module classes can reference root classes but not the other way around

[mbonnin]

Need to step away from keyboard for a bit but I'll do a bit more testing to validate that assumption later today

[Javier]

Kotlin version are matching?

[Adam S]

Dokkatoo has a compileOnly dependency on KGP 1.9.0, and I've tried the example project with both 1.8.22 and 1.9.0 - same failure for both

[Adam S]

findByName("kotlin") fetches something successfully, but it's an Any and casting it to KotlinProjectExtension fails

[mbonnin]

https://github.com/martinbonnin/test-classloaders/blob/main/module/build.gradle.kts#L5

[mbonnin]

You need to "load" all your plugins side by side

[mbonnin]

Of course this is all a giant pain because as a plugin author you usually don't control how your plugins are loaded 😕

[Javier]

Is it failing on a project or on a test?

[mbonnin]

What do you mean? It's failing during configuration time while evaluating the script

[Javier]

If he is using Gradle test kit and he is not adding KGP to the classpath that fail would be normal

[mbonnin]

Don't get me started on Gradle test kit ahah blob upside down

[Javier]

Looks like he is using test kit, so it is necessary to add the KGP and Android plugins

[mbonnin]

We should really start using GradleRunner and Tony's plugins

[mbonnin]

https://plugins.gradle.org/plugin/com.autonomousapps.testkit

[mbonnin]

"test kit" and withPluginClasspath is a huge footgun

[Javier]

val testPluginClasspath: Configuration =
    configurations.create("testPluginClasspath")

tasks.withType<PluginUnderTestMetadata>().configureEach { metadata ->
    metadata.pluginClasspath.from(testPluginClasspath)
}

and in the script something like

testPluginClasspath(libs.agp)
testPluginClasspath(libs.kgp)

[mbonnin]

I'm not touching this with gloves 😄

[Javier]

hahaha

[Javier]

what do you mean with GradleRunner? It is what TestKit uses

[mbonnin]

GradleRunner is well defined. It's an API I can use, it has artifacts & kdocs and it's working well 👍 . TestKit I have no idea what it is except it tests my plugin in conditions that are different from what my users are going to do

[mbonnin]

This is especially true for such weird classpath errors

[mbonnin]

You're better off publishing to a local maven repo and testing in real life conditions

[Javier]

I mean... for me TestKit is that GradleRunner 🤔 At least the import includes it org.gradle.testkit.runner.GradleRunner

[Adam S]

thanks for the reproducer Martin!

[mbonnin]

TestKit is that GradleRunner

For me TestKit = GradleRunner + withPluginClasspath() +?

[Adam S]

Dokkatoo uses GradleRunner but it doesn't use withPluginClasspath - it publishes to a local maven repo

[Javier]

withPluginClasspath is a configuration inside GradleRunner

[Adam S]

https://github.com/adamko-dev/dokkatoo/blob/v2.0.0/buildSrc/src/main/kotlin/buildsrc/conventions/maven-publish-test.gradle.kts

[mbonnin]

https://docs.gradle.org/current/userguide/test_kit.html
Yep, what I was reading, it's all the same lib

[Javier]

you are not forced to use it if you don't depend on other plugins

[mbonnin]

Even if you do depend on other plugins, you can avoid it

[Javier]

They (Gradle) should fix that then

[Javier]

TBH it should be done automatically, it has no sense you have to "dupe" what your script is doing in a custom configuration

[mbonnin]

Probably they should improve this indeed. Until then, I strongly recommend publishing to a local repo yourself or using Tony's plugin, especially to test classloading issues: https://github.com/autonomousapps/dependency-analysis-gradle-plugin/tree/main/testkit

[Adam S]

I made an issue for Gradle because as far as I can tell this classloader behaviour is a bug, or it should at least have warnings and documentation https://github.com/gradle/gradle/issues/27218

[Javier]

Technically it is documented

[Javier]

https://docs.gradle.org/current/userguide/test_kit.html#sub:test-kit-classpath-injection

[Javier]

Therefore, the test build does not share the same classpath or classloaders as the test process and the code under test is not implicitly available to the test build.

[mbonnin]

@Javier the bug doesn't have anything to do with TestKit

[mbonnin]

It's a Gradle bug that classloaders are weird. It's also the reason why so many plugins need to shadow their dependencies because conflict resolution does not always have the expected outcome in plugins.

[mbonnin]

See also https://github.com/gradle/gradle/issues/1370

[Adam S]

almost 7 years old 😬