hi, i've spent many days over the last few weeks o...

# compiler

[Petter Måhlén]

hi, i've spent many days over the last few weeks or so trying to figure out https://github.com/bazelbuild/rules_kotlin/issues/342 - an issue where there is a KotlinReflectionNotSupportedError when trying to use Bazel to build a Kotlin project with an annotation processor that requires reflection. The maintainers of the Bazel rules_kotlin project seem unable to provide any insight, but I figured that maybe someone here could. In particular, I wonder if anyone has any insights into whether it might be
a) a version conflict between the version of kotlin-stdlib and kotlin-reflect that eventually get used, or
b) a timing issue, perhaps due to the use of ClassPreloadingUtils.preloadClasses in https://github.com/bazelbuild/rules_kotlin/blob/master/src/main/kotlin/io/bazel/kotlin/builder/toolchain/KotlinToolchain.kt#L70, that means that the Reflection class gets initialised at a point in time when reflection isn't added?
c) or something completely different. There's a lot of ground to cover in all the Kotlin compilation ecosystem as well as the Bazel stuff, and I'm new to both.
Help very much appreciated, if possible!

[Roman Artemev [JB]]

Hi, has kotlin-reflection.jar been provided at runtime? By default reflection support is not included in class path at runtime

[Petter Måhlén]

it's supposed to be transitively resolved, to be specific, the annotation processor is moshi-kotlin-codegen, which depends on kotlinpoet, which depends on kotlin-reflect. jar. Maven handles it, Bazel doesn't. Also, in the error message when it fails (for some combinations of versions of tools, I've done huge amounts of testing), I can see things like:

Compilation failure: compile phase failed:
warning: some JAR files in the classpath have the Kotlin Runtime library bundled into them. This may cause difficult to debug problems if there's a different version of the Kotlin Runtime library in the classpath. Consider removing these libraries from the classpath
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-reflect/1.3.50/stamped_kotlin-reflect-1.3.50.jar: warning: library has Kotlin runtime bundled into it
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib/1.3.50/stamped_kotlin-stdlib-1.3.50.jar: warning: library has Kotlin runtime bundled into it

[Petter Måhlén]

for that particular build, bazel's reporting the following classpath for annotation processors

--processorpath
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/com/squareup/moshi/moshi-kotlin-codegen/1.9.2/stamped_moshi-kotlin-codegen-1.9.2.jar
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/com/squareup/kotlinpoet/1.4.4/stamped_kotlinpoet-1.4.4.jar
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib/1.3.50/stamped_kotlin-stdlib-1.3.50.jar
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/org/jetbrains/annotations/13.0/stamped_annotations-13.0.jar
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-common/1.3.50/stamped_kotlin-stdlib-common-1.3.50.jar
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-jdk7/1.3.50/stamped_kotlin-stdlib-jdk7-1.3.50.jar
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-reflect/1.3.50/stamped_kotlin-reflect-1.3.50.jar
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/com/squareup/okio/okio/1.16.0/stamped_okio-1.16.0.jar
bazel-out/host/bin/external/maven/v1/https/repo1.maven.org/maven2/com/squareup/moshi/moshi/1.9.2/stamped_moshi-1.9.2.jar

[Petter Måhlén]

it's data like the above that made me think that it could be related to classloading somehow - that the kotlin.jvm.internal.Reflection class gets loaded at a point in time before kotlin-reflect.jar is available on the classpath, or so. I looked inside the file listed above and it seems to have the right things in it, including the kotlin.reflect.jvm.internal.ReflectionFactoryImpl class

[Hanno]

I am completely lacking knowledge about bazel, but i had the given error in a project that was executed in a custom module runtime, basically having the kotlin std loaded in a parent classloader, that hadn't had reflection available because it was only loaded by the child modules. Any Info about how bazel loads and resolves depdendencies needed for annotation processing? :)

[Petter Måhlén]

@Hanno what you're describing is what I think is my best theory of what's happening, too. The way to build Kotlin with Bazel is through rules_kotlin, which invokes the Kotlin compiler through https://github.com/bazelbuild/rules_kotlin/blob/master/src/main/kotlin/io/bazel/kotlin/compiler/BazelK2JVMCompiler.kt, and creates a classloader for it like this: https://github.com/bazelbuild/rules_kotlin/blob/master/src/main/kotlin/io/bazel/kotlin/builder/toolchain/KotlinToolchain.kt#L69. I haven't been able yet to understand how those things work in detail, in particular the ClassPreloadingUtils, but I guess that's where I'll be going next. Another thing I can try to look at is trying to run Bazel with a debugger so I can inspect things inside the process, but that seems like an uphill battle. Thanks for the confirmation that a problem like this can happen due to classloading, though! That's encouraging and might help keep me going until I figure this out 🙂

[Petter Måhlén]

@Hanno btw, what was the solution in your case? adding kotlin-reflect to the parent classloader or creating a separate classloader branch where stdlib got loaded so the parent stayed 'clean'? or something else?

[Hanno]

Yea, basically that was the solution. Otoh having stdlib and reflect on the same scope is the correct solution nonetheless. But i had to adjust projects and explain it to people, or in other words: we always need to add reflect when stdlib is added. I thought about requesting a std artifact that included reflect so that people wouldn't need two dependencies, but it's too complicated to talk about those things :D

[Hanno]

I also found out about that debugging up to the point where reflect is loaded, and then i saw different classloaders and it was clear