https://kotlinlang.org logo
Join Slack
Powered by
I have a plugin project that depends on `org.jetbr...
# gradle
v
I have a plugin project that depends on 
org.jetbrains.kotlin.multiplatform
and also on 
embeddedKotlin("compiler-embeddable")
. The reason is, that one of the plugins applies 
kotlin("multiplatform")
and another is using
Copy code
val File.importedFiles: List<File>
    get() = if (!isFile) {
        emptyList()
    } else {
        PsiManager
            .getInstance(
                KotlinCoreEnvironment
                    .createForProduction(
                        Disposer.newDisposable(),
                        CompilerConfiguration().apply {
                            put(MESSAGE_COLLECTOR_KEY, NONE)
                        },
                        JVM_CONFIG_FILES
                    )
                    .project
            )
            .findFile(CoreLocalVirtualFile(CoreLocalFileSystem(), toPath()))
            .let { it as KtFile }
            .fileAnnotationList
            ?.annotationEntries
            ?.asSequence()
            ?.filter { it.shortName?.asString() == "Import" }
            ?.flatMap { it.valueArgumentList?.arguments ?: emptyList() }
            ?.mapNotNull { it.getArgumentExpression() as? KtStringTemplateExpression }
            ?.map { it.entries.first() }
            ?.mapNotNull { it as? KtLiteralStringTemplateEntry }
            ?.map { resolveSibling(it.text) }
            ?.flatMap { it.importedFiles + it }
            ?.toList()
            ?: emptyList()
    }
to get the files a given 
*.main.kts
file imports. That latter cannot be done as a worker action because this is needed at configuration time to determine the input files for a task that is executing that 
*.main.kts
file. But now when I update the Kotlin version to 2.1.0 I get a warning > w: The artifact 
org.jetbrains.kotlin:kotlin-compiler-embeddable
is present in the build classpath along Kotlin Gradle plugin. > This may lead to unpredictable and inconsistent behavior. > For more details, see: https://kotl.in/gradle/internal-compiler-symbols when applying that plugin. As it is a runtime issue, I assume that splitting into multiple plugins would also not help as in the end it would again be in the same runtime classpath. How would I solve this task in a forward-compatible way, besides putting the logic into a separate Kotlin file or script and executing it as external process instead of in-process, or splitting it into multiple plugins and ensuring those plugins are not applied to the same projects, or relocating the compiler classes into my plugin?
m
Just ignore the warning? This is probably a false positive?
v
Is it? The statement is true. KGP and 
kotlin-compiler-embeddable
are both in the build script classpath.
m
If there is ever a conflict between 
kotlin-gradle-plugin
and 
kotlin-compiler-embeddable
you'll need to ship different versions of your plugin for different versions of KGP
But I believe KGP is moving its own 
kotlin-compiler-embeddable
usages to a differrent classloader so you "should" be fine?
v
But my plugin does not in any way use KGP
😞
m
v
Well, the part that uses the embeddable compiler I mean
m
Another way to look at this:
That latter cannot be done as a worker action because this is needed at configuration time to determine the input files for a task that is executing that 
*.main.kts
file.
If it's only about input files and does not affect the task graph, can't the computation be moved to a separate task?
That separate task would never be up-to-date if it has no inputs but maybe it's okay?
v
Hm, well, the problem would then be that I need output of that task. And iirc it is not trivial to get get output from worker action for further processing. Besides writing to some file and then reading that file which feels like a crutch in that situation. 😕
But yeah, maybe that's the way to go. Or I could try to use 
kotlin-compiler
if the needed classes are in there too and hope this is not checked for presence. 😄
😄 1
m
Besides writing to some file and then reading that file which feels like a crutch in that situation.
Yea, paying the serialization price all the time 😞 I wish Gradle had a way to save that if the 2 tasks are part of the same build and the intermediate result is not "interesting". But that sounds like a complicated problem
t
KGP still brings parts of Kotlin compiler into build classpath, so it is not false-positive warning. Ideally you should not use Kotlin compiler in the build classpath - better to use worker with some isolation mode: https://kotlinlang.org/docs/whatsnew21.html#compiler-symbols-hidden-from-the-kotlin-gradle-plugin-api
at some point we should completely stop depending on these compiler parts in KGP, but it will take some releases.
👍 1
m
KGP still brings parts of Kotlin compiler into build classpath, so it is not false-positive warning
I mean it's fine if the symbols I compile against are compatible with the version used by KGP, right?
v
As I said, using worker is not really feasible here as it is not a task action, but just determining task input files. Unless somehow done via some auxiliary task that for example determines the files at task execution time and copies them to a directory and then using that directory as input files. But that's then actually a waste of build time.
m
(side note: worker + classpath isolation leaks)
t
I mean it's fine if the symbols I compile against are compatible with the version used by KGP, right?
It could be fine or could lead to some cryptic build errors 🤷‍♂️
⬆️ in case of compiler versions are different
m
Like every classpath issue out there. There's nothing KGP specific there
Only difference is I guess the compiler APIs might be a bit more unstable than something like kotlinpoet or antlr so there's a higher chance of conflict but it's the same underlying issue
t
example of such error. There were some other instances of different errors
v
Not too cryptic to me 😄 
NoSuchMethodError
is in 98.7% version conflict / classpath problem 😄
m
Exactly, at this point, I became quite used to it (sadly)
t
Not too cryptic to me
You are quite advanced Gradle user 🙂 Imagine this error will see some Gradle newbie
it is not clear how to resolve it right away
v
More Java than Gradle, but yeah, of course 😄
m
Yea but it's happening all the time on other dependencies (asm, okio, kotlinpoet, etc...), if every plugin start adding custom messages it's going to be a lot of messages 😅
Completely agree this situation is super confusing though and the ecosystem needs to do something about it.
Ideally plugins either shadow their dependencies or use isolated classloaders
t
if every plugin start adding custom messages it's going to be a lot of messages
If every message will have a resolution suggestion and they are gone once issue is resolved - I don't see a big problem here 🙂
v
Which both have extreme drawbacks up to unusability. For example exhausted heap, exhausted metaspace, inability to work together, ... Like always, every option has its pros and cons.
m
If every message will have a resolution suggestion and they are gone once issue is resolved
I think that's my main gripe with the warning, ultimately, it's only actionable to the 
kotlinter
maintainers, right?
If I'm a regular Android app developer, there's not much I can do besides wait for 
kotlinter
to move to a different classloader (which takes some time because it's no small refactor)
v
Well, it is actionable for you to update the plugin, or to complain to them and make them fix it if no update is available yet. Just like many of the Gradle deprecation warnings that are coming from plugins you apply. 🤷‍♂️
t
yeah, or pause using it until it will be updated
m
Yea I guess
Is kotlin-compiler-embeddable "stable"?
v
Better, pause the warning, or at least only show it if the version is actually incompatible, then a compatible version could be forced by strict constraint. 🙂
m
Because if 
kotlin-compiler-embeddable
is stable, it's also fine to resolve all dependencies in the root build.gradle.kts
(this trick is the only reason why most build work at all 😅 )
v
Most of yours maybe, I seldomly need to do that 😄
m
You're using way too much metaspace then 😉
v
Sounds like a good idea to me 🙂
🙌 1
t
APIs inside Kotlin compiler are not stable 🙂
👍 1
m
Made a few suggestions to the documentation there: https://github.com/JetBrains/kotlin-web-site/pull/4631
v
I reworked it now. Actually I think I like it even better. I have now an untracked task that uses the worker API and writes imported files (recursively) to a text file. The embedded compiler is 
compileOnly
and is then supplied to the isolated worker classpath again. And another task that uses the worker API to execute the 
.main.kts
scripts. Now it all works without that warning and the tasks can also run in parallel and are more maintainable too due to separating into proper classes. 🙂
👍 1
👌 1
Thanks for the heads-up 🙂
128 Views
Open in Slack
PreviousNext
Powered by