Is there some Gradle plugin to let a `*.main.kts` ...

# scripting

[Vampire]

Is there some Gradle plugin to let a *.main.kts script run?
Especially without having Kotlin installed on the local machine.
Or is it really required to have Kotlin installed and then invoke it with the script as argument?

[Vampire]

I thought it would be a good start to compile it using

plugins {
    kotlin("jvm") version "1.7.0"
}

repositories {
    mavenCentral()
}

kotlin {
    sourceSets {
        main {
            kotlin.srcDir("path/to/kts/file")
        }
    }
}

dependencies {
    implementation(kotlin("main-kts"))
}

but that results in

failed to access class org.jetbrains.kotlin.com.google.common.base.Platform from class org.jetbrains.kotlin.com.google.common.base.Ticker$1 (org.jetbrains.kotlin.com.google.common.base.Platform is in unnamed module of loader 'app'; org.jetbrains.kotlin.com.google.common.base.Ticker$1 is in unnamed module of loader java.net.URLClassLoader @11cfd2b3)

😞

[Tobias]

When I had that problem, I ended up putting the logic into buildSrc, so I'm watching this thread

[Vampire]

Which logic did you put into buildSrc?
Or you mean you made the Script a Gradle plugin?
That would unfortunately not be feasible in my case.
I want to start using the github-actions-kotlin-dsl and I'd like to have a Gradle task that can execute it,
so it must be that exact *.main.kts file that is executed.

[Tobias]

I just had the shared code in the buildSrc so I could call it from the build-scripts. It wasn't pretty, but it worked for the simple purpose of the project

[Tobias]

Like I said though, I'm interested in the responses to this thread, because my solution was more of a hack imo

[ilya.chernikov]

@Vampire, do you need to run main-kts script as a part of the build process? I haven't heard about a plugin for that, although I think it should be possible to do it, in a plugin, buildSrc or right from a gradle script.
You probably will not be able to make it run in-process (due to conflicts with the kotlin compiler embedded into gradle), but it should be possible to implement it by calling an external java process with kotlin compiler using gradle dependencies, without the need to have cli compiler installed. But you'll need to get access (via gradle deps) to quite some compiler jars, which are installed along with cli compiler.
From the top of my head, you'll need
kotlin-compiler-embeddable
kotlin-scripting-compiler-embeddable
kotlin-main-kts
kotlin-script-runtime
most likely kotlin-stdlib
and all transitive dependencies.
then you'll need to run
java -cp <classpath from deps above> org.jetbrains.kotlin.cli.jvm.K2JVMCompiler -script <your script>.main.kts.

[Vampire]

as a part of the build process

If you mean in-process, no.
I just want to be able to execute a Gradle task that runs the *.main.kts script without the need to have Kotlin available in the path of the system.

[ilya.chernikov]

Then, I believe, you can implement the task using the method I described above. Unfortunately I'm not aware of any existing plugins or code snippets that doing it now.

[Vampire]

Thanks, will play with it and cry here if I need more help.
Is it exepected that this compile error I mentioned is coming if you try to compile the script like I tried?

[ilya.chernikov]

Actually - no, I don't know what it means. I have some vague suspicions, but I'm not sure that it makes sense to dig it further, I don't see how it may help you.

[ribesg]

I feel like the use case is that it should be as easy to use a kotlin script as it is to use a bash script from a Gradle build file, given that we are already in the context of Gradle build files written in Kotlin, we are already in some kind of Kotlin context. I think it’s expected to not be a hard thing to do at all

[ilya.chernikov]

Well, if you have Kotlin cli installed, as you usually do with bash, then it is no more difficult.

[Vampire]

That's only marginally comparable.
If bash is installed, running a bash script is easy.
If Kotlin is installed, running a Kotlin script is easy.

[ribesg]

bash cli is understandably much more common than kotlin cli

[Vampire]

If Bash is not installed you will have a hard time running it.
But it would be great if it were easy for a Kotlin script. 🕵️‍♂️

[Vampire]

bash cli is understandably much more common than kotlin cli

On Linux

[ribesg]

You mean “everywhere but on windows”

[Vampire]

Don't let my help-seeking thread now swamp in nitpicking about terminology please, you understood what I meant. 😉

[Paul Woitaschek]

You can instead not use kts 🙃

[Paul Woitaschek]

With the Gradle application plugin it's pretty easy and you can write a simple shell script that pipes the arguments through to Gradle

[Vampire]

Can you elaborate on how I would run the *.main.kts script without using kts?

As I already explained,
I want to start using the github-actions-kotlin-dsl and I'd like to have a Gradle task that can execute it,
so it must be that exact *.main.kts file that is executed.

[Paul Woitaschek]

Ah sorry, didn't read all the 20 responses, my bad

[Vampire]

🙂

[Paul Woitaschek]

You could still just use a Gradle module. And then in your kts script just use the ProcessBuilder to execute the script that's located within a gradle module

[Vampire]

Another precondition was, that Kotlin should not be necessary to be available in the PATH

[Vampire]

Or did I misunderstand what you mean?

[Paul Woitaschek]

So the purpose of the script is just to print a yaml file to stout right?

[Vampire]

Basically, yes.
Though you can also directly save it to the workflow file.
And it also generates a Workflow step that compares the actual output with the present output to make sure they are in sync.
But what the script does should not really matter much, should it?

[Paul Woitaschek]

It really depends how its executed. If it is only executed as an executable you can give it a bin/bash Header

[Paul Woitaschek]

Then you could just use Gradle like ./gradlew myscript:run

[Paul Woitaschek]

In the Gradle module you would add the dependency to the workflow dsl lib

[Vampire]

If with "bin/bash Header" you mean a Shebang line, how would that help?
The precondition is, that Kotlin is not necessary to be installed / in the PATH.

[Paul Woitaschek]

It would mean that it's not interpreted as a kotlin script but a simple bash script. That would not need any kotlin installed

[Vampire]

Still don't get what you are saying.
How should a Kotlin script be sucessfully interpreted as Bash script?

[Vampire]

Can you maybe show an example of what you mean?

[Paul Woitaschek]

Just as ./mysriptm.main.kts

The contents of that script is a bin/bash Header followed by a gradlew invocation

[Vampire]

The question is how to get the Gradle task done.
The script itself does not need such header-magic.
Either you call it directly if Kotlin is installed,
or you run the Gradle task if you want to run it without Kotlin being in the path.

[Paul Woitaschek]

Why is this idea so hard to transfer? 😄
So: You move the logic out of the kts file into a real gradle module. Then you can’t need the kts file at all and can run it without kotlin installed.

[mbonnin]

Or you publish a jar with a simple main() to maven central and your main.kts becomes just 2 lines 😅

[mbonnin]

(and you call main() from Gradle)

[Paul Woitaschek]

I really don’t see the need why it needs to be a kts file.

[mbonnin]

I think it's just easier to call from Github actions and edit standalone without having to sync your whole project (but don't quote me on that)

[mbonnin]

And Github Actions has kotlin in the path

[Vampire]

So: You move the logic out of the kts file into a real gradle module. Then you can’t need the kts file at all and can run it without kotlin installed.

So you mean faking it were a *.main.kts file by naming it like that, fooling the tool calling it
and with the shebang make it a Bash script instead?
That sounds pretty ugly but would actually work in my specific case I think.

I really don’t see the need why it needs to be a kts file.

Because that is what github-actions-kotlin-dsl dictates and what actually is all the sense of it.
To have a Kotlin DSL to define a GitHub actions workflow.

Or you publish a jar with a simple main() to maven central and your main.kts becomes just 2 lines 😅

(and you call main() from Gradle)

While also pretty ugly probably, that would indeed work maybe.
Of course not published to MC, but published to a local repo that is then used.
But that seems also pretty from behind through the back into the eye as we say in German.

I think a task that just can invoke the the *.main.kts should still be the way to go.

[Paul Woitaschek]

You would still use the dsl, just not in the kts script.

[Vampire]

But I actually don't see why such a practice should be used, when it should work to just execute the script. I can also simply make an IntelliJ run configuration and run the script, so running it from a Gradle task should also work.
I'd rather put my effort in doing that than to make such a hacky work-around, but thanks for the suggestion. :-)

[Paul Woitaschek]

It's not too hacky. I'd just omit the KTS part and use a proper Gradle module

[Vampire]

But that would mean everytime it has to run, Gradle is invoked even if it were not necessary.

[Paul Woitaschek]

And is that an issue?

[Vampire]

Sure

[Vampire]

It is unnecessary work and thus wasted time

[Paul Woitaschek]

I don’t see how invoking gradle is worse than downloading and invoking kotlin, which itself also has to do the dependency resolution and compiling stuff

[Vampire]

Why downloading Kotlin?
If Kotlin is available like on GHA runners, the script can simply run.
Just when invoking it manually, it should be possible as Gradle task without pre-installed Kotlin.

[Vampire]

@ilya.chernikov I'm trying to follow your suggestion now.
What I tried is this:

val bar by configurations.creating
dependencies {
    bar(kotlin("compiler-embeddable", "1.7.0"))
    bar(kotlin("scripting-compiler-embeddable", "1.7.0"))
    bar(kotlin("main-kts", "1.7.0"))
    bar(kotlin("script-runtime", "1.7.0"))
    bar(kotlin("stdlib", "1.7.0"))
}

val foo by tasks.registering(JavaExec::class) {
    classpath(bar)
    mainClass.set(K2JVMCompiler::class.qualifiedName)
    args("-script", file(".github/workflows/test.main.kts").absolutePath)
}

This results in the call:

java -cp \
.../kotlin-compiler-embeddable-1.7.0.jar;\
.../kotlin-main-kts-1.7.0.jar;\
.../kotlin-scripting-compiler-embeddable-1.7.0.jar;\
.../kotlin-scripting-compiler-impl-embeddable-1.7.0.jar;\
.../kotlin-scripting-jvm-1.7.0.jar;\
.../kotlin-script-runtime-1.7.0.jar;\
.../kotlin-reflect-1.7.0.jar;\
.../kotlin-scripting-common-1.7.0.jar;\
.../kotlin-stdlib-1.7.0.jar;\
.../kotlin-daemon-embeddable-1.7.0.jar;\
.../trove4j-1.0.20200330.jar;\
.../jna-5.6.0.jar;\
.../kotlin-stdlib-common-1.7.0.jar;\
.../annotations-13.0.jar \
org.jetbrains.kotlin.cli.jvm.K2JVMCompiler -script .../.github/workflows/test.main.kts

Which results in

warning: unable to find kotlin-stdlib.jar in the Kotlin home directory. Pass either '-no-stdlib' to prevent adding it to the classpath, or the correct '-kotlin-home'
warning: unable to find kotlin-script-runtime.jar in the Kotlin home directory. Pass either '-no-stdlib' to prevent adding it to the classpath, or the correct '-kotlin-home'
warning: unable to find kotlin-reflect.jar in the Kotlin home directory. Pass either '-no-reflect' or '-no-stdlib' to prevent adding it to the classpath, or the correct '-kotlin-home'
error: no script runtime was found in the classpath: class 'kotlin.script.templates.standard.ScriptTemplateWithArgs' not found. Please add kotlin-script-runtime.jar to the module dependencies. (test.main.kts:3:1)

and then a whole bunch of unresolved reference errors.
Do you have any idea what is wrong?
All four mentioned jars are on the classpath.
If I add the suggested options, the warnings go away, but the errors remain.

[Vampire]

Ah, of course, I need to split the class path where the compiler is in from the class path the compiler uses to compile the script.
Here the solution for anyone still following:

val compilerClasspath by configurations.creating {
    isCanBeConsumed = false
}

val scriptClasspath by configurations.creating {
    isCanBeConsumed = false
}

dependencies {
    compilerClasspath(kotlin("compiler-embeddable", "1.7.0"))
    compilerClasspath(kotlin("scripting-compiler-embeddable", "1.7.0"))
    scriptClasspath(kotlin("main-kts", "1.7.0"))
}

val foo by tasks.registering(JavaExec::class) {
    classpath(compilerClasspath)
    mainClass.set(K2JVMCompiler::class.qualifiedName)
    args("-no-stdlib", "-no-reflect")
    args("-classpath", scriptClasspath.asPath)
    args("-script", file(".github/workflows/test.main.kts").absolutePath)
}

[Vampire]

Hm, this worked two times and now I also here get the

.github\workflows\test.main.kts: error: failed to access class org.jetbrains.kotlin.com.google.common.base.Platform from class org.jetbrains.kotlin.com.google.common.base.Ticker$1 (org.jetbrains.kotlin.com.google.common.base.Platform is in unnamed module of loader 'app'; org.jetbrains.kotlin.com.google.common.base.Ticker$1 is in unnamed module of loader java.net.URLClassLoader @5cff6d1d)

😕

[Vampire]

What the, ...
I changed the content of a string in the script and since then this error comes.
If I change the script back to the previous state, it runs fine.
If I change the script again, it fails again. o_O

[Vampire]

This doesn't make much sense to me

[ilya.chernikov]

Looks like java modules error

@Vampire, could you please file an issue in YT with the code from above? Looks like a something that we should investigate. Some callstack would help too.

Meanwhile you can first try to use non-embeddable compiler and plugin, and second, add --illegal-access=permit to the JavaExec params.

[Vampire]

I will for sure, but maybe you can help me find out what to actually report?
Why does it work with one version of the file but not with another?
Even after a computer restart the behavior is exactly the same.
So it can not be some hanging process (which I previously all killed anyway) but it has to be connected to some persistent state somewhere.
And also, I'd be happy to provide a callstack if you can tell me where to find one or how to enable it being displayed.

And no, it is indeed not a JPMS problem.
I tried to work-around it using Java 8 with javaLauncher.set(javaToolchains.launcherFor { languageVersion.set(JavaLanguageVersion.of(8)) }) and I'm still getting

.github\workflows\test.main.kts: error: tried to access class org.jetbrains.kotlin.com.google.common.base.Platform from class org.jetbrains.kotlin.com.google.common.base.Ticker$1

[Vampire]

It's more that the two classes come from different class loaders and thus Ticker is not allowed to access the package private Platform

[Vampire]

Ah, now I got a stacktrace using the debugger:

java.lang.IllegalAccessError: tried to access class org.jetbrains.kotlin.com.google.common.base.Platform from class org.jetbrains.kotlin.com.google.common.base.Ticker$1
	at org.jetbrains.kotlin.com.google.common.base.Ticker$1.read(Ticker.java:55)
	at org.jetbrains.kotlin.com.google.common.base.Stopwatch.start(Stopwatch.java:148)
	at org.jetbrains.kotlin.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3441)
	at org.jetbrains.kotlin.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2193)
	at org.jetbrains.kotlin.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2152)
	at org.jetbrains.kotlin.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2042)
	at org.jetbrains.kotlin.com.google.common.cache.LocalCache.get(LocalCache.java:3850)
	at org.jetbrains.kotlin.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3874)
	at org.jetbrains.kotlin.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4799)
	at org.jetbrains.kotlin.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4805)
	at org.jetbrains.kotlin.com.google.inject.internal.Annotations$AnnotationChecker.hasAnnotations(Annotations.java:274)
	at org.jetbrains.kotlin.com.google.inject.internal.Annotations.isBindingAnnotation(Annotations.java:340)
	at org.jetbrains.kotlin.com.google.inject.Key.ensureIsBindingAnnotation(Key.java:349)
	at org.jetbrains.kotlin.com.google.inject.Key.strategyFor(Key.java:336)
	at org.jetbrains.kotlin.com.google.inject.Key.get(Key.java:219)
	at org.jetbrains.kotlin.org.eclipse.sisu.wire.ParameterKeys.<clinit>(ParameterKeys.java:28)
	at org.jetbrains.kotlin.org.codehaus.plexus.DefaultPlexusContainer$ContainerModule.configure(DefaultPlexusContainer.java:832)
	at org.jetbrains.kotlin.com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:344)
	at org.jetbrains.kotlin.com.google.inject.spi.Elements.getElements(Elements.java:103)
	at org.jetbrains.kotlin.com.google.inject.spi.Elements.getElements(Elements.java:96)
	at org.jetbrains.kotlin.org.eclipse.sisu.wire.MergedModule.configure(MergedModule.java:53)
	at org.jetbrains.kotlin.com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:344)
	at org.jetbrains.kotlin.com.google.inject.spi.Elements.getElements(Elements.java:103)
	at org.jetbrains.kotlin.com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:137)
	at org.jetbrains.kotlin.com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103)
	at org.jetbrains.kotlin.com.google.inject.Guice.createInjector(Guice.java:87)
	at org.jetbrains.kotlin.com.google.inject.Guice.createInjector(Guice.java:69)
	at org.jetbrains.kotlin.com.google.inject.Guice.createInjector(Guice.java:59)
	at org.jetbrains.kotlin.org.codehaus.plexus.DefaultPlexusContainer.addPlexusInjector(DefaultPlexusContainer.java:481)
	at org.jetbrains.kotlin.org.codehaus.plexus.DefaultPlexusContainer.<init>(DefaultPlexusContainer.java:206)
	at org.jetbrains.kotlin.org.codehaus.plexus.DefaultPlexusContainer.<init>(DefaultPlexusContainer.java:168)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession$repositorySystem$2.invoke(aether.kt:105)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession$repositorySystem$2.invoke(aether.kt:90)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession.getRepositorySystem(aether.kt:90)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession.access$getRepositorySystem(aether.kt:47)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession$resolveDependencies$1.invoke(aether.kt:141)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession$resolveDependencies$1.invoke(aether.kt:136)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession.fetch(aether.kt:192)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession.resolveDependencies(aether.kt:136)
	at kotlin.script.experimental.dependencies.maven.impl.AetherResolveSession.resolve(aether.kt:131)
	at kotlin.script.experimental.dependencies.maven.MavenDependenciesResolver.resolve(MavenDependenciesResolver.kt:69)
	at kotlin.script.experimental.dependencies.CompoundDependenciesResolver.resolve(CompoundDependenciesResolver.kt:68)
	at kotlin.script.experimental.dependencies.AnnotationsKt.resolveFromScriptSourceAnnotations(annotations.kt:70)
	at org.jetbrains.kotlin.mainKts.MainKtsConfigurator$processAnnotations$resolveResult$1.invokeSuspend(scriptDef.kt:201)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlin.coroutines.ContinuationKt.startCoroutine(Continuation.kt:115)
	at kotlin.script.experimental.impl.RunSuspendKt.internalScriptingRunSuspend(runSuspend.kt:19)
	at org.jetbrains.kotlin.mainKts.MainKtsConfigurator.processAnnotations(scriptDef.kt:200)
	at org.jetbrains.kotlin.mainKts.MainKtsConfigurator.invoke(scriptDef.kt:153)
	at org.jetbrains.kotlin.mainKts.MainKtsConfigurator.invoke(scriptDef.kt:149)
	at kotlin.script.experimental.api.ScriptCompilationKt.refineOnAnnotations(scriptCompilation.kt:295)
	at org.jetbrains.kotlin.scripting.resolve.RefineCompilationConfigurationKt.refineScriptCompilationConfiguration(refineCompilationConfiguration.kt:236)
	at org.jetbrains.kotlin.scripting.compiler.plugin.definitions.CliScriptDependenciesProvider.calculateRefinedConfiguration(CliScriptDependenciesProvider.kt:46)
	at org.jetbrains.kotlin.scripting.compiler.plugin.definitions.CliScriptDependenciesProvider.getScriptConfigurationResult(CliScriptDependenciesProvider.kt:34)
	at org.jetbrains.kotlin.scripting.compiler.plugin.dependencies.ScriptsCompilationDependenciesKt.collectScriptsCompilationDependencies(ScriptsCompilationDependencies.kt:49)
	at org.jetbrains.kotlin.scripting.compiler.plugin.impl.CompilationContextKt.collectRefinedSourcesAndUpdateEnvironment(compilationContext.kt:302)
	at org.jetbrains.kotlin.scripting.compiler.plugin.impl.ScriptJvmCompilerImplsKt.compileImpl(ScriptJvmCompilerImpls.kt:123)
	at org.jetbrains.kotlin.scripting.compiler.plugin.impl.ScriptJvmCompilerImplsKt.access$compileImpl(ScriptJvmCompilerImpls.kt:1)
	at org.jetbrains.kotlin.scripting.compiler.plugin.impl.ScriptJvmCompilerFromEnvironment$compile$1$1.invoke(ScriptJvmCompilerImpls.kt:78)
	at org.jetbrains.kotlin.scripting.compiler.plugin.impl.ScriptJvmCompilerFromEnvironment$compile$1$1.invoke(ScriptJvmCompilerImpls.kt:67)
	at org.jetbrains.kotlin.scripting.compiler.plugin.impl.ScriptJvmCompilerImplsKt.withScriptCompilationCache(ScriptJvmCompilerImpls.kt:99)
	at org.jetbrains.kotlin.scripting.compiler.plugin.impl.ScriptJvmCompilerImplsKt.access$withScriptCompilationCache(ScriptJvmCompilerImpls.kt:1)
	at org.jetbrains.kotlin.scripting.compiler.plugin.impl.ScriptJvmCompilerFromEnvironment.compile(ScriptJvmCompilerImpls.kt:67)
	at org.jetbrains.kotlin.scripting.compiler.plugin.AbstractScriptEvaluationExtension$doEval$1.invokeSuspend(AbstractScriptEvaluationExtension.kt:135)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlin.coroutines.ContinuationKt.startCoroutine(Continuation.kt:115)
	at kotlin.script.experimental.impl.RunSuspendKt.internalScriptingRunSuspend(runSuspend.kt:19)
	at org.jetbrains.kotlin.scripting.compiler.plugin.AbstractScriptEvaluationExtension.doEval(AbstractScriptEvaluationExtension.kt:134)
	at org.jetbrains.kotlin.scripting.compiler.plugin.AbstractScriptEvaluationExtension.eval(AbstractScriptEvaluationExtension.kt:121)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:109)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:52)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:94)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:43)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:79)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:43)
	at org.jetbrains.kotlin.cli.common.CLITool$Companion.doMainNoExit(CLITool.kt:179)
	at org.jetbrains.kotlin.cli.common.CLITool$Companion.doMainNoExit$default(CLITool.kt:177)
	at org.jetbrains.kotlin.cli.common.CLITool$Companion.doMain(CLITool.kt:166)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler$Companion.main(K2JVMCompiler.kt:238)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.main(K2JVMCompiler.kt)

[Vampire]

Also happens with 1.6.20

[Vampire]

Not sure about 1.5.30, I'm getting a different error, but as far as I can tell it fails before that code would be reached

[Vampire]

So I guess why it works with the unchanged script is, because the compilation result is taken from some cache,
but if I change the script it is recompiled and on recompilation it tries to resolve the @DependsOn dependency from some local cache.
And there happens the error.

[Vampire]

Yes, if I clear the main.kts.compiled.cache and try to rerun the only working version it now also fails with the same error, so that assumption was correct

[Paul Woitaschek]

I think a simple script that pulls the kotlin cli binary would be simpler here

[Vampire]

I don't think so.
And even if it were, this bug should most probably be fixed anyway. 🙂

[Vampire]

@ilya.chernikov Any idea where that compiler / resolver stores state on disk?

[Vampire]

Ok, so here the issue: https://youtrack.jetbrains.com/issue/KT-52948/class-loader-problem-when-compiling-mainkts-scripts

[ilya.chernikov]

@Vampire thank you very much for the detailed issue. We'll try to investigate it asap.
Meanwhile I can only suggest you to try it with non-embeddable compiler and scripting plugin. My wild guess is that it caused by some classloading conflicts, so changing packages may change the situation. But I guess that more likely that we'll need to find out which dependencies should be packed into main-kts jar.

[ilya.chernikov]

Any idea where that compiler / resolver stores state on disk?

There is only two caches involved here. One is main.kts.compiled.cache, that seems you're aware of. It stores compiled script and do not perform artefact resolving, if script is not changed (or classloading error happened on attempt to run it).
And the second is the regular maven's local repo, typically '~/.m2`, since main-kts uses maven/aether resolver under the hood.

[Vampire]

Yes, about main.kts.compiled.cache I found out.
Actually I would wonder if the ~/.m2 is releated, because that was always present, as I used a custom snapshot build of the dependency I need right from the start and the one time the compile worked it was also with that, so at least ~/.m2 was present and had content.

[Vampire]

Nope, completely renamed .m2 to .m2.bak and it still fails

[ilya.chernikov]

Meanwhile I've got another idea. I will be able to experiment myself a bit later, but if you have a moment to try, I'll be grateful.
Just make dependency to main-kts intransitive:

scriptClasspath(kotlin("main-kts", "1.7.0"))  { isTransitive = false }

[ilya.chernikov]

Without it, you 're likely adding the compiler itself to the compilation classpath, and this is what may cause troubles

[ilya.chernikov]

Seems we've seen something similar in a different setting in the past - https://youtrack.jetbrains.com/issue/KT-50174/Script-failed-to-access-class-orgjetbrainskotlincomgooglecommonbasePlatform-with-mainkts-file-with-dependency#focus=Comments-27-5646417.0-0

[Vampire]

Do you have the full string notation at hand? kotlin(...) does not work with additional configuration like isTransitive = false

[ilya.chernikov]

It is the notation of configuration, not of kotlin helpert

[Vampire]

I know, but still, it will not work, there is no overload with Any as argument that takes the configuration closure

[Vampire]

No, still same error with scriptClasspath("org.jetbrains.kotlin:kotlin-main-kts:1.7.0") { isTransitive = false }

[Vampire]

What bugs me probably most is that it worked one time

[ilya.chernikov]

Could you try, for the sake of experiment, to just manually put the path to main-kts to the classpath, as you'd do if you'd run the cli compiler?
And/or print/lookup in the log the actual cli command that JavaExec runs?

[Vampire]

Btw. yes, that issue you posted seems to be exactly the same.
It is actually the same I tried first here if you look at the second post in this thread where I naively tried to compile the script as part of the normal compilation and got the same error.

[Vampire]

Could you try, for the sake of experiment, to just manually put the path to main-kts to the classpath, as you'd do if you'd run the cli compiler?

And/or print/lookup in the log the actual cli command that JavaExec runs?

This is what got run with the isTransitive = true added, that is already what you mean, isn't it?

C:\Program Files\AdoptOpenJDK\jdk-11.0.11.9-hotspot\bin\java.exe
 -Dfile.encoding=windows-1252
 -Duser.country=DE
 -Duser.language=de
 -Duser.variant
 -cp D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-compiler-embeddable\1.7.0\4d09852239f61842072b40b524882fa3caaed371\kotlin-compiler-embeddable-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-scripting-compiler-embeddable\1.7.0\69924a0f52055776049f96ee09d875233e253529\kotlin-scripting-compiler-embeddable-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-reflect\1.7.0\e660b0079fdaf744dc9cc7f6f3aa3c761ec839b0\kotlin-reflect-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-scripting-compiler-impl-embeddable\1.7.0\69ebbb984c0e9c7a9ca8a4ed0a87b36f1c793ed8\kotlin-scripting-compiler-impl-embeddable-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-scripting-jvm\1.7.0\d03ca0d4594387836eecc7c67cec679f34172b36\kotlin-scripting-jvm-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-scripting-common\1.7.0\1a19885731b40f0340c90c3edd70c157be90ac8c\kotlin-scripting-common-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-stdlib\1.7.0\a5f42c684ad9003160ef0d0f693ecf0ba7b13549\kotlin-stdlib-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-script-runtime\1.7.0\3acea1d447587e61bc3e1b3ed6227d9e9ed4ddf1\kotlin-script-runtime-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-daemon-embeddable\1.7.0\fd56798de80825c673ac93808284c5198ffa2b50\kotlin-daemon-embeddable-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.intellij.deps\trove4j\1.0.20200330\3afb14d5f9ceb459d724e907a21145e8ff394f02\trove4j-1.0.20200330.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\net.java.dev.jna\jna\5.6.0\330f2244e9030119ab3030fc3fededc86713d9cc\jna-5.6.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-stdlib-common\1.7.0\51736992f422993a1e741051bdf3c12801bc1ca1\kotlin-stdlib-common-1.7.0.jar;
D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains\annotations\13.0\919f0dfe192fb4e063e7dacadee7f8bb9a2672a9\annotations-13.0.jar
 org.jetbrains.kotlin.cli.jvm.K2JVMCompiler
 -no-stdlib
 -no-reflect
 -classpath D:\Dateien\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-main-kts\1.7.0\c18ae34753397e8d980859a4e43b63d067cf30ca\kotlin-main-kts-1.7.0.jar
 -script D:\Sourcecode\other\setup-wsl\.github\workflows\test.main.kts

[Vampire]

The problem on the other hand is probably still the same, that the Ticker is used from main-kts artifact while the Platform is used from the compiler and each from different class loaders.

[ilya.chernikov]

Yes, thank you, this is what I meant.
Ok, seems I'll have to debug it.
It looks like some layer of isolation is missing, but it remains unclear why it works from cli compiler. The cli command you extracted looks quite like something I would expect from running cli compilation.

[ilya.chernikov]

Just one last experiment, if you don't mind. Could you try to add main-kts jar to the compiler classpath, maybe even instead of the compilation classpath?

[Vampire]

If I add it to the complier classpath instead of the script classpath I get the problem described above:

error: no script runtime was found in the classpath: class 'kotlin.script.templates.standard.ScriptTemplateWithArgs' not found. Please add kotlin-script-runtime.jar to the module dependencies. (test.main.kts:5:1)

and then a whole lot of unresolved reference errors.

If I add it to both, it indeed works, probably because then in the script class path the script runtime is present, but the Ticker is used from the same class loader as `Platform`n and thus does not show the problem.


Additionally, with this it seems to work:

dependencies {
    compilerClasspath(kotlin("compiler", "1.7.0"))
    compilerClasspath(kotlin("scripting-compiler", "1.7.0"))
    scriptClasspath(kotlin("main-kts", "1.7.0"))
}

with this too:

dependencies {
    compilerClasspath(kotlin("compiler", "1.7.0"))
    compilerClasspath(kotlin("scripting-compiler", "1.7.0"))
    scriptClasspath("org.jetbrains.kotlin:kotlin-main-kts:1.7.0") { isTransitive = false }
}

and this too, which mitigates the problem that there is no overload with configuration function for `Any`:

dependencies {
    compilerClasspath(kotlin("compiler", "1.7.0"))
    compilerClasspath(kotlin("scripting-compiler", "1.7.0"))
    add(scriptClasspath.name, kotlin("main-kts", "1.7.0"), closureOf<ExternalModuleDependency> { isTransitive = false } as Closure<Any>)
}

Why did you actually suggest to use the embedded compiler instead of the compiler?

[ilya.chernikov]

Thank you, that is quite a lot of info! I guess with that we'll be able to resolve it!

[ilya.chernikov]

Why did you actually suggest to use the embedded compiler instead of the compiler?

I think I suggested a few times above to try it with regular compiler. Although it would work only with intransitive deps, I guess.
The embeddable compiler supposed to be the default for all cases, but seems we're getting a conflict in this particular scenario, which we need to fix.
I believe it works with regular compiler since now the Platform class is different - the one used in the main-kts is relocated, while the one in the compiler itself is not, so no conflict anymore.

[ilya.chernikov]

I would conclude that the best variant for now is the last one.
(I suppose that overload with Any is somewhat fishy too, but is a separate problem, and maybe more on the Gradle side).

[ilya.chernikov]

And thanks again for bringing it up, seems that we need to do something about it.

[Vampire]

I think I suggested a few times above to try it with regular compiler.

Yes, but you suggested with embedded first and had other suggestions to try and I am also doing my dayjob while trying these things alongside. 😉

Although it would work only with intransitive deps, I guess.

No, as written above, it worked with transitive main-kts.

I believe it works with regular compiler since now the Platform class is different

Ah, I see, makes sense.

(I suppose that overload with Any is somewhat fishy too, but is a separate problem, and maybe more on the Gradle side).

I think so too

And thanks again for bringing it up, seems that we need to do something about it.

Always a pleasure to complain 😄

[Vampire]

Thanks for your thorough help in investigating and fixing