I tried to use 1.3.72 JSR223 artifact from a Gradl...

# scripting

[Vampire]

I tried to use 1.3.72 JSR223 artifact from a Gradle script (don't ask why, needed blaaaaack magic if it works as I imagine)
Unfortunately I didn't yet get it working.
I added

buildscript {
    dependencies {
        classpath(kotlin("scripting-jsr223", "1.3.72"))
    }
}

And then I tried to do

println("kt: " + javax.script.ScriptEngineManager().getEngineByExtension("kt"))

but it does not work.
The output is

ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider kotlin.script.experimental.jsr223.KotlinJsr223DefaultScriptEngineFactory could not be instantiated
kt: null

If I use the debugger, I can extract this stacktrace:

java.util.ServiceConfigurationError: javax.script.ScriptEngineFactory: Provider kotlin.script.experimental.jsr223.KotlinJsr223DefaultScriptEngineFactory could not be instantiated
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:581)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:803)
	at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:721)
	at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1394)
	at java.scripting/javax.script.ScriptEngineManager.initEngines(ScriptEngineManager.java:125)
	at java.scripting/javax.script.ScriptEngineManager.init(ScriptEngineManager.java:87)
	at java.scripting/javax.script.ScriptEngineManager.<init>(ScriptEngineManager.java:62)
	at Build_gradle.<init>(build.gradle.kts:118)
    [...]
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalArgumentException: Unable to construct script definition: Unable to load base class kotlin.script.experimental.api.KotlinType@428395ed
	at kotlin.script.experimental.host.ConfigurationFromTemplateKt.getTemplateClass(configurationFromTemplate.kt:101)
	at kotlin.script.experimental.host.ConfigurationFromTemplateKt.createCompilationConfigurationFromTemplate(configurationFromTemplate.kt:37)
	at kotlin.script.experimental.jsr223.KotlinJsr223DefaultScriptEngineFactory.<init>(KotlinJsr223DefaultScriptEngineFactory.kt:74)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:779)
	... 168 more
Caused by: java.lang.IllegalArgumentException: unable to load class kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript
	at kotlin.script.experimental.jvm.JvmGetScriptingClass.invoke(jvmScriptingHostConfiguration.kt:111)
	at kotlin.script.experimental.jvm.JvmGetScriptingClass.invoke(jvmScriptingHostConfiguration.kt:64)
	at kotlin.script.experimental.host.ConfigurationFromTemplateKt.getTemplateClass(configurationFromTemplate.kt:99)
	... 175 more
Caused by: java.lang.ClassNotFoundException: kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at kotlin.script.experimental.jvm.JvmGetScriptingClass.invoke(jvmScriptingHostConfiguration.kt:109)
	... 177 more

Anyone have any idea?
Or is there maybe onther way to use Kotlin scripting?
Actually what I need is, I have a Kotlin file with an object expression and I need that object expression as reference.
But as it is in buildSrc/build.gradle.kts I have no compile task where I could throw this in and it is 1. too long to just paste it into the build script and I 2. need it in a later phase too and wanted to avoid copy&paste

[ilya.chernikov]

You need to request for the kotlin script rather that regular kotlin

val engine = ScriptEngineManager().getEngineByExtension("kts")

Here is an example:
https://github.com/Kotlin/kotlin-script-examples/tree/1.4.0/jvm/jsr223/jsr223-simple
(and in this repo you can find other examples and ways to use Kotlin scripting.)

[Vampire]

I tried both, that made no difference.

[Vampire]

Actually I solved my concrete use-case now using the K2JVMCompiler 🙂
But the JSR223 I didn't get to work.

[ilya.chernikov]

I guess that with the 1.3.x you have to add a dependency to the kotlin-scripting-compiler (or -embeddable). It should be fixed in the 1.4 - see the example I linked above.

[ilya.chernikov]

K2JVMCompiler is an internal stuff, so you're on your own with using it 🙂

[Vampire]

Actually I finally found that I need neither, as the class is already compiled in 99.8% of the cases and I can just load it in a new class loader.
So it now is just a fail-safe fallback in case the class was not compiled yet.
Maybe I'll give the the JSR223 version another chance, will let you know how it went, thank. 🙂

[Vampire]

Adding classpath(kotlin("scripting-compiler", "1.3.72")) did not help, still the same

[ilya.chernikov]

I do not have a working copy of 1.3.72 at the moment, so sorry for the trial and error approach.
If you do not want to switch to 1.4.0, you can check the deps here - https://github.com/JetBrains/kotlin/blob/1.3.70/libraries/examples/kotlin-jsr223-local-example/build.gradle.kts, and I guess that kotlin-script-util and kotlin-compiler could be missing as well.

[Vampire]

I'm fine with trial and error.
I cannot switch to 1.4.0 unfortunately.
I need to execute this in a build script and Gradle is not yet on 1.4.0, so I have to stick with 1.3.72 for the time being.

[Vampire]

Actually the class it complains about (kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript) is part of the scripting-jsr223 artifact, so I doubt that throwing more dependencies at it will help. 😕

[ilya.chernikov]

Oh, I vaguely remember that the script def class was in the script-util before, but maybe it was even earlier. Everything is moving too fast these days. 🙂
Do you have a chance to debug and see what's going on it the classloader?

[ilya.chernikov]

Or first please try to set Thread.currentThread.contextClassloader to the one that contains jsr223 and dependent jars.

[Vampire]

https://search.maven.org/search?q=fc:kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript%20and%20v:1.3.72

[Vampire]

I can debug, yes, otherwise I wouldn't be able to have shown the stack trace above

[Vampire]

I tried

val contextClassLoader = Thread.currentThread().contextClassLoader
println(contextClassLoader)
println(buildscript.classLoader)
println()
buildscript.classLoader.classPathFromTypicalResourceUrls().toList().forEach { println(it) }
println()
Thread.currentThread().contextClassLoader = buildscript.classLoader
try {
    println("kts: " + javax.script.ScriptEngineManager().getEngineByExtension("kts"))
    println("kt: " + javax.script.ScriptEngineManager().getEngineByExtension("kt"))
} finally {
    Thread.currentThread().contextClassLoader = contextClassLoader
}

output:

[Vampire]

VisitableURLClassLoader(ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings:settings:D:\Sourcecode\other\setup-wsl\buildSrc\buildSrc:root-project:kotlin-dsl:D:\Sourcecode\other\setup-wsl\buildSrc\build.gradle.kts:Project/TopLevel/
stage2(local)})
VisitableURLClassLoader(ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings:settings:D:\Sourcecode\other\setup-wsl\buildSrc\buildSrc:root-project(export)})

D:\Dateien\.gradle\caches\jars-8\b883dedb662063116c472e5e749c5b43\kotlinx-serialization-runtime-0.20.0.jar
D:\Dateien\.gradle\caches\jars-8\65a5c576b99f7982c5a592db4ec064f2\kotlin-scripting-jsr223-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\67163e1fb1e0bc1d8ef73fad58db583d\kotlin-scripting-jvm-host-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\c386c9acbaf9313653fce692658b5bb6\kotlin-scripting-compiler-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\0f98f7ea9c84133daa570a5c62d9d01e\kotlin-scripting-compiler-impl-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\58801c77577e98953cdc0f147bbc14f3\plugins-1.3.6.jar
D:\Dateien\.gradle\caches\jars-8\26543e9f0317b8ef642f3b865695b490\kotlin-gradle-plugin-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\96d43f4b578c2bb0a6d3a1f391dbf27d\kotlin-scripting-compiler-embeddable-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\e64d5c93ec6cfd12d70622b5c38e05ff\kotlin-scripting-compiler-impl-embeddable-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\e86a08ea059134c93346fe9cd3f77780\kotlin-scripting-jvm-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\bbcd8fe69b42d688df7c839797b9c3a1\kotlin-scripting-js-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\6150e5b18a00e3e58367b0d8aa04fca6\kotlin-scripting-common-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\5fa4c74c8df811d76c0c9a182a2058a8\kotlin-compiler-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\e3de0ed092eb12a8ba3dd91506356ec7\kotlin-annotation-processing-gradle-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\2a2264afeebcf1156782f8b2298747a6\kotlin-android-extensions-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\61f9c71a9cb8e4b18fee24d709dd7e17\kotlin-compiler-embeddable-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\6d5f00ccd2daf646c123008edbdef725\kotlin-compiler-runner-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\5485d5b4fa8b358f9fe5f0fe23ac75f5\kotlin-daemon-client-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\dde97575a599bd3e140dd938a7aed4fa\kotlin-reflect-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\5e7ae8e9ff6118cddee5c4c9f39ea643\kotlin-util-klib-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\05dcf298297ea4c8fe60747f04eeb869\kotlin-sam-with-receiver-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\768cf1021ee3bdc425b9b2ae1289f2f1\kotlin-stdlib-jdk8-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\00956b1df5c24bc6c04134f5f3e1972c\kotlin-gradle-plugin-api-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\2b7e8cce52f6b02cc39b5246900ac753\kotlin-native-utils-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\1103ede48d4c80cd1adc2bc524ea2697\kotlin-util-io-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\24680c452c59a7b774f8e8693fd522d7\kotlin-gradle-plugin-model-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\421b0c68a25f5d684ea66be525dfcf66\kotlin-stdlib-jdk7-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\70d15c94187de212f9a927dc262ddd63\kotlin-stdlib-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\18e6bc6874baa1a066bce674522f54bf\kotlin-stdlib-common-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\35ae3e977f0e11aacd451b5809ae98df\annotations-13.0.jar
D:\Dateien\.gradle\caches\jars-8\9a69e50d968b98e549651f54a236428a\kotlin-script-runtime-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\75af85598d36daa7a426f554c66b38f7\kotlin-serialization-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\2245aa47df9b85a88a4d7b46c0e2f53d\gradle-versions-plugin-0.29.0.jar
D:\Dateien\.gradle\caches\jars-8\72979cf2c614176ae726ca3246e21ddb\kotlinx-coroutines-core-1.2.1.jar
D:\Dateien\.gradle\caches\jars-8\5596c27e4f2e2aa07aa27f9ea93c18e9\trove4j-1.0.20181211.jar
D:\Dateien\.gradle\caches\jars-8\ff2ed1dd925e7d97edfea4bb319d6cf7\xstream-1.4.10.jar
D:\Dateien\.gradle\caches\jars-8\ced35abbe53a6b717d7d03f7b154ff9b\gson-2.8.5.jar
D:\Dateien\.gradle\caches\jars-8\6c870431a36108f92c23842863194b2e\gradle-download-task-4.0.2.jar
D:\Dateien\.gradle\caches\jars-8\9af3cb638beebff17f5f462a77fa544f\xmlpull-1.1.3.1.jar
D:\Dateien\.gradle\caches\jars-8\d42a61ddad53b1a52b9172784cb57290\xpp3_min-1.1.4c.jar
D:\Dateien\.gradle\caches\jars-8\a434bb1ed18cc90f2347c31f61598922\kotlin-daemon-embeddable-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\856225202493023c969c0f8e40821e37\kotlin-build-common-1.3.72.jar

ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider kotlin.script.experimental.jsr223.KotlinJsr223DefaultScriptEngineFactory could not be instantiated
kts: null
ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider kotlin.script.experimental.jsr223.KotlinJsr223DefaultScriptEngineFactory could not be instantiated
kt: null

[Vampire]

and I tried

val contextClassLoader = Thread.currentThread().contextClassLoader
val defaultScriptClass = Class.forName("kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript")
val jsr223ClassLoader = defaultScriptClass.classLoader
println(contextClassLoader)
println(jsr223ClassLoader)
println()
buildscript.classLoader.classPathFromTypicalResourceUrls().toList().forEach { println(it) }
println()
println(Class.forName("kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript"))
Thread.currentThread().contextClassLoader = jsr223ClassLoader
try {
    println("kts: " + javax.script.ScriptEngineManager().getEngineByExtension("kts"))
    println("kt: " + javax.script.ScriptEngineManager().getEngineByExtension("kt"))
} finally {
    Thread.currentThread().contextClassLoader = contextClassLoader
}

output:

[Vampire]

Interestingly println(kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript) does not compile but complains that script cannot be found. (Unresolved reference: script)

[Vampire]

VisitableURLClassLoader(ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings:settings:D:\Sourcecode\other\setup-wsl\buildSrc\buildSrc:root-project:kotlin-dsl:D:\Sourcecode\other\setup-wsl\buildSrc\build.gradle.kts:Project/TopLevel/
stage2(local)})
VisitableURLClassLoader(ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings:settings:D:\Sourcecode\other\setup-wsl\buildSrc\buildSrc:root-project(export)})

D:\Dateien\.gradle\caches\jars-8\b883dedb662063116c472e5e749c5b43\kotlinx-serialization-runtime-0.20.0.jar
D:\Dateien\.gradle\caches\jars-8\65a5c576b99f7982c5a592db4ec064f2\kotlin-scripting-jsr223-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\67163e1fb1e0bc1d8ef73fad58db583d\kotlin-scripting-jvm-host-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\c386c9acbaf9313653fce692658b5bb6\kotlin-scripting-compiler-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\0f98f7ea9c84133daa570a5c62d9d01e\kotlin-scripting-compiler-impl-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\58801c77577e98953cdc0f147bbc14f3\plugins-1.3.6.jar
D:\Dateien\.gradle\caches\jars-8\26543e9f0317b8ef642f3b865695b490\kotlin-gradle-plugin-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\96d43f4b578c2bb0a6d3a1f391dbf27d\kotlin-scripting-compiler-embeddable-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\e64d5c93ec6cfd12d70622b5c38e05ff\kotlin-scripting-compiler-impl-embeddable-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\e86a08ea059134c93346fe9cd3f77780\kotlin-scripting-jvm-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\bbcd8fe69b42d688df7c839797b9c3a1\kotlin-scripting-js-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\6150e5b18a00e3e58367b0d8aa04fca6\kotlin-scripting-common-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\5fa4c74c8df811d76c0c9a182a2058a8\kotlin-compiler-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\e3de0ed092eb12a8ba3dd91506356ec7\kotlin-annotation-processing-gradle-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\2a2264afeebcf1156782f8b2298747a6\kotlin-android-extensions-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\61f9c71a9cb8e4b18fee24d709dd7e17\kotlin-compiler-embeddable-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\6d5f00ccd2daf646c123008edbdef725\kotlin-compiler-runner-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\5485d5b4fa8b358f9fe5f0fe23ac75f5\kotlin-daemon-client-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\dde97575a599bd3e140dd938a7aed4fa\kotlin-reflect-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\5e7ae8e9ff6118cddee5c4c9f39ea643\kotlin-util-klib-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\05dcf298297ea4c8fe60747f04eeb869\kotlin-sam-with-receiver-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\768cf1021ee3bdc425b9b2ae1289f2f1\kotlin-stdlib-jdk8-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\00956b1df5c24bc6c04134f5f3e1972c\kotlin-gradle-plugin-api-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\2b7e8cce52f6b02cc39b5246900ac753\kotlin-native-utils-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\1103ede48d4c80cd1adc2bc524ea2697\kotlin-util-io-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\24680c452c59a7b774f8e8693fd522d7\kotlin-gradle-plugin-model-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\421b0c68a25f5d684ea66be525dfcf66\kotlin-stdlib-jdk7-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\70d15c94187de212f9a927dc262ddd63\kotlin-stdlib-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\18e6bc6874baa1a066bce674522f54bf\kotlin-stdlib-common-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\35ae3e977f0e11aacd451b5809ae98df\annotations-13.0.jar
D:\Dateien\.gradle\caches\jars-8\9a69e50d968b98e549651f54a236428a\kotlin-script-runtime-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\75af85598d36daa7a426f554c66b38f7\kotlin-serialization-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\2245aa47df9b85a88a4d7b46c0e2f53d\gradle-versions-plugin-0.29.0.jar
D:\Dateien\.gradle\caches\jars-8\72979cf2c614176ae726ca3246e21ddb\kotlinx-coroutines-core-1.2.1.jar
D:\Dateien\.gradle\caches\jars-8\5596c27e4f2e2aa07aa27f9ea93c18e9\trove4j-1.0.20181211.jar
D:\Dateien\.gradle\caches\jars-8\ff2ed1dd925e7d97edfea4bb319d6cf7\xstream-1.4.10.jar
D:\Dateien\.gradle\caches\jars-8\ced35abbe53a6b717d7d03f7b154ff9b\gson-2.8.5.jar
D:\Dateien\.gradle\caches\jars-8\6c870431a36108f92c23842863194b2e\gradle-download-task-4.0.2.jar
D:\Dateien\.gradle\caches\jars-8\9af3cb638beebff17f5f462a77fa544f\xmlpull-1.1.3.1.jar
D:\Dateien\.gradle\caches\jars-8\d42a61ddad53b1a52b9172784cb57290\xpp3_min-1.1.4c.jar
D:\Dateien\.gradle\caches\jars-8\a434bb1ed18cc90f2347c31f61598922\kotlin-daemon-embeddable-1.3.72.jar
D:\Dateien\.gradle\caches\jars-8\856225202493023c969c0f8e40821e37\kotlin-build-common-1.3.72.jar

class kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript
ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider kotlin.script.experimental.jsr223.KotlinJsr223DefaultScriptEngineFactory could not be instantiated
kts: null
ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider kotlin.script.experimental.jsr223.KotlinJsr223DefaultScriptEngineFactory could not be instantiated
kt: null

[Vampire]

Ok, the ClassLoader used to lookup that class only contains the Gradle shipped jars

[ilya.chernikov]

You're doing it inside gradle. That's probably the root of the problem.
The gradle is known to fiddle with the classloading, in particular it filters out some kotlin jars, because of the kts build scripts API.
It is interesting to see the classloader of the ScriptEngineFactory class and try to load KotlinJsr223DefaultScript from it. I suspect you'll get exactly the same error.

[ilya.chernikov]

The usage of the JSR-223 in the gradle plugin is something I never actually tried.

[Vampire]

Well, that's where I need it unfortunately.
Or well, I would be happy with any way to come from a kt file to the object inside dynamically, or at least to the class as from class on I already have a working way

[Vampire]

Does not necessarily have to be JSR-223

[ilya.chernikov]

Do you need it for the build script itself or in the project?

[Vampire]

For the build script

[ilya.chernikov]

You can try to write something as simple as

val compilationConfiguration = ScriptCompilationConfiguration {
        jvm {
            dependenciesFromCurrentContext(wholeClasspath = true)
        }
    }
BasicJvmScriptingHost().eval(scriptText.toScriptSource(), compilationConfiguration, null)

using kotlin-scripting-jvmhost as a dependency. But it may actually face exactly the same problem.
If so, I'm out of the simple ideas to try and probably need to do some experiments myself.

[Vampire]

It is interesting to see the classloader of the ScriptEngineFactory class and try to load KotlinJsr223DefaultScript from it. I suspect you'll get exactly the same error.

That works.
I'm currently in a breakpoint, having the stack frame of the constructor of KotlinJsr223DefaultScriptEngineFactory line 38 selected.
If I execute this in the evaluation dialog, I get the engine factory.
If I execute this.getClass().getClassLoader().loadClass("kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript") I get the loaded KotlinJsr223DefaultScript class.

[Vampire]

Actually it is even fancier (didn't try your BasicJvmScriptingHost example yet).
I'm currently at a breakpoint in JvmGetScriptingClass / jvmScriptingHostConfiguration.kt method invoke line 96, the if (classLoader == null) { line where then the classLoader is determined to be baseClassLoader or a new one with dependencies and baseClassLoader as parent.
There fromClass actually is a KClass<KotlinJsr223DefaultScript> so it is loaded already.

[Vampire]

It is like this:

[Vampire]

That is from generateSequence(fromClass.java.classLoader) { it.parent }.toList()

[Vampire]

And the contextClassLoader is not the TCCL, but the class loader of the class kotlin.script.experimental.api.ScriptCompilationConfiguration

[Vampire]

Which is the ant-and-gradle-loader, as it is part of kotlin-scripting-common-1.3.72.jar which is part of the Gradle distribution. And because of that, it cannot find the default script class that is lower in the hierarchy

[Vampire]

So it seems you cannot get JSR223 Kotlin working if KotlinJsr223DefaultScript is lower in the classloading hierarchy than ScriptCompilationConfiguration.

[ilya.chernikov]

Looks like the problem. But in this case the chance to get the BasicJvmScriptingHost example up and running are quite good, even if it will show the same problem, there is a way to configure classloading via the configuration.

[Vampire]

Is the check

val fromClass = classType.fromClass
        if (fromClass != null) {
            if (fromClass.java.classLoader == null) return fromClass // root classloader
            val actualClassLoadersChain = generateSequence(contextClassLoader) { it.parent }
            if (actualClassLoadersChain.any { it == fromClass.java.classLoader }) return fromClass
        }

actually correct?
It checks whether fromClass is either in the root class loader or whether the fromClass class loader is the same or a parent of the ScriptCompilationConfiguration class loader.
Shouldn't the check be reversed, whether the ScriptCompilationConfiguration class loader is the same or a parent of the fromClass class loader?

[ilya.chernikov]

Although looking at the code, I still think that you can fix it by setting TCCL to the right value before requesting the engine. But I need to try it myself to be sure.

[ilya.chernikov]

actually correct?

let me check...

[Vampire]

I tried setting the TCCL and it had no effect

[Vampire]

And additionally setting System.setProperty("kotlin.jsr223.experimental.resolve.dependencies.from.context.classloader", "true") also does not work

[Vampire]

Oh, I'm stupid:

Interestingly println(kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript) does not compile but complains that script cannot be found. (Unresolved reference: script)

That's not interesting at all.
Besides that it is not valid syntax but should have been println(kotlin.script.experimental.jsr223.KotlinJsr223DefaultScript::class), the reason it complains abou tunresolved reference is,
that you guys had the great idea to have the package kotlin and the project extension kotlin.
And it seems project extensions beats FQCN, is that expected?
You basically cannot use a FQCN starting with kotlin in a Gradle script as it is always resolved to the extension instead of the package.
If I instead import the class, I can reference it just fine in the script.
Is that intended?

[Vampire]

I now tried your other suggestion with the BasicScriptingHost but it misses coroutines o_O

java.lang.NoClassDefFoundError: kotlinx/coroutines/BuildersKt
        at kotlin.script.experimental.host.BasicScriptingHost.runInCoroutineContext(BasicScriptingHost.kt:35)
        at kotlin.script.experimental.host.BasicScriptingHost.eval(BasicScriptingHost.kt:45)
        at Build_gradle.<init>(build.gradle.kts:79)
        [...]
Caused by: java.lang.ClassNotFoundException: kotlinx.coroutines.BuildersKt
        ... 162 more

[Vampire]

kotlinx.coroutines.BuildersKt is in the artifact main-kts? o_O

[Vampire]

Hm, and also in org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.8, but adding that to the buildscript classpath does not help
So where do I need to add it? You said classpath is configurable. 🙂

[ilya.chernikov]

You basically cannot use a FQCN starting with kotlin in a Gradle script as it is always resolved to the extension instead of the package.

This is a known problem of the gradle kotlin DSL, I think it is documented somewhere. Worse is that java is also an extension.

[ilya.chernikov]

I'm not sure kotlinx-coroutines-core v 1.3.8 is compatible, it is better to take 1.3.7, I guess.

[ilya.chernikov]

To configure the classpath and classloader you need to write something like:

val myHostConfiguration = defaultJvmScriptingHostConfiguration.with {
            jvm {
                baseClassLoader(...)
            }
        }
val compilationConfiguration = ScriptCompilationConfiguration {
        jvm {
            dependenciesFromCurrentContext(wholeClasspath = true)
        }
        hostConfiguration.update { muHostConfiguration }
    }
...

[Nikky]

i also tried to do this from gradle a long time ago.. i opted to javaExec a shadowJar of my library and pass it the info it needed to execute.. but this way i also never got the class back.. just the result via stdout

[ilya.chernikov]

Too late for me for today to think clearly, I'll try to wrap my head about this problem tomorrow.

[Vampire]

What I tried so far is replacing dependenciesFromCurrentContext(wholeClasspath = true) by dependenciesFromCurrentContext(wholeClasspath = true) but that didn't change anything. Probably because it is not my class that misses coroutines, but the BasicScriptingHost.

[Vampire]

@Nikky I already have a working verison.
I just learned from Ilya that it is not a public API to use the K2JVMCompiler.
So I'm hoping to get a supported version working with his help.

[Vampire]

And no,

val compilationConfiguration = ScriptCompilationConfiguration {
        jvm {
            dependenciesFromCurrentContext(wholeClasspath = true)
        }
        hostConfiguration.update {
            defaultJvmScriptingHostConfiguration.with {
                jvm {
                    baseClassLoader(buildscript.classLoader)
                }
            }
        }
    }

didn't work either, still same error 😞

[Vampire]

I still fear no configuration will help as it is the BasicScriptingHost that wants to call runBlocking from kotlinx.coroutines.BuildersKt, not the class I'm trying to compile.

[Vampire]

I guess it is basically the same or a similar problem.
BasicScriptingHost is in kotlin-scripting-common which is shipped with Gradle.
When it needs another class, in this case BuildersKt it looks for it in the class loader where it was loaded from and there it does not find it as the coroutines artifact is not shipped with Gradle.

[Vampire]

Is JvmScriptCompiler public supported API and can I go on from there?
Because I was able to successfully do

runBlocking {
        println(JvmScriptCompiler()(file("src/main/kotlin/net/kautler/dao/ResultSerializer.kt").readText().toScriptSource(), compilationConfiguration).valueOrThrow().getClass(null).valueOrThrow())
    }

[Vampire]

Yay, I got it working. Would be nice if you would have an approving look whether it looks fine now and also is public API and not internaly like with K2JVMCompiler 🙂

[Vampire]

runBlocking {
    BasicJvmScriptingHost().run {
        compiler(
                """
                    ${file("src/main/kotlin/net/kautler/dao/ResultSerializer.kt").readText()}
                    ResultSerializer
                """.toScriptSource(),
                ScriptCompilationConfiguration {
                    jvm {
                        dependenciesFromCurrentContext(wholeClasspath = true)
                    }
                }
        ).onSuccess { evaluator(it) }
    }
}
        .valueOrThrow()
        .returnValue
        .let { it as Value }
        .value
        as KSerializer<Result>

[ilya.chernikov]

This looks correct, and as far as I understand the only difference with the my initial proposed code is that instead of using host's eval, you're calling runBlocking with separate calls to the compiler and evaluator yourself, which allows you to avoid coroutines classloading issue. And this is fine. I'm glad that it works.
Some minor comments:
- you probably want to replace dependenciesFromCurrentContext(wholeClasspath = true) with more precise CP calculation. The whole classpath in gradle is pretty big and will result in some heavy script compilation overheads, and probably your scripts does not need all of it. something like

dependenciesFromCurrentContext("kotlin-stdlib", "<jar-with-KSerializer>", "<other required dependency jar name without version>")

should work better.
- it is not necessary to use BasicJvmScriptingHost, you can create compiler and evaluator directly

[ilya.chernikov]

And I have to investigate the issue with classloading in gradle. I created an issue - https://youtrack.jetbrains.com/issue/KT-41412, if you have anything to add - especially if you have links to projects or other repro - please add there.
And thank you for reporting and investigating!

[Vampire]

This looks correct, and as far as I understand the only difference with the my initial proposed code is that instead of using host's eval, you're calling runBlocking with separate calls to the compiler and evaluator yourself, which allows you to avoid coroutines classloading issue. And this is fine. I'm glad that it works.

Yes, that's basically it. 🙂

- you probably want to replace dependenciesFromCurrentContext(wholeClasspath = true) with more precise CP calculation. The whole classpath in gradle is pretty big and will result in some heavy script compilation overheads, and probably your scripts does not need all of it. something like

Well, performance is not that relevant anymore as it is only for the 0.02% case, but I want it clean, so I adopted it, thanks.

it is not necessary to use BasicJvmScriptingHost, you can create compiler and evaluator directly

also adopted, this is the current version now:

runBlocking {
    JvmScriptCompiler(defaultJvmScriptingHostConfiguration)(
            """
                ${file("src/main/kotlin/net/kautler/dao/ResultSerializer.kt").readText()}
                ResultSerializer
            """.toScriptSource(),
            ScriptCompilationConfiguration {
                jvm {
                    dependenciesFromCurrentContext(
                            "gradle-versions-plugin",
                            "kotlinx-serialization-runtime",
                            "groovy-all"
                    )
                }
            }
    ).onSuccess { BasicJvmScriptEvaluator()(it) }
}
        .valueOrThrow()
        .returnValue
        .let { it as Value }
        .value
        as KSerializer<Result>

if you have anything to add - especially if you have links to projects or other repro - please add there.

All information I had / have are included here, including the reproducing example

And thank you for reporting and investigating!

Always a pleasure, thanks for helping me investigating and properly solving it. 🙂

[neugartf]

Thanks for this thread, this helped me a lot 🙇 . It is unfortunately still an issue.