Hello Scripters! With moving my own DSL project to...

# scripting

[Olaf Gottschalk]

Hello Scripters!
With moving my own DSL project to Kotlin 2.1 (coming from 1.9), I experience some problems with the tooling setup, specifically with IntelliJ offering syntax highlight and code completion on my own script files with their own extension.

It used to work like this:
In my DSL library project, there is a class with a @KotlinScript annotation like this:

const val TRALA_FILE_EXTENSION = "trala.kts"

@KotlinScript(
    displayName = "TraLa Script",
    fileExtension = TRALA_FILE_EXTENSION,
    compilationConfiguration = TraLaScriptConfiguration::class
)
abstract class TraLaScript

plus, there is a resource file at this location:
resources/META-INF/kotlin/script/templates/com.bmw.transformation.scripting.TraLaScript (which is empty)

The compilation configuration looks like this:

@Suppress("JavaIoSerializableObjectMustHaveReadResolve")
object TraLaScriptConfiguration : ScriptCompilationConfiguration(
    {
        defaultImports(traLaDefaultImports)

        ide {
            acceptedLocations(ScriptAcceptedLocation.Everywhere)
        }

        jvm {
            jvmTarget("17")
            compilerOptions("-Xcontext-receivers")
        }
    }
)

The idea is that any project that uses my DSL library can have files anywhere (see acceptedLocations with the extension .trala.kts and the editor will offer syntax highlight and code completion / inspection.

This used to work. Sometimes I had to restart IntelliJ... it was never straight forward. But it worked.

Now, I cannot see any way to make the editor behave like I want. And I do not know what changed.
In projects that DEPEND on my lib (with this setup), nothing works any more.
In my library project itself, the default imports don't seem to work anymore! When I add a new file foo.trala.kts and start typing code, it inserts import statements that it should not, because they are part of the default imports...

What am I doing wrong here? Is there any good documentation on these matters?

Thanks!
Olaf

[ilya.chernikov]

I'm sorry for the late response.

You may try to add isStandalone(false) to your ScriptCompilationConfiguration. Although this is an old change - scripts with definitions without it are ignored in source roots by default. So it is quite suspicious that you're seeing it only now, but maybe something else was at play. For example there is a workaround with -Xallow-any-scripts-in-source-roots compiler flag.

[ilya.chernikov]

If it doesn't help, please create an YT issue - maybe it's an IDE bug after all.

[Olaf Gottschalk]

Hi Ilya! I tried your isStandalone(false) but still, nothing changes. The scripts are not getting their default imports at all, they are basically completely broken in IntelliJ. I would like to open a YT issue, but can it really be I am the only one who uses this? Is there any kind of documentation I can read again, like a demo project that showcases this rather odd setup with the "magic" ScriptCompilationConfiguration class and the magic resource file META-INF/kotlin/script/templates/...? I would like to test that in order to even write a proper YT issue as I cannot post all of my script language code. Thanks!

[ilya.chernikov]

Hi Olaf!
The combination should be supported in principle, because it is used for the Gradle precompiled scripts, that work fine with K2.
But I'm not sure that we tested it thoroughly, so it maybe indeed be a bug here that surfaces in your case.
In general, compiling scripts with other sources (via e.g. isStandalone(false)) is considered quite an advanced case, where you need to understand quite well how your scripts will interact with the other sources in the project.
Therefore, we hide it under the opt-ins, and thus, we hope, there are not many users who use this.
Anyway, I recommend you to file an issue as is, using just the description you gave above, and we'll test the functionality on our tests and come back to you on it later.
cc: @Vladislav Koshkin

[Olaf Gottschalk]

Hi @ilya.chernikov, turns out that everything works as expected once I switch off K2 Mode in IntelliJ... 😞 With K2 Mode enabled, my scripts are not rendering/working as before...

[Vladislav Koshkin]

hi, I'll also take a look into that this week loading

[Olaf Gottschalk]

@ilya.chernikov @Vladislav Koshkin I created a demo project to showcase exactly what problem I am facing. Before I create an issue, could you please have a look???
https://github.com/Zordid/kotlin-demo-dsl

This replicates exactly what I do in a bigger DSL. The "mysterious" resource file that goes by the name of the @KotlinScript annotated file is recognized only when not in K2 mode. I also added a readme with two images showing exactly how it looks on my screen with the two different modes!

Thanks!

[Vladislav Koshkin]

I've finally managed to get it working
1. so I cloned your project, made Gradle > Build
2. made sure there is marker annotation in resource folder under Gradle build dir
3. after that I mentioned the error by IDEA:

[KOTLIN_SCRIPTING] Cannot load script definition class demodsl.MyDslScript

java.lang.UnsupportedClassVersionError: demodsl/MyDslScript has been compiled by a more recent version of the Java Runtime (class file version 65.0), this version of the Java Runtime only recognizes class file versions up to 61.0
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at com.intellij.util.lang.UrlClassLoader.consumeClassData(UrlClassLoader.java:303)
	at com.intellij.util.lang.FileLoader.findClass(FileLoader.java:187)
	at com.intellij.util.lang.ClassPath.findClassInLoader(ClassPath.java:240)
	at com.intellij.util.lang.ClassPath.findClass(ClassPath.java:190)

4. so I just removed

kotlin {
    jvmToolchain(21)
}

and rebuild the project
there is also an action in Kotlin Scripting Settings that allow you to force re-scan your classpath, it was handy in that case

[Vladislav Koshkin]

action point for us in that case - to add more robust logging

[Olaf Gottschalk]

Wow, so my aim to be "modern" in the toolset to build crashed silently! 😉 Where did you find the exception that revealed this?
And why can I get this to work again by disabling K2 mode? How does that interact with one another? Need to go and try it today! Thanks so far! Will come back to you asap!

[Olaf Gottschalk]

Okay, by removing the Kotlin jvmToolChain, I changed the situation quite a bit! ...but still not without problems in my real-world library...

First, I see that the script definition now appears and is enabled! This is good!

But in my demo script, the default imports don't work properly... and I cannot even manually import my functions... normal Kotlin like println works, but nothing from my project is available, even when trying to import manually, even though the code completion offers my packages...

Any idea why that is?

I wanted to attach screenshots, but.... Slack does not accept them, they all get a red exclamation mark...

[Olaf Gottschalk]

As soon as I disable K2 mode, it works...

[Olaf Gottschalk]

This is what K1 mode looks like...

[Olaf Gottschalk]

Switch to K2 mode and nothing works anymore:

[Olaf Gottschalk]

removing the jvmToolChain(21) allowed me to see that my scripting is now known (no matter if K2 mode is on or off)

[Olaf Gottschalk]

@Vladislav Koshkin okay. I got behind this now, but I cannot make any sense about how the K2 mode affects this!

I have my scripting defined in the main source set. But the screenshots you see from my scripts failing, are scripts that reside in the test source set! With K2 mode enabled, I cannot access the DSL code from main source set! As soon you disable K2 mode, the scripts can be in any location and work, with K2 mode, they only work when within the main source set!

Why?!?

[Olaf Gottschalk]

I managed to showcase the very same thing in the demo DSL... this is how it looks with K2 mode enabled when you put the exact same script in main and in test source set, side by side:

[Olaf Gottschalk]

Disabling K2 mode and, voila, it works like it should:

[Olaf Gottschalk]

I pushed this to my repo for you to check, @Vladislav Koshkin!

[Olaf Gottschalk]

Ok, and the next still unsolved issue is: my DSL compiles into a JAR file that I use in other projects. E.g. I have a demo project that hosts demo scripts. There, the jar is added as a dependency through the Gradle build file - but looking into the Kotlin scripting settings, the format is missing. Only when adding the "magic" file into the resource path of that second project is the script type known. This used to work before as well... Plus, sadly: even after adding this resource file, the scripts do not work, everything is colored red...

[Olaf Gottschalk]

My goal is actually very simple: my customers do not need to understand the behind-the-scenes stuff. They start a Kotlin/Gradle project, add my DSL dependency and are good to go and create their script file ending in my extension in their project - and it just works.

[Olaf Gottschalk]

When K2 mode is disabled - this all works just like I want to. When K2 mode is enabled, it does not.

[Olaf Gottschalk]

@Vladislav Koshkin did you have time to check my findings again? I am still really stuck here... :(

[Vladislav Koshkin]

Hi, sorry for responding late
sure, I'll take a look