https://kotlinlang.org logo
Join Slack
Powered by
Hello Scripters! With moving my own DSL project to...
# scripting
o
Hello Scripters! With moving my own DSL project to Kotlin 2.1 (coming from 1.9), I experience some problems with the tooling setup, specifically with IntelliJ offering syntax highlight and code completion on my own script files with their own extension. It used to work like this: In my DSL library project, there is a class with a 
@KotlinScript
annotation like this:
Copy code
const val TRALA_FILE_EXTENSION = "trala.kts"

@KotlinScript(
    displayName = "TraLa Script",
    fileExtension = TRALA_FILE_EXTENSION,
    compilationConfiguration = TraLaScriptConfiguration::class
)
abstract class TraLaScript
plus, there is a resource file at this location: 
resources/META-INF/kotlin/script/templates/com.bmw.transformation.scripting.TraLaScript
(which is empty) The compilation configuration looks like this:
Copy code
@Suppress("JavaIoSerializableObjectMustHaveReadResolve")
object TraLaScriptConfiguration : ScriptCompilationConfiguration(
    {
        defaultImports(traLaDefaultImports)

        ide {
            acceptedLocations(ScriptAcceptedLocation.Everywhere)
        }

        jvm {
            jvmTarget("17")
            compilerOptions("-Xcontext-receivers")
        }
    }
)
The idea is that any project that uses my DSL library can have files anywhere (see 
acceptedLocations
with the extension 
.trala.kts
and the editor will offer syntax highlight and code completion / inspection. This used to work. Sometimes I had to restart IntelliJ... it was never straight forward. But it worked. Now, I cannot see any way to make the editor behave like I want. And I do not know what changed. In projects that DEPEND on my lib (with this setup), nothing works any more. In my library project itself, the default imports don't seem to work anymore! When I add a new file 
foo.trala.kts
and start typing code, it inserts import statements that it should not, because they are part of the default imports... What am I doing wrong here? Is there any good documentation on these matters? Thanks! Olaf
👀 1
i
I'm sorry for the late response. You may try to add 
isStandalone(false)
to your 
ScriptCompilationConfiguration
. Although this is an old change - scripts with definitions without it are ignored in source roots by default. So it is quite suspicious that you're seeing it only now, but maybe something else was at play. For example there is a workaround with 
-Xallow-any-scripts-in-source-roots
compiler flag.
If it doesn't help, please create an YT issue - maybe it's an IDE bug after all.
o
Hi Ilya! I tried your 
isStandalone(false)
but still, nothing changes. The scripts are not getting their default imports at all, they are basically completely broken in IntelliJ. I would like to open a YT issue, but can it really be I am the only one who uses this? Is there any kind of documentation I can read again, like a demo project that showcases this rather odd setup with the "magic" 
ScriptCompilationConfiguration
class and the magic resource file 
META-INF/kotlin/script/templates/...
? I would like to test that in order to even write a proper YT issue as I cannot post all of my script language code. Thanks!
i
Hi Olaf! The combination should be supported in principle, because it is used for the Gradle precompiled scripts, that work fine with K2. But I'm not sure that we tested it thoroughly, so it maybe indeed be a bug here that surfaces in your case. In general, compiling scripts with other sources (via e.g. 
isStandalone(false)
) is considered quite an advanced case, where you need to understand quite well how your scripts will interact with the other sources in the project. Therefore, we hide it under the opt-ins, and thus, we hope, there are not many users who use this. Anyway, I recommend you to file an issue as is, using just the description you gave above, and we'll test the functionality on our tests and come back to you on it later. cc: @Vladislav Koshkin
👀 1
o
Hi @ilya.chernikov, turns out that everything works as expected once I switch off K2 Mode in IntelliJ... 😞 With K2 Mode enabled, my scripts are not rendering/working as before...
v
hi, I'll also take a look into that this week loading
👍 1
o
@ilya.chernikov @Vladislav Koshkin I created a demo project to showcase exactly what problem I am facing. Before I create an issue, could you please have a look??? https://github.com/Zordid/kotlin-demo-dsl This replicates exactly what I do in a bigger DSL. The "mysterious" resource file that goes by the name of the 
@KotlinScript
annotated file is recognized only when not in K2 mode. I also added a readme with two images showing exactly how it looks on my screen with the two different modes! Thanks!
👀 1
v
I've finally managed to get it working 1. so I cloned your project, made Gradle > Build 2. made sure there is marker annotation in resource folder under Gradle build dir 3. after that I mentioned the error by IDEA:
Copy code
[KOTLIN_SCRIPTING] Cannot load script definition class demodsl.MyDslScript

java.lang.UnsupportedClassVersionError: demodsl/MyDslScript has been compiled by a more recent version of the Java Runtime (class file version 65.0), this version of the Java Runtime only recognizes class file versions up to 61.0
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at com.intellij.util.lang.UrlClassLoader.consumeClassData(UrlClassLoader.java:303)
	at com.intellij.util.lang.FileLoader.findClass(FileLoader.java:187)
	at com.intellij.util.lang.ClassPath.findClassInLoader(ClassPath.java:240)
	at com.intellij.util.lang.ClassPath.findClass(ClassPath.java:190)
4. so I just removed
Copy code
kotlin {
    jvmToolchain(21)
}
and rebuild the project there is also an action in Kotlin Scripting Settings that allow you to force re-scan your classpath, it was handy in that case
action point for us in that case - to add more robust logging
o
Wow, so my aim to be "modern" in the toolset to build crashed silently! 😉 Where did you find the exception that revealed this? And why can I get this to work again by disabling K2 mode? How does that interact with one another? Need to go and try it today! Thanks so far! Will come back to you asap!
Okay, by removing the Kotlin jvmToolChain, I changed the situation quite a bit! ...but still not without problems in my real-world library... First, I see that the script definition now appears and is enabled! This is good! But in my demo script, the default imports don't work properly... and I cannot even manually import my functions... normal Kotlin like 
println
works, but nothing from my project is available, even when trying to import manually, even though the code completion offers my packages... Any idea why that is? I wanted to attach screenshots, but.... Slack does not accept them, they all get a red exclamation mark...
As soon as I disable K2 mode, it works...
This is what K1 mode looks like...
Switch to K2 mode and nothing works anymore:
removing the jvmToolChain(21) allowed me to see that my scripting is now known (no matter if K2 mode is on or off)
@Vladislav Koshkin okay. I got behind this now, but I cannot make any sense about how the K2 mode affects this! I have my scripting defined in the main source set. But the screenshots you see from my scripts failing, are scripts that reside in the test source set! With K2 mode enabled, I cannot access the DSL code from main source set! As soon you disable K2 mode, the scripts can be in any location and work, with K2 mode, they only work when within the main source set! Why?!?
I managed to showcase the very same thing in the demo DSL... this is how it looks with K2 mode enabled when you put the exact same script in main and in test source set, side by side:
Disabling K2 mode and, voila, it works like it should:
I pushed this to my repo for you to check, @Vladislav Koshkin!
Ok, and the next still unsolved issue is: my DSL compiles into a JAR file that I use in other projects. E.g. I have a demo project that hosts demo scripts. There, the jar is added as a dependency through the Gradle build file - but looking into the Kotlin scripting settings, the format is missing. Only when adding the "magic" file into the resource path of that second project is the script type known. This used to work before as well... Plus, sadly: even after adding this resource file, the scripts do not work, everything is colored red...
My goal is actually very simple: my customers do not need to understand the behind-the-scenes stuff. They start a Kotlin/Gradle project, add my DSL dependency and are good to go and create their script file ending in my extension in their project - and it just works.
When K2 mode is disabled - this all works just like I want to. When K2 mode is enabled, it does not.
@Vladislav Koshkin did you have time to check my findings again? I am still really stuck here... :(
v
Hi, sorry for responding late sure, I'll take a look
please wait 1
o
Hey @Vladislav Koshkin, are you able to see what I see? I am still having massive issues and even though my script type is shown in "Kotin Scritping", my scripts simply do not work in IntelliJ with K2 mode.
When I set my project to Java 11 instead of Java 17, it somehow seems to work in both modes - I am totally lost here.
v
hi, I see two issues here: 1. we don't treat 
acceptedLocations
key properly (ScriptAcceptedLocation.Everywhere should include test folder) 2. we do not detect definition when we put compiled jar as gradle dependency
about the second one: how do you build the project with script definition? does the final jar contain this demodsl.MyDslScript file?
by the way what's your idea version? because it works on my side
o
Regarding #1: I have the problems with the scripts everywhere, not only in test folder, but also in main. Regarding #2: that seems to work. I have my DSL project included as a dependency in my "developer project" which I want to give customers to write my DSL scripts. (but the problems we are talking about here are also present when in the DSL project itself... so, how does that play with this?) I just looked... apparently the DSL script file is missing in the final JAR file... sigh Probably need to add something to the Gradle Build Script to include it?
Sorry, that was wrong. My DSL file is included in the final jar. Opened the wrong location...
image.png
v
okay, I'll try to build the project and reference it in gradle dependencies
by the way could you try to click 'Scan classpath' action via kotlin scripting settings?
it (should) allows IDE to discover definition in classpath
o
I always check and do that. The problem exists even when it's discovered (shown in the list). Then I scan again, which does not change things visible here.
v
okay, I made this working first thing - file extension was not enough - I had to add filePathPattern like this:
Copy code
@KotlinScript(
    displayName = "MyDsl Script",
    fileExtension = "mydsl.kts",
    filePathPattern = ".*\\.mydsl\\.kts", //<<<<<<<<<<<<<<-----
    compilationConfiguration = MyDslScriptConfiguration::class
)
next, I have to be sure that my jar build with gradle publish task contains all the script dsl: so I've added this to my build.gradle.kts file:
Copy code
publishing {
    publications {
        create<MavenPublication>("something") {
            groupId = "com.testing.demo-dsl"
            artifactId = "demo-dsl"
            version = rootProject.version.toString()
            from(components["kotlin"])
        }
    }
    repositories {
        mavenLocal()
    }
}
then I took a look inside the jar, everything was in place:
Copy code
❯ ls -l
total 24
-rw-r--r--  1 Vladislav.Koshkin  staff  12145 17 Jul 12:00 demo-dsl-1.0-SNAPSHOT.jar
~/.gradle/caches/modules-2/files-2.1/com.testing.demo-dsl/demo-dsl/1.0-SNAPSHOT/290f2b6777ee91619b9d52f05a52e03d97d64ad4
❯ jar tvf demo-dsl-1.0-SNAPSHOT.jar
     0 Thu Jul 17 11:58:00 CEST 2025 META-INF/
    25 Thu Jun 05 18:32:52 CEST 2025 META-INF/MANIFEST.MF
     0 Thu Jul 17 11:58:00 CEST 2025 demodsl/
   678 Thu Jul 17 11:58:00 CEST 2025 demodsl/MyDslScript.class
  1338 Thu Jul 17 11:58:00 CEST 2025 demodsl/DemoDslKt.class
  4522 Thu Jul 17 11:58:00 CEST 2025 demodsl/MyDslImpl.class
  4447 Thu Jul 17 11:58:00 CEST 2025 demodsl/MyDslScriptConfiguration.class
   667 Thu Jul 17 11:58:00 CEST 2025 demodsl/Shout.class
   663 Thu Jul 17 11:58:00 CEST 2025 demodsl/Say.class
  2734 Thu Jul 17 11:58:00 CEST 2025 demodsl/ScriptingKt.class
   535 Thu Jul 17 11:58:00 CEST 2025 demodsl/MyDslMarker.class
   837 Thu Jul 17 11:58:00 CEST 2025 demodsl/MyDsl.class
   434 Thu Jul 17 11:58:00 CEST 2025 demodsl/Verb.class
   828 Thu Jul 17 11:58:00 CEST 2025 demodsl/MyDsl$DefaultImpls.class
    71 Thu Jul 17 11:58:00 CEST 2025 META-INF/demo-dsl.kotlin_module
  1690 Thu Jul 17 11:58:00 CEST 2025 MainKt.class
     0 Thu Jul 17 11:57:58 CEST 2025 META-INF/kotlin/
     0 Thu Jul 17 11:57:58 CEST 2025 META-INF/kotlin/script/
     0 Thu Jul 17 11:57:58 CEST 2025 META-INF/kotlin/script/templates/
     0 Thu Jul 17 11:57:58 CEST 2025 META-INF/kotlin/script/templates/demodsl.MyDslScript
then I've create a new gradle project and put this line in my 
build.gradle.kts
Copy code
implementation("com.testing.demo-dsl:demo-dsl:1.0-SNAPSHOT")
and then I did scan classpath action via kotlin scripting settings tab and when I created a .mydsl.kts file - highlighting started to work
1. you can right click to the status bar and enable kotlin script definition option for .kts files it really helps to understand definition matching
also there is a registry flag, that allows you to see each script classpath:
like this:
^^^ but note that these options are pure experimental and can be removed/transformed into something else 🙂 hope it helps for now
in the latest idea there is also an option to force reloading configuration for the script maybe we should made it also to drop definition caches on use 🤔
first thing - file extension was not enough - I had to add filePathPattern like this:
never mind - this one is wrong your definition was all right
👍 1
o
So on a colleagues' Windows PC we had to both disable K2 mode and set the Java target to 11 to make the script appear ok in the editor, on my Mac machine it's enough to go to Java 11.
Hey Vlad! The really annoying part of all of this is that the IDE is in itself so inconsistent. See, I got two of my scripts in this project as an example. When I open the first, it works as expected - but in the same setup, another file does not work. I really do not know what to do and how to transport to you the needed information to find the problem's root cause. See here:
In fact, in this project, the only file that works fine is the one you see here. All the other 
trala.kts
files I open end up all in red. What do I need to do to see just why it does not understand this script?
v
could you open just a red file in the editor (with no splitted editors) and show me the status bar?
o
same - it is recognized as just that
image.png
By the way: the status bar concept does not really work well with split views, right? 😉
🙈 1
v
could you please also share the classpath from project structure window?
o
With this error on screen, when I just restart IntelliJ - the same file is ok after restart!
image.png
afk for lunch!
After restart, it now can open the files correctly. But: I have gone through exactly this so many, many times - it never works stable, and the next time it shows everything red and still don't know exactly what to do other than invalidating, restarting switching on and off K2 mode, etc. Those all seem to be triggers to somehow get it back in a working state, but I never know exactly why. And on top of that, I got internal customers who ask me about this as well and I cannot really tell them what to do.
Now, with IntelliJ still running, new files that correctly identify as my 
TraLa Script
according to the Status bar indicator, do NOT work again. I am so frustrated with this. What should I look for in this situation to see WHY it does not work?
The most frustrating part of all is this complete lack of information. For example: I want to always start my scripts with a call to the function 
jsonTransformaion
which takes a code block. Typing the letters 
j-s-o-n
and waiting for code completion offers shows stuff, but NOT my method. Where does that "stuff" come from, why is MY function missing even though the script type is correctly identified and shown?
In the same instance of IntelliJ, just another editor that had this working, the same code completion offerings look totally different (and correct)
Hello @Vladislav Koshkin... any way you can help fix this issue? This is maddening me... Every time I use this project, I have SOME scripts that work fine, but when I create new scripts, same extension, also detected as the status bar says, but does not work. All red. What should I do?
I am so tired of seeing this, having to restart/clear caches, and at some point, it works again
Now I got more new problems after updating my JDKs using brew update... Only within IntelliJ and ONLY within my scripts, I get hundreds of errors attached to any String functions stating: 
Cannot access 'Serializable' which is a supertype of 'String'. Check your module classpath for missing or conflicting dependencies.
I did all I could, checked the module classpaths, made sure it's all pointing to the existing JDK 21, invalidated caches, deleted the 
.idea
folder restarts, repairs, etc. To no avail. It does not work. From a command line use with Gradle build, it's all working, just in the editors I got these errors everywhere.
What exactly is the "module classpath" for a script anyhow? I only know (and checked) it in the Project Structure of the IntelliJ project, but that had absolutely no effect on the scripts. 😞
v
Hi, let's setup a quick call next week? So we can take a look together
I think it can be more productive
o
Absolutely! I would love to show you in person!
Monday and Wednesday would be easy for me as I am working from home then
Hey @Vladislav Koshkin - just ping me when you're available, please! Thanks!
loading 1
16 Views
Open in Slack
PreviousNext
Powered by