:kodee-welcoming: Hi, I was wondering if there is ...

# scripting

[Simon Marquis]

kodee welcoming Hi, I was wondering if there is a special flag to "only compile" *.main.kts files (and not continue with execution).
I have many scripts, and some of them are depending on each others. Modifying a script makes it very brittle since it can break other scripts.
I wanted to run a global compilation on CI of all *.main.kts files to be able to detect such cases as soon as possible.
Unfortunately, I did not find a good way of doing it for now (providing a custom arg to the script to prevent the regular execution forces us to add this to all our scripts and is error prone).

[ilya.chernikov]

The scripts could be compiled just by passing them as regular sources to the compiler, without the -script option.
But scripts that are not designed to be used among regular sources (for which isStandalone==true in the script definition) are ignored by the compiler.
To work around it, use the -Xallow-any-scripts-in-source-roots compiler option.

[Simon Marquis]

I'm not sure I understand it.
I'm using self-contained scripts files as described in https://github.com/Kotlin/kotlin-script-examples/blob/master/jvm/main-kts/MainKts.md

Using kotlinc with -Xallow-any-scripts-in-source-roots seems to get around the error: no source files , but it forces us to configure many more things.
For instance, I get this new warning:
warning: scripts are not yet supported with K2 in LightTree mode, consider using K1 or disable LightTree mode with -Xuse-fir-lt=false
Adding it leads to a new error, where the compiler do not take into account the configured compiler option

@file:CompilerOptions("-jvm-target", "21")
@file:CompilerOptions("-language-version", "2.3")

While running kotlin instead of kotlinc works without configuring anything but has the side-effect of also running the script.

[ilya.chernikov]

Ah, yes, you still need -Xuse-fir-lt=false for now, until the 2.4 is out.
And compiler options are ignored in this mode, as well as some other annotations. You need to convert them to the CLI compiler options.
kotlin will run the scripts, this is by design. It is the same as using kotlinc -script in this case.

[Simon Marquis]

You need to convert them to the CLI compiler options

What do you mean? Are there args for kotlin / kotlinc to replicate these?

So then there is no way to have the same behavior as kotlin foo.main.kts (or kotlinc -script foo.main.kts) without also running the script? Since using kotlinc wihtout -script requires more configurations/arguments.

[ilya.chernikov]

@file:CompilerOptions are literally CLI compiler options in a form of an annotation. So in your case adding -jvm-target 21 to the compiler CLI will do the same as the first annotation.

So then there is no way to have the same behavior as...

Yes. If you have good arguments for it, please add an YT issue.
Or you can hack it yourself with the Scripting API.

[Simon Marquis]

Oh you're right, they are simply extracted at compile time. Works for my usecase 👍

I'm not sure there is a good enough argument for it. I just wanted to add an extra "validation" step, and potentially persist the compilation cache for these scripts to avoid compiling again when running them on CI.
But I'm pretty sure creating a proper CLI project would be advised at this point (and compiling it as any other project).

[Simon Marquis]

👋 A new issue appeared while doing some refactoring.
I migrated some scripts to use @file:Import("utils/foo.main.kts") where we centralize some utility functions and classes.
And for some reason i do not fully understand right now, if I reference one of the classes from the foo.main.kts inside another script, the kotlinc ... will fail with this kind of stacktrace (although the script work fine when executing it): (is this another missing compiler argument?)

kotlinc -Xallow-any-scripts-in-source-roots -Xuse-fir-lt=false -jvm-target 25 -d build/kotlinc/ test.main.kts
exception: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Exception was thrown during transformation of class org.jetbrains.kotlin.fir.declarations.impl.FirPropertyImpl
        at org.jetbrains.kotlin.fir.backend.generators.Fir2IrCallableDeclarationsGenerator.createIrProperty(Fir2IrCallableDeclarationsGenerator.kt:1202)
        at org.jetbrains.kotlin.fir.backend.Fir2IrDeclarationStorage.createAndCacheIrProperty(Fir2IrDeclarationStorage.kt:578)
        at org.jetbrains.kotlin.fir.backend.Fir2IrDeclarationStorage.createAndCacheIrProperty$default(Fir2IrDeclarationStorage.kt:566)
        at org.jetbrains.kotlin.fir.backend.Fir2IrConverter.processMemberDeclaration(Fir2IrConverter.kt:515)
        at org.jetbrains.kotlin.fir.backend.Fir2IrConverter.processScriptLikeDeclaration(Fir2IrConverter.kt:601)
        at org.jetbrains.kotlin.fir.backend.Fir2IrConverter.access$processScriptLikeDeclaration(Fir2IrConverter.kt:72)
        at org.jetbrains.kotlin.fir.backend.Fir2IrConverter.processMemberDeclaration(Fir2IrConverter.kt:493)
        at org.jetbrains.kotlin.fir.backend.Fir2IrConverter.processFileAndClassMembers(Fir2IrConverter.kt:199)
        at org.jetbrains.kotlin.fir.backend.Fir2IrConverter.runSourcesConversion(Fir2IrConverter.kt:98)
        at org.jetbrains.kotlin.fir.backend.Fir2IrConverter.access$runSourcesConversion(Fir2IrConverter.kt:72)
        at org.jetbrains.kotlin.fir.backend.Fir2IrConverter$Companion.generateIrModuleFragment(Fir2IrConverter.kt:700)
        at org.jetbrains.kotlin.fir.pipeline.Fir2IrPipeline.runFir2IrConversion(convertToIr.kt:196)
        at org.jetbrains.kotlin.fir.pipeline.Fir2IrPipeline.convertToIrAndActualize(convertToIr.kt:153)
        at org.jetbrains.kotlin.fir.pipeline.ConvertToIrKt.convertToIrAndActualize-MT2kVtw(convertToIr.kt:119)
        at org.jetbrains.kotlin.fir.pipeline.ConvertToIrKt.convertToIrAndActualize-MT2kVtw$default(convertToIr.kt:92)
        at org.jetbrains.kotlin.cli.jvm.compiler.legacy.pipeline.JvmCompilerPipelineKt.convertToIrAndActualizeForJvm-_I1IR-w(jvmCompilerPipeline.kt:98)
        at org.jetbrains.kotlin.cli.pipeline.jvm.JvmFir2IrPipelinePhase.executePhase(JvmFir2IrPipelinePhase.kt:30)
        at org.jetbrains.kotlin.cli.pipeline.jvm.JvmFir2IrPipelinePhase.executePhase(JvmFir2IrPipelinePhase.kt:25)
        at org.jetbrains.kotlin.cli.pipeline.jvm.JvmFir2IrPipelinePhase.executePhase(JvmFir2IrPipelinePhase.kt:19)
        at org.jetbrains.kotlin.cli.pipeline.PipelinePhase.phaseBody(PipelinePhase.kt:68)
        at org.jetbrains.kotlin.cli.pipeline.PipelinePhase.phaseBody(PipelinePhase.kt:58)
        at org.jetbrains.kotlin.config.phaser.NamedCompilerPhase.invoke(CompilerPhase.kt:102)
        at org.jetbrains.kotlin.backend.common.phaser.CompositePhase.invoke(PhaseBuilders.kt:22)
        at org.jetbrains.kotlin.config.phaser.CompilerPhaseKt.invokeToplevel(CompilerPhase.kt:53)
        at org.jetbrains.kotlin.cli.pipeline.AbstractCliPipeline.runPhasedPipeline(AbstractCliPipeline.kt:109)
        at org.jetbrains.kotlin.cli.pipeline.AbstractCliPipeline.execute(AbstractCliPipeline.kt:68)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecutePhased(K2JVMCompiler.kt:51)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecutePhased(K2JVMCompiler.kt:42)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:91)
        at org.jetbrains.kotlin.cli.common.CLICompiler.exec(CLICompiler.kt:363)
        at org.jetbrains.kotlin.cli.common.CLICompiler.exec(CLICompiler.kt:341)
        at org.jetbrains.kotlin.cli.common.CLICompiler.exec(CLICompiler.kt:303)
        at org.jetbrains.kotlin.cli.common.CLICompiler$Companion.doMainNoExit(CLICompiler.kt:442)
        at org.jetbrains.kotlin.cli.common.CLICompiler$Companion.doMainNoExit$default(CLICompiler.kt:437)
        at org.jetbrains.kotlin.cli.common.CLICompiler$Companion.doMain(CLICompiler.kt:429)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler$Companion.main(K2JVMCompiler.kt:232)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.main(K2JVMCompiler.kt)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:565)
        at org.jetbrains.kotlin.preloading.Preloader.run(Preloader.java:87)
        at org.jetbrains.kotlin.preloading.Preloader.main(Preloader.java:44)
Caused by: java.lang.IllegalStateException: Source classes should be created separately before referencing
        at org.jetbrains.kotlin.fir.backend.Fir2IrClassifierStorage.createFir2IrLazyClass(Fir2IrClassifierStorage.kt:247)
        at org.jetbrains.kotlin.fir.backend.Fir2IrClassifierStorage.getIrClass(Fir2IrClassifierStorage.kt:202)
        at org.jetbrains.kotlin.fir.backend.Fir2IrClassifierStorage.getIrClassSymbol(Fir2IrClassifierStorage.kt:268)
        at org.jetbrains.kotlin.fir.backend.utils.SymbolConversionUtilsKt.toIrSymbol(SymbolConversionUtils.kt:63)
        at org.jetbrains.kotlin.fir.backend.Fir2IrTypeConverter.toIrType(Fir2IrTypeConverter.kt:121)
        at org.jetbrains.kotlin.fir.backend.Fir2IrTypeConverter.toIrType$default(Fir2IrTypeConverter.kt:98)
        at org.jetbrains.kotlin.fir.backend.Fir2IrTypeConverter.toIrType(Fir2IrTypeConverter.kt:85)
        at org.jetbrains.kotlin.fir.backend.Fir2IrTypeConverterKt.toIrType(Fir2IrTypeConverter.kt:375)
        at org.jetbrains.kotlin.fir.backend.Fir2IrTypeConverterKt.toIrType$default(Fir2IrTypeConverter.kt:373)
        at org.jetbrains.kotlin.fir.backend.generators.Fir2IrCallableDeclarationsGenerator.createIrProperty(Fir2IrCallableDeclarationsGenerator.kt:273)
        ... 40 more

[Simon Marquis]

I tried with the current kotlinc 2.4-Beta1, but there seems to have been some breaking changes because it no longer understand that the files are kotlin scripts (CompilerOptions, DependsOn, etc. are no longer resolved) 🤔

[Simon Marquis]

This compiles without the error

println(Foo())

This brings up the compilation error

Foo()
// or
val foo = Foo()

[ilya.chernikov]

Please try it first without -Xuse-fir-lt=false, I think it is not needed anymore.
And if it doesn't help - try to use e.g. jvmTarget 17, I've seen a report about problems on the recent JDKs, that I haven't checked yet.
If both don't help, please make a small repro and submit an YT issue.

[Simon Marquis]

I'll definitely try that next week and report my findings and/or create bug reports on youtrack 👍
Thanks

[Simon Marquis]

Sorry for doing it this late. Here is the result without the -Xuse-fir-lt=false

❯ kotlinc -Xallow-any-scripts-in-source-roots -jvm-target 25 -d /dev/null test.main.kts
warning: scripts are not yet supported with K2 in LightTree mode, consider using K1 or disable LightTree mode with -Xuse-fir-lt=false
error: no source files

[Simon Marquis]

with -jvm-target 17 -Xuse-fir-lt=false

❯ kotlinc -Xallow-any-scripts-in-source-roots -jvm-target 17 -Xuse-fir-lt=false  -d /dev/null test.main.kts
exception: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Exception was thrown during transformation of class org.jetbrains.kotlin.fir.declarations.impl.FirNamedFunctionImpl
        at org.jetbrains.kotlin.fir.backend.generators.Fir2IrCallableDeclarationsGenerator.createIrFunction(Fir2IrCallableDeclarationsGenerator.kt:1093)

with -jvm-target 17 but without -Xuse-fir-lt=false i get the same scripts are not yet supported with K2 in LightTree mode

[Simon Marquis]

What I ended up doing (which is not perfect, but is good enough for our usecase), is to filter for clikt-based main.kts files.
And execute them with the flag --help which is a default to dump the usage page.
This triggers the expected compilation.

[ilya.chernikov]

Seems there are many things going on in parallel.
• you can try to fight the error: no source files error with -Xallow-any-scripts-in-source-roots option, as I mentioned earlier
• the strange behavior of -Xuse-fir-lt=false - I need to investigate, but this is not something I recommend using in general, I just wanted to quick check, whether the error you're facing, caused by the new functionality, that this flag suppose to turn off
• The original problem of unresolved symbols could actually be the same as Olaf reported in the adjacent thread. It is fixed in the 2.3.21-RC2 (already out) and 2.4.0-Beta2 (hopefully out this week), so you may want to try it too

[Simon Marquis]

👋 Kotlin 2.4.0 brings new issues. We are no longer able to simply execute some script files like we used to do (with the shebang).

error: while analysing test.main.kts:38:1: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Expected FirResolvedTypeRef with ConeKotlinType but was FirUserTypeRefImpl (test.main.kts): org.jetbrains.kotlin.util.FileAnalysisException: While analysing test.main.kts:38:1: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Expected FirResolvedTypeRef with ConeKotlinType but was FirUserTypeRefImpl

[ilya.chernikov]

Oh, for this I'd like to have a repro. The type resolution failed for some reason.

[Simon Marquis]

It seems like it happens when using code from an @file:Import() code.
Here is my minimal example:

test.main.kts

#!/usr/bin/env kotlin

@file:DependsOn("com.github.ajalt.clikt:clikt-jvm:5.1.0")
@file:Import("utils/clikt.main.kts")

import com.github.ajalt.clikt.core.CliktCommand

class Test : CliktCommand() {
    private val projectDir by projectDir() // The error is reported on this line
    override fun run() = Unit
}

utils/clikt.main.kts

@file:DependsOn("com.github.ajalt.clikt:clikt-jvm:5.1.0")

fun BaseCliktCommand<*>.projectDir(): OptionWithValues<Path, Path, Path> = option(
    "--project-dir",
    help = "The project directory. Defaults to the current working directory.",
)
    .path(mustExist = true, canBeFile = false)
    .defaultLazy { Paths.get("").toAbsolutePath() }

[Simon Marquis]

error: while analysing tools/test.main.kts:9:31: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Expected FirResolvedTypeRef with ConeKotlinType but was FirUserTypeRefImpl (test.main.kts): org.jetbrains.kotlin.util.FileAnalysisException: While analysing tools/test.main.kts:9:31: org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Expected FirResolvedTypeRef with ConeKotlinType but was FirUserTypeRefImpl

[Simon Marquis]

If the extension is in the same source file then there is no error.

[Piotr Krzemiński]

hi, github-workflows-kt's owner here - this bug makes the library basically unusable with Kotlin 2.4.0 because when running GitHub Actions workflows, by default a consistency check is performed (the Kotlin script is executed and compared with the committed YAML). GitHub updated their runner's image yesterday with 2.4.0, lots of things are failing.

Is it tracked as "KT-86352 K2 scripting: FirResolvedTypeRef exception when resolving extension functions from imported scripts"?

[Simon Marquis]

Kotlin 2.4.10 produces a different variant of the error 😕

error: exception was thrown during transformation of class org.jetbrains.kotlin.fir.declarations.impl.FirPropertyImpl (testing.main.kts): org.jetbrains.kotlin.utils.exceptions.KotlinIllegalArgumentExceptionWithAttachments: Exception was thrown during transformation of class org.jetbrains.kotlin.fir.declarations.impl.FirPropertyImpl

[ilya.chernikov]

@Simon Marquis, can you, please, file a new issue with a repro.

[Simon Marquis]

youtrack.jetbrains.com/issue/KT-87958

[Simon Marquis]

Tested on the released 2.4.20 and it works 👍
Unfortunately I came across a new bug that i did not found earlier : youtrack.jetbrains.com/issue/KT-89247

[Simon Marquis]

Happy to report that all our issues are now fixed in 2.4.21 :)