hello all! I need to dynamically compile Kotlin co...

# scripting

[Alex Ostrovsky]

hello all! I need to dynamically compile Kotlin code and then get the resulting .class files, I've been using kotlin-compiler-embeddable for this with KotlinCoreEnvironment and KotlinToJVMBytecodeCompiler. is that the right way to go, or is there some way to extract the class files from the Kotlin scripting API?

[ilya.chernikov]

Using internal compiler APIs is not a reliable way to use the compiler, because we might sooner or later break or even remove it without prior notice.
But we do not have official compiler API (yet) so probably you don't have much choice.
As of the scripting compiler - things are more stable there, it is an open API, albeit an experimental one. And you can use it to extract class files too. But you can use it only for scripts and there are some script-specific overheads too.
But if you interested, here is the test that shows saving compiled scripts to class files - https://github.com/JetBrains/kotlin/blob/master/libraries/scripting/jvm-host-test/test/kotlin/script/experimental/jvmhost/test/ScriptingHostTest.kt#L104
You can also access class bytes without saving, but to access this functionality you'll need some more hacking.

[Alex Ostrovsky]

very cool, thanks, will check this out! is there somewhere where I could read more about the script-specific overheads?

[ilya.chernikov]

Probably not. But if you explain a bit more about your scenario, maybe I'll be able to give some hints.

[Alex Ostrovsky]

it's an analytics system which uses Kotlin as a language for user-defined expressions. for each user defined expression, a source file is generated for a class that implements a certain interface (e.g. Predicate, Function), where the user's code is inserted. these generated source files are about 50-100 lines, and there are about 10,000 such source files. all the of source files need to be compiled to make sure that they typecheck and that there aren't any syntax errors. originally I tried using the Kotlin Scripting API, but it was too slow so I switched to the compiler API in order to be able to cache the generated class files. I will try to switch back to scripting, using the link you provided as an example. my main concern with scripting performance overhead is that the current compiler approach works really well (except for the memory leak) and I would like to avoid a performance regression.

[Alex Ostrovsky]

btw before trying to use Kotlin as the expression language I used SpEL from the Spring Framework. it was horrible because it was very slow, didn't support type checking, didn't have lambdas, didn't properly work with default interface methods or inheritance, and sometimes it would even compile incorrect bytecode, so Kotlin really saved me. Kotlin has a clean syntax and pretty good DSL support with type checking and pretty good performance, so I believe it's great for this use case (the project itself is written in Java)

[ilya.chernikov]

The Kotlin scripting have got caching support some time ago, so if your sources are changing rarely, there should be very limited cost for using scripting now. To check how it works, you can try to use simple script files .main.kts extension e.g. with the kotlin runner from command line. E.g. try to run the examples https://github.com/Kotlin/kotlin-script-examples/blob/master/jvm/main-kts/MainKts.md several times.
For simplified implementation you may have a look at the relevant parts in this definition - https://github.com/Kotlin/kotlin-script-examples/blob/master/jvm/simple-main-kts/simple-main-kts/src/main/kotlin/org/jetbrains/kotlin/script/examples/simpleMainKts/scriptDef.kt, or check the implementation of these tests - https://github.com/JetBrains/kotlin/blob/master/libraries/scripting/jvm-host-test/test/kotlin/script/experimental/jvmhost/test/CachingTest.kt
If default caching doesn't work for you for some reasons, please tell and we'll see what could be done next.

[Alex Ostrovsky]

excellent, especially the CachingTest looks informative. I will do some experimentation and post another question here if I have any more issues. thank you very much for your help!

[Alex Ostrovsky]

btw I see that you're assigned to the issue that I opened KT-47044. I just wanted to say that I'm impressed with the community outreach from the Kotlin team, and it makes me feel good about using this language 🙂

[ilya.chernikov]

Thank you for the nice words. Unfortunately it is not always possible cover everything, but we're trying to do our best.