https://kotlinlang.org logo
Join Slack
Powered by
Can we get an interpreted mode for Kotlin? :pray::...
# compiler
l
Can we get an interpreted mode for Kotlin? 🙏🏼 https://twitter.com/ablenessy/status/1626570080094920705
z
I have plenty of problems with how Gradle handles Kotlin, but this is a pretty misleading way to compare them
how often is the average developer making abi changes (or first use) to buildSrc? Who is still using buildSrc in 2023 while claiming to be conscientious of build times?
l
buildSrc remains the easiest way to share logic with Kotlin code across modules FYI. The other ways are so not plug and play that I'm sticking to buildSrc… 😔
z
you pay an upfront cost for compilation because it's a compiled language, and the result is a faster runtime later. A better way to read the above is that groovy will never be faster than 9 seconds while kotlin is nearly 2x faster at runtime in the common case
buildSrc is not the easiest way in 2023. Convention plugins are a thing
l
The ones you put in buildSrc, right?
z
no
l
Like 
android-lib.gradle.kts
z
there's no buildSrc necessary
l
Honestly, I disagree with the fact that what's shown in NowInAndroid is easier than buildSrc. You need a lot more configuration code.
z
you can write a full plugin or you can write standalone scripts that you apply as well
e
I have a hard time seeing how you'll build a Kotlin interpreter that's faster than the current script compiler.
l
I know buildSrc is not state of the art, but it's the easiest, I already find myself spending way too much brainpower on build logic, I don't want to make it worse when I maintain full projects, beyond the focus of build logic, which is only a mean to the end.
z
I mean, that's just your opinion really. I disagree that it's the easiest, and it's definitely not worth the cost
l
@ephemient Basically, it'd be like the Groovy interpreter was able to understand Kotlin code. You could use this while tweaking build-logic, and have a faster feedback loop
e
scripting re-uses compiler infrasturcture
do you make that faster, which is good for all users, or do you start over?
z
gotta head off to a meeting, but moral of the story for me is this is a wildly misleading comparison and doubling down on buildSrc despite its costs just because of perceived ease of use is not worth a major change to the kotlin compiler 🙃
e
and it is part of my job as a platform engineer to know how to use Gradle effectively to make things better for the rest of my team
l
I agree about that comparison being misleading. I disagree that setup for "convention plugins" is simpler than using buildSrc as you don't need to write plugin registering boilerplate in the latter case, you get the precompiled script or the code available just like that, no questions asked.
Please, don't think I'm a platform engineer
z
you've hijacked your own thread at this point with that bit of bike shedding 🙃
l
I'm not paid to smooth the build of colleagues
e
maybe you should hire somebody to :p
but seriously, there's no setup beyond settings.gradle. if the string IDs really bug you, then put them in the version catalog and use them that way
l
Anyway, feedback loop to edit buildSrc content and try changes is slow if using Kotlin, and I'm sorry if it wasn't clear that it's the thing that's bugging me.
I don't understand why buildSrc should be slower than includedBuild also
It seems like it is, but I've yet to grasp the reason.
e
because it is added into the root project's classpath, it affects every single project whether it's relevant or not
also historically, Gradle would unconditionally run its tests, because there was no other way to do so (changed as of Gradle 8 )
l
Is that the only reason? How crowded the classpath is?
e
it's not how crowded it is, it's that any change invalidates everything
at least with Kotlin it is possible to tell what ABI changes are
l
Same would apply for includedBuild if you change a plugin you use about everywhere in the projects/modules
e
sure, but I have 12 different plugin includeBuild in my main project, most of which are not used everywhere
(well, one includeBuild with many subprojects, but same idea)
in any case, beyond your possible inefficiencies in Gradle usage, there is clearly room for performance improvement on both the Gradle and Kotlin side. but I disagree that building a new interpreter is the way to solve it.
l
So you need one includedBuild subproject per plugin?
Otherwise, touching just one file invalidates everything the same, correct?
e
some of them are grouped together
it really depends on how they're used
l
That's what I dislike so much: explosion of configuration, you need a module to configure the module, then you include it and can use it…
The fact that we're having this conversation shows how intuitive this is not IMO.
If the Kotlin compiler and Gradle find a way to get the feedback loop faster, I'm all in, but seeing so little progress for years while Groovy gets fast feedback loop despite slower builds when things don't change, makes it hard to hope for, plus K2 is set to be 2X faster from what I've read. Good, but not 3.8x good. I'll still take it of course 🙂
j
+1 to @louiscad about 
buildSrc
build-logic
(or whatever included build) can't be simpler than 
buildSrc
as 
build-logic
can be, literally, 
buildSrc
configuration plus additional configuration. For middle-point projects should be great that 
buildSrc
was just an included build automatically added which only codegen plugin accessors, no more than that. And some movements from Gradle are in that direction (I think 
buildSrc
is not running the tests now for example)
z
it's like 1% more work than buildSrc and 10x less of a cost, I think you're both making a bit of a meal of this
j
Just talking about the setup. Personally the benchmark shared above warns about a regression on Kotlin side too, I wouldn't care about that until it is fixed. Additionally I prefer the Slack approach of a custom DSL over convention plugins too. Sadly Gradle is missing, IMO, one phase to allow this approach to be easy to get it working without thinking a lot on the order or depending on 
afterEvaluate
, or a callback hell to avoid it. And I am not a fan of adding phases, I had enough "phases" with Android Activity and Fragment lifecycles...
d
@louiscad regarding original question: how do you imagine this interpreter should work? Should it perform all static analysis with resolution and inference on runtime or what? And what are usecases despite improving 
buildSrc
performance which is quite rare (comparing to all other compiler usescases)
g
Anyway, feedback loop to edit buildSrc content and try changes is slow if using Kotlin
@louiscad But… it’s literally faster x1.7 times for non-abi case using Kotlin, exactly what you are asking for
l
@gildor Aren't changes in precompiled scripts always treated as abi changes?
g
I don't know. One more reason to use normal plugins if so
l
@dmitriy.novozhilov It'd perform all resolution and static analysis lazily on runtime if there's no up to date fully-compiled version. Use case would be opt-in for fast prototyping, especially scripts/CLI, or WEB dev use cases, when you don't need the safety for checking the entire program before running in that prototyping phase. I think people in datascience would also like such a mode where you don't wait until it starts running the code, even though it's slower at executing until you opt back out of this "on-the-fly" mode. Could also be helpful when you need to test a tiny ABI impacting thing in a module that a lot of other modules depend on and would get recompiled for. In those cases, you don't care if the program is 5x slower because you only want to see what happens when you change that one line of code that is buried upon layers of modules, and you want the feedback loop to be fast rather than the execution time to be fast. Also, it's possible that at runtime, not so much code needs to be interpreted before you are able to reach that piece of code you're changing and testing, so that 5x (or whatever) slower execution time of something much smaller would be much cheaper than the compilation time you'd have paid.
I guess it'd be possible on the JVM since we have Kotlin REPL already when you run the… 
kotlin
command, without any argument blob thinking upside down
m
I wonder if GraalVM's stuff like Truffle could help in implementing interpreted or on-the-fly mode. Or maybe via WASM - like to quickly compile (that backend seems to be the simplest) an unoptimized in-memory module and feed in to WASM runtime. It has a puls that WASM is meant to be fully sandboxed (unlike JVM) so maybe such a mechanism could be used also for compilation time evaluation.
d
Could also be helpful when you need to test a tiny ABI impacting thing in a module that a lot of other modules depend on and would get recompiled for
There is a thing which supports this usecase. It is an incremental compilation which is supported on compiler level and on gradle level. AFAIU gradle does not reuse compiler IC atm but our and gradle teams are working on it
It'd perform all resolution and static analysis lazily on runtime if there's no up to date fully-compiled version.
Same, it's what IC does. But on compile time
I think people in datascience would also like such a mode
There is a Jupiter kernel for Kotlin
l
Here's another thing that could kinda address the same dev UX issues of paying compilation upfront when you only wanted to see the result of a fairly small change: Performing what I call "tentative eager compilation", so when you're ready to run the code, most, or all of it is already compiled. I realized that most of my code is run long before I actually try to run it. Typical scenario: 1. You write your code 2. You fix errors shown in the IDE 3. You think or double-check your code, or you get distracted for a bit while the machine waits 4. You finally press the run button 5. *You wait*… while the machine compiles 6. Compilation completes and the program starts running That "tentative eager compilation" would be connected the IDE if active, or connected to the filesystem otherwise. The trigger time would depend whether the IDE is in smart mode and can see beyond syntax errors, if it can see syntax errors, or if it can't see that at all, and on which files are being touched. You'd get the best experience in smart mode since it'd wait for the IDE to let the eager compiler know that there are no errors it can see in files that previously had errors. This is not about an interpreted mode at all, but since it's around the same problem, and still related to the compiler, I'm sharing that idea here. Also, if an interpreted mode was to come under consideration, I'm thinking that these two things could actually work together, where compiled and non invalidated code would still be at play, while the missing parts that didn't get a chance to be fully compiled would be on the fly at runtime by the interpreter, if needed in the code path.
There is a Jupiter kernel for Kotlin
Yep, I know, never tried it but I guess it's fully compiling the code on every change, correct?
There is a thing which supports this usecase. It is an incremental compilation which is supported on compiler level and on gradle level. AFAIU gradle does not reuse compiler IC atm but our and gradle teams are working on it
As I was writing all this, I actually started to wonder: it shouldn't be that slow… IC exists, is it disabled somehow? Now you're confirming me that it isn't. Is there an issue to follow about IC in Gradle and its buildSrc + includedBuild? I think that would become my #1 issue.
d
This is not about an interpreted mode at all, but since it's around the same problem, and still related to the compiler, I'm sharing that idea here.
This proposal is not about compiler, but about IDE. I'm not a fan of it (because it's better for me to wait explicit compilation but without sporadic compiler invocation in background) but it's worth to make a feature request in YT
Yep, I know, never tried it but I guess it's fully compiling the code on every change, correct?
AFAIK it works over repl and reuses compiled code from snippets which weren't changed
Is there an issue to follow about IC in Gradle and its buildSrc + includedBuild?
Almost any change in 
buildSrc
triggers recompilation of build scripts, it's just how 
buildSrc
works. So if you have any pains with it, it's worth to discuss them with Gradle team
t
Is this convention plugins/includeBuild working well with KMM projects?
e
yes. plugins themselves are normal JVM builds and work fine. Kotlin multiplatform had issues with included builds/dependency substitution, but those issues were with dependencies, not plugins, and are being resolved soon anyway. https://kotlinlang.org/docs/whatsnew-eap.html#preview-of-gradle-composite-builds-support-in-kotlin-multiplatform
9 Views
Open in Slack
PreviousNext
Powered by