:wave: hello folks -- i've been using dataframe fo...

# datascience

[Greg]

👋 hello folks -- i've been using dataframe for about a week and so far loving it. Until I tried it, i was processing data to produce flat csv files, that i'd them copy to e.g google sheets and attempt to extract info or charts from, which was quite painful or slow. So far i've sort of kept that process (but then generate plos with kandy and html tables with df), but now that I understand df better, i want to "unflatten" my data. Now to do so, i'd love to benefit from the compiler plugin to refactor my code safely.

Another bit of context -- i'm writing scripts with a kotlinc -script hashbang --

#!/usr/bin/env kotlinc -script

@file:CompilerOptions("-jvm-target", "23")
@file:DependsOn("org.jetbrains.kotlinx:dataframe:1.0.0-Beta3")
@file:DependsOn("org.jetbrains.kotlinx:kandy-lets-plot:0.8.0")

import org.jetbrains.kotlinx.dataframe.annotations.DataSchema
[...]

So, long story short, I'm wondering if there's a way to use the compiler plugin without Gradle? I know kotlinc has the -P flag to "Pass an option to a plugin." but that only works for built-in plugins, doesn't it?

[Nikita Klimenko [JB]]

Hi!
I think likely compiler plugin won't work in script files specifically, because resolve there works differently from usual files
Statement 1

val df = dataFrameOf("a" to columnOf(123))//.also { it.a }

Statement 2
This might not compile already

df.a

But you can try -Xplugin= as seen here https://kotlinlang.org/docs/no-arg-plugin.html#command-line-compiler
And here's the artifact
https://mvnrepository.com/artifact/org.jetbrains.kotlin/kotlin-dataframe-compiler-plugin-experimental/2.3.0-Beta1

[Nikita Klimenko [JB]]

Actually it shouldn't be hard to make it work
Problem is with resolve inside bodies of top level properties:

val df = dataFrameOf("a" to columnOf(123))
df.a // cannot resolve

But inside function scope resolve should work

fun dummyScope() {
  val df = dataFrameOf("a" to columnOf(123))
  df.a
}
dummyScope()

[Greg]

haaa! -Xplugin= is they key ingredient i was missing 🙂

[Greg]

now of course this is when my intellij decides to crash -- not sure what's going on with that since last night

[Greg]

@Nikita Klimenko [JB] does this ring a bell by any chance?

$ kotlinc -version
info: kotlinc-jvm 2.2.21 (JRE 25.0.1)

$ kotlinc -Xplugin=~/Downloads/kotlin-dataframe-compiler-plugin-experimental-2.3.0-Beta1.jar -script my_script.main.kts
error: while analysing my_script.main.kts:292:5: java.lang.NoSuchMethodError: 'boolean org.jetbrains.kotlin.fir.declarations.FirClassLikeDeclaration.isLocal()' (my_script.main.kts): org.jetbrains.kotlin.util.FileAnalysisException: While analysing my_script.main.kts:292:5: java.lang.NoSuchMethodError: 'boolean org.jetbrains.kotlin.fir.declarations.FirClassLikeDeclaration.isLocal()'

[Nikita Klimenko [JB]]

Version of compiler plugin needs to match compiler, try this:
https://mvnrepository.com/artifact/org.jetbrains.kotlin/kotlin-dataframe-compiler-plugin-experimental/2.2.21

[Greg]

oh i see, i was wondering what 2.3.0-beta* was about, since dataframe is on 1.0.0-beta* -- trying that now, thanks 🙂

[Greg]

alright, that's more promising:

kotlinc -Xplugin=~/Downloads/kotlin-dataframe-compiler-plugin-experimental-2.2.21.jar  -script my_script.main.kts
my_script.main.kts:633:5: error: 'public' property exposes its 'local' type argument 'Key_69'.
val allData = results.toDataFrame()
    ^
my_script.main.kts:691:5: error: 'public' property exposes its 'local' type argument 'Key_92'.
val mainMetrics = allData.select(
    ^

don't think i can work this out without a working intellij though -- back to debugging that problem now

[Greg]

thanks so much for the help though 🙂

[Nikita Klimenko [JB]]

Yes, that's what i thought. You can work around this by wrapping it in a function body, so they become local variables. In script properties are treated as top level class properties, thus this warning

[Greg]

gotcha

[Greg]

yeah that should be easy to do 🙂

[Greg]

that did the trick indeed, thanks 🙂 Hoping I can finally get my intellij in order to benefit from this tho

[Greg]

alright, i got intellij to work again (...) -- and while I can now use typesafe properties for column selectors, intellij doesn't seem to be aware -- this setting doesn't seem to be enough

[Nikita Klimenko [JB]]

I asked if frontend compiler plugins work in IDE in scripts, need to wait for response

[Greg]

(i relented and added a build.gradle in the meantime)

[Nikita Klimenko [JB]]

KTIJ-36527
Not possible for now. But i agree, it can be interesting to have. I created an issue

[Greg]

Thanks so much @Nikita Klimenko [JB] ❤️