I created some Kotlin Notebooks on AWS (Sagemaker) and noticed that if the output of a cell is HTML, there is some bug. Seems that instead of only the generated HTML snippet, the whole page is included the output. This only happens in Jupyter-Lab mode and not classic (which makes sense since only Jupyter-Lab mode actually includes snippets). Anyone else seeing this issue?

Kotlin on AWS SageMaker Kotlin notebooks work very nice on AWS SageMaker (disclaimer: only tested so far in the context of my own project roboquant.org). The response when executing a cell is snappy and graphs and other HTML outputs display correctly. Both classic notebooks and labs are working. Initial loading of packages is a bit slow, but that I guess that is partially because I use a lot of “SNAPSHOT” packages where Maven local is of less help. I typically use a

ml.c5.xlarge

but that is mainly because of some more CPU intensive calculations. A cheaper EC2 instance type should be fine for simpler work-loads. Since there is no out-of-the-box support for Kotlin kernels, you’ll have to provide a “lifecycle script” that will install the required packages. You can find more info including the lifecycle script I created at https://roboquant.org/tutorial/install.html#_aws_sagemaker But basically it boils down to installing these two packages when the notebook instance is created:

conda install -y openjdk

conda install -y -c jetbrains kotlin-jupyter-kernel

That is all. And the above is only done once during creation and after that when you start/stop the notebook it is just like any other notebook. If you have any questions, feel free to post them.

Did some investigation on Kotlin kernels in Jupyter Classic vs Lab. Right now Kotlin packages for notebooks differentiate between on how they render HTML output between the two. In a Classic environment they use isolation (resulting in an iframe) and Lab they use non-isolated (resulting in insert of plain HTML snippet). This approach works fine, but means you have to be able to differentiate between the two which is hard to due reliable. Letting the user decide is also not ideal. This differentiation is however not required. Classic & Lab both support non-isolated mode. The real problem packages encounter with Classic notebooks is that Jupyter Classic is including requirejs library that changes how third-party JavaScript libraries behave when they are loaded. If they detect require/define functions, they will not load themselves in global scope but instead use the require API. Quick workaround: ============== Disable this detecting behaviour in the JavaScript library you are loading so they don’t detect require framework anymore. They will behave now the same as Jupyter Lab, basically load themselves in the global namespace and work fine in non-isolated mode. In my case with ECharts JavaScript library, the following change was all that was required to disable detection: Replace:

"function"==typeof define

With

"function"==typeof define_none_existing

Of course if you load the script from a CDN, changing the code is not possible. But if you load it from resources directory (classPath option in your JupyterIntegration implementation) it is an easy change for most JavaScript libraries. Future better workaround: ===================== The code that Kotlin kernel generates to embed external JavaScript libraries could take care of this behaviour. It could detect requirejs presence and change it loading behavior. But this requires more investigation.

Hi everyone! I am starting with Kotlin DataFrame. I have a table with a

kotlinx.datetime.Instant

column

timestamp

and would like to add a new column with the difference between the timestamps of the current row and the previous row. It seems I would need something similar to Panda's

diff

operation but I can't find anything in Kotlin DataFrame documentation. How can I do this?

For people using Elasticsearch/Opensearch, I support jupyter now as well with my kt-search project: https://github.com/jillesvangurp/kt-search Great platform for creating some aggregations and visualizations. I'll likely be doing some of that myself in the next few weeks. I'm relatively new to jupyter.

Hi there, is there any written reference documentation about how to create html extensions for jupyter kernel library integrations?

Hi all! 👋 📣 :kotlinnew: The Kotlin team is trying to determine how to develop the Kotlin Spark API further, and we’d love your input. If you are actively using it in your personal or professional projects, please share your experience with us in a phone interview. It will take no more than one hour, and you’ll get a 1-year All Products Pack subscription or a $100 Amazon Gift Card as a reward. 👉 Please choose a time slot in my Calendly set up a call.

Hi folks 👋 I’m trying to run Kotlinx Dataframe 0.8.1 on Kotlin 1.7.20 (because I want the shiny new file visitor support), i.e.:

plugins {
    kotlin("jvm") version "1.7.20"
    id("org.jetbrains.kotlinx.dataframe") version "0.8.1"
}

During compilation, I get the warning:

ksp-1.7.10-1.0.6 is too old for kotlin-1.7.20. Please upgrade ksp or downgrade kotlin-gradle-plugin to 1.7.10.

And when I try to run the code, I get the error in the 🧵. Is there a way to “upgrade ksp”, or is that something DataFrame would need to do?

Is there a good way to plot multiple lines in letsplot with different y-axis scales? I have time series sensor data (e.g. think ECG versus accelerometer or unfiltered versus filtered waveforms), and having them overlayed but stretched for different y scales would be helpful for some analysis. ECG/accelerometer have totally different scales, while filtered data is 0 centered and raw data is not. I tried adding two geomLines on the same plot. They show up, but due to different y axis scales the lines aren’t readable. I also tried doing a GGBunch() without changing the x/y coordinates but only a single line shows up which makes me think one graph is completely covering the other. Other ideas or approaches for doing this?

Bonjour! I want to learn data science later since years The thing that blocked me, I think, is that I had no idea what I don’t know that I don’t know. Therefore I tried to make an high level overview of that. Please provide feedback Can you have a look at what I’ve done so far and provide feedback? --> 🤔 Data Science? High level overview

Is there an easy way to only compile an Jupyter Notebook cell but not run it. For example, currently I use the following code snippet for regression testing of expected outcome vs real outcome of a notebook. Please notice the use of the

exec

method to run a cell.

for (cell in notebook.cells.filterIsInstance<CodeCell>()) {
    val cellResult = exec(cell.source)
    val result = if (cellResult is MimeTypedResult) cellResult.entries.first().value else cellResult.toString()

    if (cell.outputs.isNotEmpty()) {
        val firstOutput = cell.outputs.first()
        if (firstOutput is ExecuteResult && firstOutput.data.isNotEmpty()) {
            val output = firstOutput.data.entries.first()
            assertEquals(output.value.removeUUID(), result.removeUUID())
        }
    }
}

But some cells take very long to run, so I would like to have the option to only compile the cell and get compilation errors, but not run (exec) the cell. Any hints how to achieve this?

How can we use experimental features from Multik inside a notebook ? I tried the following without success :

Have been optimizing my Kotlin notebooks to run better on public (low powered) infra like MyBinder. And been happy with the progress so far. One thing I’m not yet be able to fixed is how to include my own javascript file. Seems only options are to specify a local file or a URL (CDN). But problem with local file is that all the JavaScript get included in the notebook itself. Is it possible to reference a JavaScript file via and URL that is then served by the Kotlin Kernel ? You can check the progress out at: https://mybinder.org/v2/gh/neurallayer/roboquant-notebook/main?urlpath=tree/notebooks/ and for example try the charts notebook.

Is there a kotlin native dataflow or pipeline engine available, which i can also run on embedded device such as android? Unfortunately, spark will not be an option for me.

Hello friends! I beseech your wisdom! 😊 I have to tackle a

.csv

file (exported from a third-party system) which has integer columns with

,

as thousands separator (e.g,

47,302

,

48,000

). Needless to say, this is potentially problematic. 😅 When I load my

.csv

file into my Jupyter notebook, I believe

dataframe

relies on my system locale (

pt_BR

) and thus parses these integer columns as doubles —

pt_BR

has

,

as decimal separator and

.

as thousands separator. I end up having wrong values in those columns (

.csv

is of course to blame, not

dataframe

). So I was wondering if I could (a) disable type inference for either the entire

.csv

or selected columns and get everything as string, so I can manually parse these values, (b) change the underlying locale and see if it helps the type inference mechanism, or (c) have parsing rules associated to certain columns. Any suggestions are highly appreciated! I apologise in advance if this is trivial, but I failed to identify a similar scenario in the documentation. Cheers! :thank-you:

Not sure if any of you will join WSC2022 in Singapore starting in a few days, where the global simulation community celebrates its annual summit. I'll bring Kotlin to the stage there, by presenting kalasim, a code first discrete event simulation DSL/engine written in 100% Kotlin. Read more at https://www.linkedin.com/posts/holgerbrandl_wsc22-activity-7002255950950637568--wMw

Hello everyone, I've just recently discovered the Kotlin kernel for Jupyter. While trying to set it up in Intellij, I've come across a strange error: the highlighter always marks the code with an error stating "Expecting a top level declaration". Did any of you perhaps came across a similar problem, and if so do you have any tips on how to fix it? This doesn't happen if I launch Jupyter "manually", it seems to be related to Intellij itself.

Kotlin Spark API v1.2.2 is released! https://github.com/Kotlin/kotlin-spark-api/releases/tag/1.2.2 It's just a small update this time, mostly version bumps: • Added

BigInteger

support in #182 thanks to #181 • New Spark versions: 3.2.3, 3.3.1, 3.2.2 • New Scala versions: 2.12.17, 2.13.10 • Updated Kotlin to 1.7.20 • Small bugfix regarding Map encoding You can get the version that works with your Spark/Scala setup using the following table: https://github.com/Kotlin/kotlin-spark-api#supported-versions-of-apache-spark (Might take a couple of hours for Maven Central to update)

Does anyone have experience with AWS Glue using Kotlin? I’m struggling to find anything on this topic Thanks in advance

Huge and exciting news! KotlinDL is now available on Android and using this release you can run practically any ONNX neural network on your device. Moreover, the data preprocessing step has also been significantly reworked and feeding data to the network is much easier, more elegant, more versatile. Also with KotlinDL 0.5, it is possible to run models on optimized hardware which is crucial to get proper performance. There is also an Android Demo Application that shows how to use KotlinDL ONNX for Android. Learn more: https://blog.jetbrains.com/kotlin/2022/12/kotlindl-0-5-has-come-to-android/

Do you know if it's possible to create a jupyter-notebook in a kotlin-project which has access to the project classes (and its dependencies)? I know I could publish to maven-local but this always feels a bit clumsy for quick exploration/prototyping.

Noticed a new book coming out soon “Numerical Methods Using Kotlin: For Data Science, Analysis, and Engineering” (it uses the NM Dev library). Nice to see more traction of using Kotlin for Data Science. Wondering how this is different from the Java version and if the NM Dev library now has Kotlin specific usability improvements over the Java API. P.S This is not a recommendation, since I’m not too familiar with NM Dev.

Kotlin Spark API v1.2.3 is released! https://github.com/Kotlin/kotlin-spark-api/releases/tag/1.2.3 Due to high demand, we backported the project to support Java 8 for all modules (except Jupyter). You can get any of the supported versions https://github.com/Kotlin/kotlin-spark-api#supported-versions-of-apache-spark from Maven Central as usual 🙂 Let us know if you have any issues with it!

Hi everyone, I've got problems updating the Kotlin Jupyter Kernel. I followed the procedure on its website, i.e.: ``pip install kotlin-jupyter-kernel --upgrade``. It correctly downloaded and installed the

0.11.0.208

version. However, everytime I launch

jupyter notebook

and connect to it with the Kotlin Notebook IDEA plugin I get told that the kernel version (0.1.0-170) is incompatible the kotlin-dataframe version. If I scroll at the top of the log, I can see that indeed it tries to use the 0.11.0-170 version:

Kernel args: /tmp/kotlin-kernel-config-kernelProcessConnection15069667844898789499.json, -home=/tmp/kernelProcess16547460509079481210, -cp=/tmp/kernelProcess16547460509079481210/lib/kotlin-jupyter-api-0.11.0-170.jar:/tmp...

Deleting the

/tmp/kernelProcess*

directories changes nothing.

When using a kotlin jupyter notebook it fails to provide help for me (see screenshot). This works flawless from a browser. Are cell commands broken in the IDE? Related to that, how I can fetch the used version of the jupyter kernel kotlin when using it within the IDE in a kotlin jupyter notebook?

With (legacy) krangl (or dplyr in R) I could do

irisData.groupBy("Species").summarizeAt({ all() }, SumFuns.mean)

. How could I rewrite this to use kotlin-dataframe instead? Essentially I want to summarize all columns in a grouped data-frame to their mean. Conceptually, one may even want to use different aggregators here to not compute mean but also standard deviation or other aggregates at once. (cc @Jolan Rensen [JB])

🎉 🎉 Folks, huge update for the Kotlin Dataframe library is here • OpenAPI type schemas support • New JSON reading options include type clash tactics and key/value paths. • writing Apache Arrow files • ton of smaller things and fixes! Read more here https://blog.jetbrains.com/kotlin/2023/01/kotlin-dataframe-0-9-1-released/