In the Data Science podcast https://talkingkotlin.com/catching-up-with-data-science-at-jetbrains/ @roman.belov talks about “type providers” I found the concept fascinating but I wasn’t able to find any information about Kotlin implementation. Does anyone have pointers about that?

Hello! You may start from reading this code along with this library

thank you!

There was a lection about that in Russian by Roman Belov. I am not sure from your name if you are Russian-speaking, but here you are: https://research.jetbrains.org/ru/news/kotlin-dataframes-by-roman-belov. And here is a YouTube link:

You may be also interested in this PR https://github.com/JetBrains/kotlin/pull/3516

I've also enjoyed listening to this podcast. Still I'd like to read/talk more about it. E.g. is it Jupiter only or would it also work in the ide? Where are under the hood created classes are deposited? How to try it out? I'd have tons of other questions 😊

@holgerbrandl actually your library was chosen as a first target and there is already wrapper for it. The video I refenced is in Russian, but you can probably understand what happens from looking into the screen. As for IDE, it is in theory possible to port this functionality to IDE worksheet mode and REPL, but as far as I know, no concreete plans for that.

Thx for the pointer. So it is Jupyter only. Though I dont want give up my beloved ide yet🙈. Maybe kotlin-kernel integration into ij is the way forward?

Type provider API PR which I mentioned above is a kind of plugin for Kotlin scripting compiler, so it may be used in any place where scripting functionality is used (but not in IDE now, hopelessly)

@Ilya Muradyan I believe that both scratches and worksheets use embedded scripting compiler. So I guess it is not that hard to add the functionality. Of course, we will need to have module deffinition for the plugins. And I really looking forward for those.

Classes created now exist in form of Class instances, they are stored in memory, and they are also saved in files (mostly for purposes related to Spark). If you're interested in source code for these classes, enable

%trackExecution

magic.

Scratches and repl are great, but at the end of the day I always need to convert my data workflow into some kt/kt to operationalize the solution. I wonder how compatible the type provider is with such a more operandi?

You can copy generated classes and interfaces from the notebook and paste them into IDE. And it will work

👍

@holgerbrandl Not without significant changes in the runtime or idea plugin. It is not hard to generate a class wrapper on-flight, but it is much harder to make IDE understand that the new classes are created. It should probably be possible to create a compiler plugin that will make it seem like we have staged compillation, but it is a question for future.

Yes, type providers in kernel are already available, but were not publicly announced because of their non-stable API

@altavir Thanks for further links! I do understand Russian, let me take a look at that video 🙂