Hey I remember this Slack Channel being referenced somewhere

Question

Hey I remember this Slack Channel being referenced somewhere related to the Kotlin Jupyter project Has anyone done any working on embedding the Kotlin Kernel into another application context

altavir · Answer

Do you mean non notebook or non jupyter

altavir · Answer

If you are talking about something non notebook than you can use kotlin scripting for that jupyter kernel is built on top of it As for other ipython environments the work is being done with them as well Jupyter is just the most popular environment so it goes first

Florian Magin · Answer

I am not sure if we are using the same terminology For me Jupyter is the project overall and the general architecture of kernels that you attach to Notebook is the Notebook frontend that has some issues like requiring that the notebook process started the kernel The thing I want to do which might just work that easily is to start a thread in an application that invokes the main method of `kotlin jupyter kernel jar` which I should then be able to attach too I already tested that I can simply start this jar with `java jar ` in the same way the Notebook does and I will then have a kernel running that I can attach to with `jupyter console existing`

altavir · Answer

If you want a console REPL you do not need Jupyter at all You can use Kotlin scripting directly and it gives a lot more freedom Jupyter kernel is created specifically for ipython transport protocol server

Florian Magin · Answer

ipython transport protocol server is somewhat the goal because this means I can attach a CLI terminal or qtconsole to it and don t need to implement the GUI inside the application myself

altavir · Answer

Ok this is the second part of my answer I know that currently work being done to use kotlin kernel for other ipython servers as well Like Zepelin or Idea own ipython plugin I think this work will be more intensive once the jupyter kernel is stabilized

Florian Magin · Answer

But if kotlin scripting is significantly easier overall I can use it as a first prototype Could you point me to an example for kotlin scripting that is able to read in run arbitrary kotlin code and evaluate inside the full Java context available

Florian Magin · Answer

What do you mean by ipython servers

Florian Magin · Answer

For context I have dealt with a lot of IPython related code but only actual Python code and have built projects already that embed a python kernel into an application that provided a Python interpreter and the possibility of Python plugins

altavir · Answer

ipython is a protocol In order to use it you need both the server and the client Kernel is connected to the server via ZMQ pipe Jupyter framework has both

altavir · Answer

Kotlin scripting examples could be found here <https github com Kotlin kotlin script examples> For REPL you can see jupyter plugin source code it is not very hard to read

Florian Magin · Answer

Ok I think this is where I am getting confused IPython for me is the simple ipython program that IIRC doesn t involve ZMQ or kernels

Florian Magin · Answer

`jupyter console` looks like IPython on the surface but does different things in the background AFAIK

altavir · Answer

But how does the data from your evaluation gets to your browser

Florian Magin · Answer

In what setup With jupyter and python or with my vision of the kotlin setup

Florian Magin · Answer

Or with pure IPython

altavir · Answer

I am trying to explain how does Jupyter works You need a web server in order to communicate with Jupyter Zepelin whatever frontend Your REPL code does not run any server

Florian Magin · Answer

I think I can simply prove you wrong there

Florian Magin · Answer

I can run ``` java jar home fmagin local lib python3 8 site packages run kotlin kernel jars kotlin jupyter kernel 0 8 2 61 jar classpath= home fmagin local lib python3 8 site packages run kotlin kernel jars jupyter lib 0 8 2 61 jar home fmagin local lib python3 8 site packages run kotlin kernel jars kotlin script runtime 1 4 20 dev 2342 jar home fmagin local lib python3 8 site packages run kotlin kernel jars kotlin reflect 1 4 20 dev 2342 jar home fmagin local lib python3 8 site packages run kotlin kernel jars kotlin stdlib 1 4 20 dev 2342 jar home fmagin local lib python3 8 site packages run kotlin kernel jars kotlin stdlib common 1 4 20 dev 2342 jar home fmagin local lib python3 8 site packages run kotlin kernel jars annotations 13 0 jar home fmagin local share jupyter runtime kernel 641b8904 e16a 483b be0b 58b6c73b8412 json home= home fmagin local lib python3 8 site packages run kotlin kernel```

Florian Magin · Answer

On my terminal This is the copied process arguments that normally `python m run kotlin kernel home fmagin local share jupyter runtime kernel 641b8904 e16a 483b be0b 58b6c73b8412 json` would start

altavir · Answer

I am not sure what you a trying to prove Kotlin kernel is a stand alone process but it does not communicate with browser It communicates with jupyter server Here is the appropriate page from documentation <https github com Kotlin kotlin jupyter blob master src main kotlin org jetbrains kotlin jupyter repl kt>

Florian Magin · Answer

I can attach to that kernel with `jupyter console existing= home fmagin local share jupyter runtime kernel 641b8904 e16a 483b be0b 58b6c73b8412 json`

altavir · Answer

Sorry wrong link <https jupyter client readthedocs io en stable messaging html>

altavir · Answer

Exactly Jupyter ups the server and communicates with kernel via ZMQ

altavir · Answer

To summarize Yes Kotlin Kernel runs repl in a stand alone process but it designed to communicate with jupyter via jupyter specific protocol I am not sure if other ipython servers share the same protocol

Florian Magin · Answer

And the thing I want to do is that my application starts the kernel instead Then jupyter console can attach to this kernel

Florian Magin · Answer

No notebook process needs to be running for all of this

altavir · Answer

Yes it should be quite possible right now

Florian Magin · Answer

Do you have an expectation of how well that kernel will then be able to provide access of other libraries of the JVM it is running inside Does importing simply work or are there some pitfalls to expect

altavir · Answer

As I already said kotlin REPL is working in a separate process so you will be able to access the same functionality as in notebook including plots in browser with plotly kt blush

altavir · Answer

Just started kotlin kernel in console mode it is fully functional The only problem is that kotlin kernel dumps a lot of debug info into the console and bloats the output < Ilya Muradyan> I think it should be somehow fixed in future

altavir · Answer

And here is the plot

Florian Magin · Answer

But my goal is that it isn t a separate process though and can access the entire API that the application provides

Florian Magin · Answer

While also running inside the context of the application

altavir · Answer

I am not sure what you mean Do you want to run Kotlin REPL inside kotlin application

Florian Magin · Answer

I understand that my use case is probably really weird

Florian Magin · Answer

Florian Magin · Answer

Florian Magin · Answer

Florian Magin · Answer

are some examples of this approach for other tools used in reverse engineering

Florian Magin · Answer

The tool I want to integrate the REPL into is <https github com NationalSecurityAgency ghidra>

altavir · Answer

If it is Java Kotlin project then you do not need to use IPython at all just call kotlin REPL directly Few years ago I did it with Groovy an Kotlin supports several ways to do that as well Just see the link I ve shared below the thread You just need to copy some code form that file without including any IPython machinery Also isn t Hydra and NSA seems a bit brrr I am a bit of Marvel fan

Florian Magin · Answer

gt Also isn t Hydra and NSA seems a bit brrr I am a bit of Marvel fan I don t understand what you are saying with that Ghidra is as far as I remember a reference to Godzilla

Florian Magin · Answer

I think I ll test a really basic kotlin REPL first then and see if this works as I imagine it and if not I will have more concrete questions to ask Thank you for your time and patience in helping me slightly smiling face

altavir · Answer

Kotlin REPL is in experimental phase but is is fully functional And there is a lot of information in < C0BT46EL8|scripting> about that

Michal Harakal · Answer

Discussion in TalkingPodcast points out some interesting informations regarding jupyter kotlin <https talkingkotlin com catching up with data science at jetbrains >

altavir · Answer

< Michal Harakal> It is a little bit old There is a more recent exciting presentation by Roman in JBR channel but it is in Russian There is also my article Both Kotlin Jupyter team and module developers made a lot of progress since then Still the question is not solved by Jupyter in this case

Michal Harakal · Answer

< altavir> could you pls point me on that russian talk I didn t find it BTW your medium article is very useful

Ilya Muradyan · Answer

There is also my talk for team in Russian about kernel internals I can share it if you wish

altavir · Answer

< Ilya Muradyan> Oh I would like to hear this one as well I know how it works but it is always interesting to hear interesting things again

Michal Harakal · Answer

I am also interested in