https://kotlinlang.org logo
Title
n

Nikky

02/23/2019, 9:53 AM
is there some kind of standard realife task that is used fow notebooks ? i am writing something very similar that can process data in multiple steps and visualite it now it would be useful to copy some sort of sample usecase from somewhere that can be done using jvm libraries
a

altavir

02/23/2019, 10:00 AM
Not sure I understand the question
There is a beakerx jupyter extension and it in dire need of kotlin evaluator replacement
Here is the issue https://github.com/twosigma/beakerx/issues/7894. Help there would be very welcome. I intended to do it myself, but I'am currently very short on time.
n

Nikky

02/23/2019, 10:02 AM
well i started making my own completely seperate from jupyter, @amanda.hinchman-dominguez is doing the tornadofx vizualization we are just not happy with the need to run a webserver to run kotlin snippets
a

altavir

02/23/2019, 10:05 AM
Jupyter has its limitations, but everyone use it and I am not sure it is possible to create a standalone visualization framework with comparable capabilities. Beakerx already has some visualization capabilities for JVM languages and, what is much more important, autotranslation feature to move data between python, and, say, kotlin. It seems reasonable to invest in it instead of creating something completely new.
n

Nikky

02/23/2019, 10:05 AM
also a few other things.. output of pages is cached, any page can depend on data from multiple other pages, output of a page can be reused multiple times and filewatchers ensure a script is reevaluated on change and all steps that were invalidated run again combine that with intellij idea as the IDE and tornadofx visualization and some static file outputs.. i think it would be something i'd be happy to use
a

altavir

02/23/2019, 10:07 AM
Indeed, but this project is impossible to manage with one or two people team. I think it is important to separate back and front-end. If you have good back, than you can use it for jupyter and then create your own front. My team has some plans for such front, but it is a huge task.
Kotlin is currently not very appealing for scientific people due to lack of proper visualization. We need to create something that will just work as soon as possible.
n

Nikky

02/23/2019, 10:10 AM
data2viz is getting somewhere i think
but yes the visualitatiuon is probably where this will make a difference or drown
a

altavir

02/23/2019, 10:11 AM
Not sure about it. The charts are propreitary which is not acceptable for science and you need some kind of environment to run them. Just remember that scientists are not programmers.
Beakerx is a good platform, so any help with it would be appreciated.
n

Nikky

02/23/2019, 10:13 AM
well i sarted this project as a test for how much kotlin scripting is ready to use .. and one of the features there is the excellent IDE integration can i work with beakerX having the notebook open in idea ?
a

altavir

02/23/2019, 10:14 AM
I don't know. Never tried it. It is just regular jupyter notebook.
n

Nikky

02/23/2019, 10:15 AM
well either way feel free to pick my brains about scripting, but i have not really plans for making a jupyter thing in the near future but i am sure the core code can be ported to BeakerX
t

thomasnield

02/24/2019, 5:14 PM
If you're playing with notebooks for the first time, this is worth watching

https://youtu.be/7jiPeIFXb6U

n

Nikky

02/24/2019, 5:44 PM
yeah i saw that one.. it partially inspired me that i need to make something i would actually want to use
a

amanda.hinchman-dominguez

02/24/2019, 5:49 PM
Thanks @thomasnield! We're making some good progress. We'll be sure to show you when we're a little further along
a

altavir

02/24/2019, 7:05 PM
By the way, I also do not like notebooks. The problem is that IDEA is too heavy weight to do quick-fix things that are usually needed. Also, a notebook has this markdown rendering capabilities that allows to document the process as it goes. What I would really love is a notebook with fixed cell dependencies, where changing something above, automatically invalidates everything below. I think it is possible to achieve that behavior with dataforge-like executor and plain jupyter notebook, but I just don't have time for it now.
☝️ 1
n

Nikky

02/24/2019, 7:46 PM
change something and it invalidates and reruns all cells (i call them pages) below it? seems familar.. i implemented that yesterday
now the next big hurdle is to get the UI going
o

otakusenpai

02/26/2019, 1:31 PM
It's a humble opinion, but couldn't a standalone Kotlin specific editor plus C/C++ support could be more useful ?
a

altavir

02/26/2019, 1:33 PM
@otakusenpai Not sure what are you talking about and what does C++ have to do with this
o

otakusenpai

02/26/2019, 1:33 PM
idk, I'm quite new to Data Science. Though C++ doesn't have a need here
a

altavir

02/26/2019, 1:37 PM
You should avoid writing anything C++ unless you have a lot of experience with it or you are creating games. Kotlin is very good language, but the problem that it lacks sufficient scripting and visualization support favored by "data scientists" for "quick and dirty" style programming. Bare bones compiler won't help you in that.
o

otakusenpai

02/26/2019, 1:40 PM
How do you apply faster computation in higher level languages such as kotlin? I have had 4 years of C++ experience but it was mainly normal stuff(just C++11, not much STL besides strings and vectors and a few other algos). Basically what I learnt on my own.
There is JNI, which can help in accessing native computing power
a

altavir

02/26/2019, 1:44 PM
You should understand that "fast programs in C++" is for most part a myth. One can write something super-fast in C++, but not your general level programmer. In my experience, scientific programs in C++ in most cases are slower in real life just because people who write them are not that experienced. Latest JVM test show that without specific code optimization it is as fast or even faster that native program in C++. The exceptions are heavy duty super-optimized native libraries like tensor flow, but you can use JNI for them.
You can count the number of libraries you need native-base acceleration for by fingers on one hand... I can remember only tensorflow, because matrix algebra in libraries like ojalgo or EJMA are already as fast as native BLAS.
o

otakusenpai

02/26/2019, 1:51 PM
Hmm.... I see that your kmath library is fully Kotlin based.... yeah i can agree that using Kotlin is more type safe and generally productive to use than C++....
a

altavir

02/26/2019, 1:53 PM
It is not in fact fully kotlin based, I t just does not yet have native part which will depend on native libraries. It has an interconnection with koma, which does have some native wrappers.
o

otakusenpai

02/26/2019, 1:53 PM
Hmm
a

altavir

02/26/2019, 1:54 PM
Koma also has some wrappers for native BLAS libraries, but recent study shows that on JVM one just does not need them with JDK 11.
o

otakusenpai

02/26/2019, 1:55 PM
Does the probability part depend on Statistics and Probablity? Like Poisson or Bernoulli(stuff taught to us during 2nd year). I'm not that great in Maths though.
a

altavir

02/26/2019, 1:56 PM
I have to work with some C++ libraries and it is my sincere belief that with current level of tooling the language is unsuitable for general purpose scientific development. Maybe if they get modules and repositories...
The probability part of kmath is not implemented yet. But yes, it will include distributions and a lot more since mathematical statistics and Bayesian methods are my specialty.
I have them in older project and will gradually migrate everything into multi-platform kmath.
o

otakusenpai

02/26/2019, 1:58 PM
Yeah I too want to install JDK 11. But I'll be using some other JVM like Eclipse OpenJ9 or something like that.
Hmm noice 😬
a

altavir

02/26/2019, 1:59 PM
There is OpenJ9 for JDK 11 in adoptopenjdk package. But I did not have time to play with it yet.
I don't know if it supports nice vectorization features from oracle pack
n

Nikky

02/26/2019, 2:00 PM
looks cpp does not actuallyc unlock any faster processing
its not gonna make your program magically speed up and its not suited for scientists
a

altavir

02/26/2019, 2:01 PM
@Nikky it unlocks better flexibility. You can do things faster in c++, but in most cases there are a lot of ways to things slower and bugier.
n

Nikky

02/26/2019, 2:01 PM
writing code is definitly slower
a

altavir

02/26/2019, 2:01 PM
correction: You can do things faster in c++ if you now how and have a lot of time to debug everything
n

Nikky

02/26/2019, 2:01 PM
and that why python and R are so poular with data scientists i guess
a

altavir

02/26/2019, 2:01 PM
yep
n

Nikky

02/26/2019, 2:02 PM
but writing code in kotlin is fast too,.. the problem is all the setup around making a kotlin project
gradle and whatever
a

altavir

02/26/2019, 2:02 PM
err.. have yo tried to setup a project in C++? Let's start another thread.
n

Nikky

02/26/2019, 2:06 PM
well cpp is very much offtopic.. this thread cwas supposed to be aboutsome sample to showcase our notebook with
a

altavir

02/26/2019, 2:09 PM
yep. Slack really needs functionality to split threads