Hi, I would like to recieve opinions on Kotlin for...

# datascience

[Filipe Duarte]

Hi, I would like to recieve opinions on Kotlin for data science and scientific research. I'm have been using Python but know I want to invest some time in learning another language with tools good performance. So, I started to learn Scala and have been enjoying, but I saw some articles showing the tools developed for Kotlin in respect to libraries for deep learning and differential programming. #datascience #science

[altavir]

What scientific field do you have in mind?

[Filipe Duarte]

Machine Learning and computational intelligence. Also, I work with applications in Finance and time series forecasting.

[altavir]

Well, I am not familliar with the field (we could ask @Ролан who comes from computational finance), but I think that Kotlin has very good perspectives here, at least as default language. The number of available libraries is still small compared to Python and Julia bindings, but the relyabllity is much better. Also the tooling for application and library development is supreme. The notebook experience is not the same as in python yet, but it is already quite close and I think it will be even better in future.

[Filipe Duarte]

In terms of speed, I tried using Python with numba but it felt so sensible and not compatible with a lot of python code. Also, the lack of static typing and compilation really intrigues when the code breaks after several hours.
Julia was my first bet, but it does not have tooling and static types which I think are essential for building reproducible and stable projects in scientific projects and applications.

Thanks for helping me with this. I like Scala's tooling ... Intellij Idea help me a lot. But I'm thinking about the future. So, I thought maybe Kotlin will overtake Scala because of its simplicity and investment.

[Filipe Duarte]

I use jupyter notebook for exploration and scripts with Pycharm for executing scientific experiments and applications programs for data science in finance.

[altavir]

I have experience both with Python and Julia. Python is OK for small scripts and is overall good as long as you do not do actual new code development in it (so it is good for non-programmers). Julia is a nice language, but lack of proper tooling and LLVM-based JIT are its curse.

As for Scala... the problems is the same as with C++, good scala developers are very rare and expensive. If you intend to grow, you will have problems finding them. Kotlin is not as flexible as Scala, but it allows to achive almost the same level muuuch faster.

[altavir]

As for perfomance. The peak performance is more or less the same for all compiled languages (including C++, kotlin and Julia). It is always a question of how hard it is to obtain it. In my experience for average program, it is important how tolerant the language and compiler is to programmers mistakes. And JVM is quite tolerant.

[Filipe Duarte]

Good points! I need performance because experiments in mahcine learning and finance take a lot of time. And time series is a pain because we can't vectorize a lot of things.

Do you have some guidelines and material to start learning Kotlin for science besides Jetbrains' website?

[Ролан]

@Filipe Duarte I recommend you to start with krangl http://holgerbrandl.github.io/krangl/ by @holgerbrandl

[zaleslaw]

@Filipe Duarte welcome to #kotlindl channel and read KotlinDL tutorials for building Deep Learning models https://github.com/JetBrains/KotlinDL

Tooling libraries are at the baby level, of course, and DS Community in Kotlin is young and small comparing with Python, but it will be better than Scala DS ecosystem in the nearest future, if not yet

[Filipe Duarte]

Awesome, krangl is similar to pandas.
KotlinDL uses DL4J as backend? Is there some lib similar to PyTorch? I saw that for machine learning I could use Spark.

[Filipe Duarte]

Is Facebook creating a Torchlib for Kotlin?

[zaleslaw]

It's TensorFlow under the hood in KotlinDL, to taste a PyTorch engine you could try java library DJL From Amazon, it Supports inference on PyTorch. Unfortunately, Facebook doing nothing for wide JVM support for PyTorch Also, as I know, @Ролан experimented with PyTorch in Kotlin, but it's not released yet

[zaleslaw]

For ML you could use Smile library, it has Kotlin API, and it's better than Spark ML

[Ролан]

@altavir which Kotlin wrapper for Spark were you recommending me the other day?

[Ролан]

@Filipe Duarte don't forget quantlib which has Java bindings http://www.jquantlib.com/en/latest/

[altavir]

Official one by @asm0dey: https://github.com/JetBrains/kotlin-spark-api

[Ролан]

Of course, if you are into finance then kdb has an allrightish Java API as well.

[altavir]

Actuallt there is a #kotlin-spark as well as #kotlindl here. If we are talking about krangl-like things, there is an incubating https://github.com/nikitinas/dataframe, which does more or less the same thing but in more kotlinized way. There is also my own multiplatform implementation: https://github.com/mipt-npm/dataforge-core/tree/dev/dataforge-tables, but it is on hold for the moment because a lot of other work.

[Filipe Duarte]

What? So timely: https://twitter.com/kotlin/status/1380542018439544838?s=20

[Filipe Duarte]

Krangl deal with null values? I read a csv file with empty values and the DataFrame.readCSV function converted a numeric columns as String.
Also, is it possible to call Scala or Java libraries in Kotlin using Gradle?

[holgerbrandl]

Yes it it models missing data as null

[holgerbrandl]

Yes you can call scala from kotlin, at least via libraries

[Filipe Duarte]

@holgerbrandl Do you have a tutorial explaining how to do this?

[holgerbrandl]

There is http://holgerbrandl.github.io/krangl/ . I'm happy to extend it it needed

[altavir]

Kotlin-JVM has seamless integration with Java. For Scala it would depend on Scala library. If it is compatible with Java, the call from kotlin will lool the same way as from java

[Filipe Duarte]

@holgerbrandl thanks you.
@altavir ok!