https://kotlinlang.org logo
Title
n

Nusret Özateş

02/23/2020, 9:55 PM
Hi everyone! How can use krangl with Smile library? For example with Linear Regression (OLS in Smile)
a

altavir

02/24/2020, 6:44 AM
I do not think that it is possible to do it out of the box. You need to convert data from krangl internal representation to smile internal representation. I am currently working on Table API that could allow to do it autimatically in future, but it is in early stage.
Also you can try to ping @holgerbrandl. Maybe he has some kind of solution for that.
n

Nusret Özateş

02/24/2020, 8:09 AM
Okay thanks a lot! I am trying to do a machine learning project ( neural network project actually) this semester and I want to do it in Kotlin. So I'm trying to find my tools that I can work without problem😊
a

altavir

02/24/2020, 9:27 AM
The ecosystem is really youg. Please fill free to explain your problem and we will try to preorityze tools for it. I've already implemented auto-transformation of different matrix representations in kmath. I want to do the same for tables sooner or later.
n

Nusret Özateş

02/24/2020, 12:35 PM
Well unfortunately I cannot find a research problem for now😂 But this semester I need to find a research problem that can be solved with Neural Networks and write a nice documentation/article about it. And I like Kotlin a lot and want to add something to community with that project 😊 If you have an idea to do with Neural Networks I can work on it too!
a

altavir

02/24/2020, 12:38 PM
What we really lack right now is the problem description. People coming from python usualy just say "do it like pandas/scipy/whatever". Kotlin actually could do better since it allows for more functional-style transformations so in order to do something, we need first to understand what needs to be done without references to existing Python solutions. Typical example is the recurring theme of large in-memory files. You usually can't avoid them in Python since that is how NumPy works, but in kotlin we can do streaming API. I personally do not do machine learning so it hard for me to understand what is needed.
Also there are some ideas to unify the API of mathematics and data access libraries to add a Python-like experience where you can expect different libraries adhere to the same interfaces for data structures.
n

Nusret Özateş

02/24/2020, 12:46 PM
For now I am looking for a workflow, for example in Python when you think about it Pandas to get and play with data Numpy to do mathematical/matrix operations Matplotlib to show data as graph Sklearn to do machine learning things like linear regression logistic regression etc Maybe tensorflow for neural networks and machine learning ( not sure about it) And they work with each other pretty well For Kotlin Pandas = krangl Matplotlib = kravis Sklearn and Tensorflow = Smile and Deeplearning4j and OjAlgo Numpy = Nd4j But for example Krangl dataframes cannot be smile dataframe, I need to convert it manually (I couldn't) or use Tablesaw library, but krangl works good with kravis since same person made them but I don't know how tablesaw and kravis will work with each other
a

altavir

02/24/2020, 12:51 PM
I will add krangl bindings for DataForge-tables, and there are some bindings for ndarrays in kmath (ND4J is planned) but it is a bit more complicated in between. The problem is that pandas-like reader produces typed columns and it is not exactly the same thing as dense nd-arrays. You need to make a transformation of sorts and we need to think about the way tables are transformed into number arrays.
n

Nusret Özateş

02/24/2020, 1:10 PM
You are right! Okay I will try to find my way after I learn more about neural networks, data science and after I decide my reaserch problem. It seems I can ask more detailed and helpful questions only after I decide what will be my project😂
a

altavir

02/24/2020, 1:11 PM
Feel free to leave feature requests in kmath repository. I will separate table api from dataforge repo in near future and then we can discuss it more.
n

Nusret Özateş

02/24/2020, 1:12 PM
Okay thanks a lot again! 😊
Oh hey what will be the advantages of KMath over Nd4j?
a

altavir

02/24/2020, 1:15 PM
Kmath is a wrapper. It has naive implementations for multiplatform, but basically, it delegates to platforms-specific libraries. ND4J is not implemented yet, but is planned: https://github.com/mipt-npm/kmath/issues/52
And it is possible in kmath to implicitely convert data from one library representation to another one... need to add an example about that.
n

Nusret Özateş

02/24/2020, 1:26 PM
I get the idea and it seems awesome!
a

altavir

02/24/2020, 1:28 PM
I proposed the same mechanics to Kotlin for DS team for other math tools and we are discussing it.
j

jimn

02/28/2020, 10:53 AM
you can upload a bigquery table and run linear regression wtih about 15 minutes of RTFM, fwiw. it gets a little more involved if you want to upload a tensorflow model; and even though you are paying gcp for processed gigabytes, and not compute time, your time spent doing any kind of bigquery operations will be better spent offline in any local tool at all that you are familiar with, so you can get quick linear regression results with a well formed dataset you prep offline
I also showed up in the channel and asked about the same basic usecases, and wrote a tool in Kotlin that does what pandas does. I've spent no time at all dealing with smile, but if it is supported by a jackson adapter it's probably a few lines of code to set up a serialization solution to do some transforms.
i have come upon some time if you want to PM me. my wife is in Bali and I've ended my current engagement so I planneed to dedicate a few cycles to the tool.
h

holgerbrandl

05/15/2020, 2:38 PM
@Nusret Özateş Indeed the krangl repo contains some example where we use smile to model data. Feel welcome to contact me via github or in case you have any questions or comments.