kotlin has a better "score" than java, to be sure,...

# datascience

[jimn]

kotlin has a better "score" than java, to be sure, for most situations. java is low entropy. The reason why Java is viable at all is because in early 90's, c++ was dominant, and there were a ton of limitations to the available late binding options; in c++ at the time, progress was generally a code-generation tool or or a precompiler of c++ extensions which emitted c++ virtuals.

I don't think that the jvm evolution should stop with the current decisions of kotlin, personally. data science is heavily dependant on information thoery and to ignore the signposts where a given solution depends on approaching Shannon's limit is to reverse the efficiency of the solution in question. intelligence is based on comprehension. comprehension is the orthogoanl arrangement of information to align meaning with content. code is ultimately an expression of data.

[altavir]

Sorry, but all this talk about entropy does not make any sense (I am not from comuter sceince, but I give lectures on mathematical statistics so I know a few things about information). Algorithm complexity does not have anything to do with API or language design.

[jimn]

what part of language doesn't directly define expression complexity ?

[altavir]

It is not about language, but about algorithms. We are not talking about algorithms here

[jimn]

what part of algorithms are not directly expressed in language, therefore, an expresion with complexity?

[jimn]

if Kotlin defines a minimum syntax no more terse than the IR that llvm emits, would there be any benefit to not using that IR directly?

[altavir]

There were a few efforts to formally measure language, but as far as I know, results never correlate with usability. And yes, language design has nothing to do with algorithms. Algorithms in most cases are implemented by basic loops with imperative code. If you do not write in Haskel.

[altavir]

You can write everything in assembler if you like.

[jimn]

s/assembler/kotlin/g <<"`all prior @jimn comments in here`"

[jimn]

you make a compelling case for haskell

[jimn]

when i read formula for quantization of electromagnetic fields, there are iterative algorithms using symbolic operators

[altavir]

Forget about symbolic operations. It is a completely different world which does not intersect with numerically efficient algorithms. If you want symbolic, take Mathematica or Mapple and live with that.

[jimn]

they were replaced by pandas

[altavir]

nope, pandas does not have anything to do with symbolic algebra.

[jimn]

having a basis for turing complete metaprogramming seems impossible for the outcome you're championing. I expect somehow to hear "it's a mistake because it's too complicated" argument any minute.

[altavir]

I think we are not talking about the same thing.

[altavir]

And I am not championing anything. Just trying to understand what do you want. No luck so far.

[jimn]

yes, a guy named Fred, or Alex can decide what to name a function pertaining to a mathematical operation, can specify his idea of the best compromise in verbosity, flexibility, and document it in his favorite language. then i can be a human compiler of the fred/alex methodology and transcribe an algorithm to said compromise language

[jimn]

papers tend to publish mathematical notation, assuming somehow that the reader understands all the context necessary to apply that as proof of the paper's premise. sometimes pseudo-code, but generally something where the algorithm to take credit for is in an notation that isn't ready for an execution context.

[altavir]

It is exactly the resason why algorithms spend tens of years bering written in a papaer, but not-implemented. It takes some effort and completely different skill set to create numerically-effective algorithm implementation.

[jimn]

if i choose kotlin as the basis for an expression evaluator what are the upsides of that over java or python options which already exist? what is the expected readability of such a codebase? I can think of very, very few examples of imperative languages that lend readability to non-trivial indirection like expression evaluation. in c++, through unplanned consequences of functional purity, you can shape a new grammar of compilable c++.

[jimn]

java generics were designed expressly to prohibit such things but to hold onto the generic flexibility of "typesafe" benefits

[jimn]

you're proposing that algorithmic complexity is a programming domain where the details of the expression language are not significant departure from fortran 77 or BASIC.

[jimn]

something baroque lets you make code run really really fast, and "why the hell would you worry about reducing the syntax complexity" is a hallmark

[altavir]

Kotlin has a lot of nice trick to simplify code (not algorithms). But if we are talking about scientific programming, you can try to read my articles on context-oriented design. It opens principially new possibilities in some places.

[jimn]

kotlin has a lot of nice tricks, agreed. syntax is kittywumpus at this point in time to my eyes. at least for having so much promise in advancing so many different directions, syntax is not on the map at all

[jimn]

maybe borrwing from c# syntax and golang execution makes everybody happy, but it's just prolonging the gap between execution and expression contexts.

[jimn]

i think that the current decisions relating to kotlin syntax will most practically fail to create a compelling move from numPy as the basis for machine learning. it's great that some features may be more teachable than java in compsci 101 but that's not the consumers of ML

[jimn]

having flexible operator overloading and metaprogramming as a goal does not need to impact the propeller-head algorithmic efficiencies of kotlin-1.3 or whatever. but veto'ing advancements in this direction because java programmers might encourage anti-patterns is just corporate speak job security

[jimn]

im not really talking about simplifying code, im talking about reducing the complexity to express new concepts using the kotlin language, which would by definition best support inline language extensions instead of building gradle projects with antlr code.

[Amir Gur]

+1 for even more syntax simplicity goals / tricks in Kotlin.
Will help adoption in data science and in general.

[jimn]

kolmogorov complexity is a succinct scale against which we can assign the concept of aleph-null as it relates to the states that can be expressed using a given language, within the relevance of human scales, e.g. time, attention, even lifespan, of a person to comprehend, catalog, and later assign value to the available states. our languages; machine programmable or otherwise, impose costs of time and energy to use, transmit, process, compile, interpret, etc.

the understanding i have is that for kotlin we set the bar to an incremental improvement over an uncomfortable predecessor; however that predecessor is not python, or haskell, or latex output, or 7 dimensional bitmap matrices, but is essentially a java reboot. applying a more expressive language of equal performance capability stands to reduce the number of states associated with a program and its creation process.

the physical simulation world depends on scaling up. when i was working on the bleeding edge of modellica simulations there were clunky systems language compiler implementations(MSVC and MFC), flexible but buggy implementations on the jvm, and there was a mainly python open source project getting the job done in the usual advanced degree community that had occupational skills but lacked programming elegance. it's been a few years since i had any part in that but it absolutely defines a relentless breadth of numerical expression and library capabilities that don't match the available programming options well. this doesn't even begin to broach the fact that physical system simulation and fluid dynamics really don't share a lot of common ground and there's a never-ending landscape of reinterpreting finite systems to fulfill engineering niches. the impression i had of this scene overall was that the code was just too ugly and too far of a distance from the end-users to have more than a handful of individuals who could understand the customer demands and the implementation strategies and those individuals have almost no time at all to influence the business conditions that drive moementum ahead of cleanup in this area.

[jimn]

we don't need another BLAS interpretation written in kotlin, though it would be nice. what would be handy is to dissolve the boundary between assembly(gradle) conventions and compiled code, and to work with the concept of metaprogramming and compiler-resident data at scale. java classes have a hard compilation limit of 64kb methods and symbols. this makes natural language processing efficiency a joke with jvm tools. pandas is no better, the shims and GIL and the native wrappers blow out the heap beyond practical hardware limits. grid-computation you say? in kotlin? the best of all worlds? i think that's a good proof of concept at least. but i don't feel like kotlin is addressing complexity in a way that reduces the bloat of handling complex data and state. @altavir I don't know what you lecture about, or what problems you are solving, but these are the progress markers that will move the needle, beyond the group-by data manipulation shells im aware of to date.