conversation kotlinlang #announcements

If you could only += iterators/collections/other lists, then it would push you away from very slow usages of List. But, potentially it would still be un-ergonomic in some cases...

Personally I think I'd be using mutable lists a lot as local variables in functions, and using List a lot in function signatures and as class members.

camdenorrb

07/08/2019, 8:34 PM

Yeah, this shit confused a new programmer to the language

Nir

07/08/2019, 8:34 PM

(not really sure if your original post was about the cost of += or more about the type deduction of arrayOf)

camdenorrb

07/08/2019, 8:34 PM

Almost started doing that stuff

adamd

07/08/2019, 8:36 PM

looks like you really dodged a bullet 😉

camdenorrb

07/08/2019, 8:37 PM

he dodged a bullet*

Not me 😛

adamd

07/08/2019, 8:39 PM

not really sure what is confusing here 🙂

camdenorrb

07/08/2019, 8:40 PM

How?! lol

Nir

07/08/2019, 8:40 PM

I'm not sure whether he's referring to the type deduction, or the performance of +=

but both are pretty confusing IMHO

adamd

07/08/2019, 8:42 PM

I mean, what are you expecting when adding an object to an array?

i understand that this type deduction can be tricky, but why

+=

is confusing?

sxtanna

07/08/2019, 8:44 PM

In any other place its semantics indicate that it is "adding" onto its target

Here it creates an entirely new array, with one size larger, and replaces the null element at the end.

(which actually explains why it must be a var and not val)

adamd

07/08/2019, 8:45 PM

oh, I understand. But it's not an operator issue, but really immutability thing

sxtanna

07/08/2019, 8:45 PM

(since it has to replace the reference with the new array)

adamd

07/08/2019, 8:46 PM

and it's done that way by design, not by accident

sxtanna

07/08/2019, 8:46 PM

Ofc it is

That doesnt mean its a good idea xD

adamd

07/08/2019, 8:47 PM

and you cannot compile reassigning a val

so still - I get that you might not like this type of programming, but i don't get why this is confusing - it never confused me before

sxtanna

07/08/2019, 8:49 PM

It was never confusing to me

I just dont like it

adamd

07/08/2019, 8:49 PM

why?

sxtanna

07/08/2019, 8:49 PM

Because of its semantics

It implies its adding to the array like its a collection that supports resizing, when its not

adamd

07/08/2019, 8:50 PM

and how it behaves with mutable collections?

sxtanna

07/08/2019, 8:51 PM

adamd

07/08/2019, 8:51 PM

what would happen if

things

would be a mutable collection, instead of immutable?

sxtanna

07/08/2019, 8:52 PM

Depends on the backend of the collection implementation

For arraylist, the array would be resized much larger than 1 to 1 on inserts

adamd

07/08/2019, 8:52 PM

as I understand, we are talking about kotlin stdlib library 🙂

sxtanna

07/08/2019, 8:52 PM

For set, it would depend on the backing map implementation used

Nir

07/08/2019, 8:54 PM

@adamd it's confusing because appending a single element costs O(N)

And you don't really see many languages make doing something like this, so easy (to the point of including syntactic sugar to make it easier)

In languages that are "mostly mutable" like e.g. C++, push_back is an amortized operation and very fast.

In languages that are mostly immutable, your List is backed by a try immutable array implementation.

That doesn't do anything as naive as copy out the whole list.

*try -> true

adamd

07/08/2019, 8:57 PM

ok, I get your point, it's valid and interesting

Nir

07/08/2019, 8:57 PM

In Kotlin, when you can call += on a List, it doesn't actually know if it's even an immutable list or not. It just knows that it's List, or some subtype of List.

adamd

07/08/2019, 8:57 PM

I just still don't get how:

Yeah, this shit confused a new programmer to the language

sxtanna

07/08/2019, 8:58 PM

A new programmer would see that an assume it did the same thing as appending an element to a collection

adamd

07/08/2019, 8:58 PM

I mean, if I am a

new programmer to the language

, the cost of an adding an element to an array is not the first thing I'm expecting/thinking of

camdenorrb

07/08/2019, 8:59 PM

It should be something to note when using collections

sxtanna

07/08/2019, 8:59 PM

How would you explain the difference between those @adamd?

Nir

07/08/2019, 8:59 PM

Well, thinking about the constants, no,

I agree

but, if you do += on single elements in a loop

on a decent size loop

you will literally get way worse performance than e.g. python

adamd

07/08/2019, 9:00 PM

yeah, doing this on an immutable array, probably yes

Nir

07/08/2019, 9:00 PM

I guess a lot of these things are this way, because of the JVM and Java.

sxtanna

07/08/2019, 9:00 PM

well its java

adamd

07/08/2019, 9:01 PM

but it's not suprising for me that an immutable array is not optimized for lots of element adding

Nir

07/08/2019, 9:01 PM

But personally, I think languages should either just go full immutable, or they should support

const

a la C++, Rust, D

Anything other than one of these two approaches is a pretty big compromise

adamd

07/08/2019, 9:02 PM

it is more suprising that it has

operator fun

supporting adding an element

Nir

07/08/2019, 9:02 PM

Well, x += y will desugar to x = x + y

camdenorrb

07/08/2019, 9:03 PM

Can't they deprecate the plus assign function?

adamd

07/08/2019, 9:03 PM

maybe this is the issue you all are talking about - this convenient method that does not make you call

copy

explicitly

Nir

07/08/2019, 9:03 PM

You don't have to call copy explicitly.

x = x + y

is already making copies, and it's pretty explicit about it

adamd

07/08/2019, 9:03 PM

@camdenorrb because of less performance?

camdenorrb

07/08/2019, 9:03 PM

Yeah

adamd

07/08/2019, 9:04 PM

@Nir you're right

Nir

07/08/2019, 9:04 PM

So, for me, my Kotlin style guide would say: use MutableList as a local variable, and use List in function signatures and as a member of classes

and that would basically be it

sxtanna

07/08/2019, 9:05 PM

I dont think the replacement of the copy function with

is explicit enough to show what its actually doing

adamd

07/08/2019, 9:05 PM

why?

sxtanna

07/08/2019, 9:05 PM

Because its not adding to the array

adamd

07/08/2019, 9:05 PM

if it copies whole array

sxtanna

07/08/2019, 9:06 PM

Its creating a new array, and throwing the entire old one away

Nir

07/08/2019, 9:06 PM

If you use MutableList locally you avoid performance pitfalls, and local variables by their nature are already immune to most of the pitfalls of mutability. If you use List in function signatures and as members, you are protected from most of the pitfalls of mutability.

sxtanna

07/08/2019, 9:06 PM

For collections that have the

add

function, it makes sense for them to support the

operator

Because semantically, thats already there

camdenorrb

07/08/2019, 9:06 PM

It could literally be an IDE intention

To like, not do this

adamd

07/08/2019, 9:06 PM

may I ask - what are you programming on a daily basis? 🙂

Nir

07/08/2019, 9:07 PM

Who is that directed to?

adamd

07/08/2019, 9:07 PM

I guess all discussion participants

Nir

07/08/2019, 9:07 PM

Mostly C++, some python.

adamd

07/08/2019, 9:07 PM

that is a lot of interesting points I've never thought of

camdenorrb

07/08/2019, 9:07 PM

MineCraft plugins lol...

sxtanna

07/08/2019, 9:07 PM

Kotlin 90% of the time

And a good sprinkle of trashy Java

camdenorrb

07/08/2019, 9:08 PM

Almost 100% Kotlin for me

adamd

07/08/2019, 9:09 PM

ok, so you are more from KotlinNative - world than Kotlin-JVM, huh? 😉

camdenorrb

07/08/2019, 9:09 PM

Nir?

Please ping the people you're referring to lol

adamd

07/08/2019, 9:10 PM

Because I'm one of these JVM-spoiled kids, eating RAM for breakfast 🙂

sxtanna

07/08/2019, 9:10 PM

ehhh

Im JVM spoiled too, but I still am constantly concerned with abusive stuff like that

adamd

07/08/2019, 9:11 PM

So in my case we do not deal with this kind of memory issues, because we are more concerned not to be killed with our code readers

I get this, maybe this convenient method is, in some ways, confusing, but maybe some things are just left for programmers to think?

I mean, if you are doing a lot of heavy adding on immutable collection because there is a convenience method for this, I don't think that it is a language issue 😉

camdenorrb

07/08/2019, 9:13 PM

Still should be an IDE intention

adamd

07/08/2019, 9:14 PM

what? warning about copying whole immutable collection?

sxtanna

07/08/2019, 9:15 PM

language issue?

Its an extension funtion

...

adamd

07/08/2019, 9:16 PM

not really convenient for users not doing this in the loop

@sxtanna my point exactly

sxtanna

07/08/2019, 9:16 PM

Who said its a language issue?

adamd

07/08/2019, 9:19 PM

I used it as a synonime for

this shit

- maybe it was my misunderstanding, but I tried to understand where it came from 🙂

sxtanna

07/08/2019, 9:19 PM

Who said that?

adamd

07/08/2019, 9:20 PM

cam, I guess

probably ~100 messages ago 😉

camdenorrb

07/08/2019, 9:21 PM

I didn't say it was a language issue

I said it should be an intention in Intellij

adamd

07/08/2019, 9:24 PM

like I said, misunderstanding

Nir

07/08/2019, 9:24 PM

I'm just investigating Kotlin, not using it really in prod

I'd probably prefer native but don't really care

it's not fast anyway so it's irrelevant to me

(fast by C++ standards)

adamd

07/08/2019, 9:25 PM

nevertheless, interesting points was made here 🙂 I will take it into account on my next array-issues

Nir

07/08/2019, 9:25 PM

There are lots of things that suck in C++ but const is not one of them.

adamd

07/08/2019, 9:25 PM

@Nir that's for sure, but I don't think it was designed to be faster than C++

Nir

07/08/2019, 9:25 PM

I think for languages that want to have mutable data structures, and let's face it they are more intuitive for most people

then, those languages (if statically typed) should have

const

At the end of the day, 95% of what I want is just to guarantee that the functions don't modify their arguments

adamd

07/08/2019, 9:26 PM

maybe, I'm not big fan of mutability

Nir

07/08/2019, 9:26 PM

Well, Kotlin is all about mutability, so... 🙂

List

is not an immutable data structure, it's just a read-only view of some data structure, that may or may not be immutable

adamd

07/08/2019, 9:27 PM

well, have you met Java? 😉

Nir

07/08/2019, 9:27 PM

Meh fundamentally it's not any different

MutableList

inheriting from

List

, that wasn't invented in Kotlin

val x: MutableList<Int> // populate
foo(x) { x += 5 }

adamd

07/08/2019, 9:28 PM

I'm not stating it solves all my problems, but I believe that points me into a good direction

Nir

07/08/2019, 9:28 PM

If you look at a situation like this, you can see that if

foo

takes e.g.

List

and a lambda

it clearly can't assume even in the local function scope, that its argument isn't changing

adamd

07/08/2019, 9:29 PM

ofcourse, but we are talking about functional programming principles, not language itself

Nir

07/08/2019, 9:30 PM

Yeah, I agree, it makes decent compromises. The thing is when you say "I'm not a fan of mutability", if you consider my suggestion with making locals mutable and function interfaces/class members not-mutable

adamd

07/08/2019, 9:30 PM

don't blame him for not being not-functional

Nir

07/08/2019, 9:30 PM

you will see that you avoid 95% of real world issues with mutability

If not more.

Fully immutable can be good too though, but that works much better if you have proper immutable implementations, which Kotlin doesn't have.

So, in Kotlin I think you have to be pragmatic and use both List and MutableList, hence my advice

If you use List/Map/Set everywhere, especially on a team of varying experience levels, you will likely be running into performance explosions on a semi-regular basis

adamd

07/08/2019, 9:32 PM

not in my part of IT-world

like I said before, I'm JVM-spoiled-kid

we have a lot of memory to eat

I'm not proud of that, but that is a fact

if we have performance issues, it is mostly because of some misuse of a framework or library, or a memory leak

or, if somebody fetches really large amounts of data and makes calculations on them

Nir

07/08/2019, 9:35 PM

Making something quadratic that should be linear can bring things down very very very fast

No matter the world

adamd

07/08/2019, 9:35 PM

still - not my case, but I like to see other points of view - it provides me with constant improvement of my skills

Nir

07/08/2019, 9:35 PM

SO quite literally went down for nearly an hour

because they used some regex whose implementation happened to be quadratic in the number of consecutive whitespace characters

someone had a weird post with like 2K consecutive whitespace characters, boom

that's all it took

The problem is actually not really peak memory usage because the JVM is pretty smart about seeing that an object is short lived and recycling it.

It's just literally all the performance cost of copy those elements over and over.

adamd

07/08/2019, 9:37 PM

like I said - this is the reason why I took part in this discussion in the first place

this is not something I was thinking of when coding

but it is also important

and very often ignored

Nir

07/08/2019, 9:38 PM

Right. So, for me, coming from C++ which is an edge case filled language, we are used to coding guidelines. if you can come up with a good coding guideline that reduces complexity and gives you 99% of the benefit of making an informed decision in each specific case, it's a great thing. especially in a team environment.

If you use MutableList locally you avoid these problems for free, and there's almost no chance of running into other problems.

As long as you keep using List elsewhere.

camdenorrb

07/08/2019, 9:42 PM

Jit can make Kotlin and Java faster than C++ at times

Also these are mainly structure problems you're referring to, with the proper setup you won't run into these issues

ghedeon

07/08/2019, 9:45 PM

talking about

plus/+

and confusion: https://youtrack.jetbrains.com/issue/KT-9992

how about

plus

and

plusElement

🙂

Nir

07/08/2019, 9:47 PM

@camdenorrb this is one of those things that's technically true, yes.

But when you are talking about people using C++ for serious performance requirements it's basically never going to be true.

Before C++11 there were places using Java in my industry, but you really had to fight a lot to get the performance the way you wanted, turn off GC, etc

at that point it's not really idiomatic java and you lose a lot of the benefit. That sort of thing has largely vanished as C++ has improved considerably

camdenorrb

07/08/2019, 9:49 PM

What are you making exactly?

Nir

07/08/2019, 9:51 PM

HFT

camdenorrb

07/08/2019, 9:52 PM

Ah hmmmmm, idk I don't feel like nanosecond differences would be that big of an issue

Nir

07/08/2019, 9:56 PM

what do you mean, nanosecond differences?

The differences between well written C++ and well written Java can be a lot more than that

I mean, minecraft was rewritten, right.

Believe me they don't care about nanos there. They barely care about micros.

In C++ you can define a struct and you know that

vector<Foo>

is completely contiguous in memory.

That's a really big deal and not currently possible in Java.

(or kotlin)

camdenorrb

07/08/2019, 10:02 PM

Please don't compare anything about Java to Minecraft

Minecraft's code is actual spaghetti

It just really depends on how you write the code for the performance you'll get out of it, I personally haven't experienced any performance issues over the past 5 years besides when I wanted to make a screensharing program and realize the JVM impl for screenshots was hella slow

But I can just use LWJGL for that

Nir

07/08/2019, 10:10 PM

My understanding is that in minecraft it because a really difficult issue that some basic type, I think

Vec3

, which was literally 3 doubles, was pointer-indirected (like all Java types other than int, etc)

But anyhow if you are writing anything that only has to be fast enough for humans, you are operating on wildly different timescales.

When a microsecond is a ton of time you can imagine how much of a control freak you need to be.

We're regularly looking at how various C++ abstractions get compiled to assembly to convince ourselves whether the compilers can consistently optimize them out (and therefore whether we can use them freely or not)

etc

Toddobryan

07/08/2019, 11:01 PM

Note that for a generic

List

, Kotlin uses

ArrayList

, which is amortized O(1) to add elements, so it's doing the same thing under the hood, but isn't nearly so awful. tl;dr Use

List

instead of

Array

if the size will change.

Nir

07/08/2019, 11:11 PM

@Toddobryan that would be if you use

MutableList

When you have a

List

you can't perform any operations that modify the object (unless you downcast first)

x += 1

for a

List

is just syntactic sugar for

x = x.plus(1)

(assuming

is a

var

)

Toddobryan

07/09/2019, 3:15 PM

@Nir Good point. I guess what I meant to say is that it's more efficient to append a single element at the end of a mutable

List

. This is different than Scala, which uses

Cons

lists, so it's O(1) to add at the beginning, but O(n) to add at the end. Because of the way

ArrayList

works, the reverse is true in Kotlin.

6 Views

#announcements Join Slack