https://kotlinlang.org logo
Title
c

camdenorrb

07/08/2019, 8:26 PM
n

Nir

07/08/2019, 8:30 PM
The whole handling of immutable in Kotlin is... pretty interesting.
It's not immediately obvious how expensive += can be.
On the other hand it's extremely convenient in some cases.
I think maybe += for Array would actually be better if it only had the overloads that accepted a bunch of new elements at once
When += is defined for adding a single element it makes it rather easy to write code that +='s in a loop
If you could only += iterators/collections/other lists, then it would push you away from very slow usages of List. But, potentially it would still be un-ergonomic in some cases...
Personally I think I'd be using mutable lists a lot as local variables in functions, and using List a lot in function signatures and as class members.
c

camdenorrb

07/08/2019, 8:34 PM
Yeah, this shit confused a new programmer to the language
n

Nir

07/08/2019, 8:34 PM
(not really sure if your original post was about the cost of += or more about the type deduction of arrayOf)
c

camdenorrb

07/08/2019, 8:34 PM
Almost started doing that stuff
a

adamd

07/08/2019, 8:36 PM
looks like you really dodged a bullet 😉
c

camdenorrb

07/08/2019, 8:37 PM
he dodged a bullet*
Not me 😛
a

adamd

07/08/2019, 8:39 PM
not really sure what is confusing here 🙂
c

camdenorrb

07/08/2019, 8:40 PM
How?! lol
n

Nir

07/08/2019, 8:40 PM
I'm not sure whether he's referring to the type deduction, or the performance of +=
but both are pretty confusing IMHO
a

adamd

07/08/2019, 8:42 PM
I mean, what are you expecting when adding an object to an array?
i understand that this type deduction can be tricky, but why
+=
is confusing?
s

sxtanna

07/08/2019, 8:44 PM
In any other place its semantics indicate that it is "adding" onto its target
Here it creates an entirely new array, with one size larger, and replaces the null element at the end.
(which actually explains why it must be a var and not val)
a

adamd

07/08/2019, 8:45 PM
oh, I understand. But it's not an operator issue, but really immutability thing
s

sxtanna

07/08/2019, 8:45 PM
(since it has to replace the reference with the new array)
a

adamd

07/08/2019, 8:46 PM
and it's done that way by design, not by accident
s

sxtanna

07/08/2019, 8:46 PM
Ofc it is
That doesnt mean its a good idea xD
a

adamd

07/08/2019, 8:47 PM
and you cannot compile reassigning a val
so still - I get that you might not like this type of programming, but i don't get why this is confusing - it never confused me before
s

sxtanna

07/08/2019, 8:49 PM
It was never confusing to me
I just dont like it
a

adamd

07/08/2019, 8:49 PM
why?
s

sxtanna

07/08/2019, 8:49 PM
Because of its semantics
It implies its adding to the array like its a collection that supports resizing, when its not
a

adamd

07/08/2019, 8:50 PM
and how it behaves with mutable collections?
s

sxtanna

07/08/2019, 8:51 PM
?
a

adamd

07/08/2019, 8:51 PM
what would happen if
things
would be a mutable collection, instead of immutable?
s

sxtanna

07/08/2019, 8:52 PM
Depends on the backend of the collection implementation
For arraylist, the array would be resized much larger than 1 to 1 on inserts
a

adamd

07/08/2019, 8:52 PM
as I understand, we are talking about kotlin stdlib library 🙂
s

sxtanna

07/08/2019, 8:52 PM
For set, it would depend on the backing map implementation used
n

Nir

07/08/2019, 8:54 PM
@adamd it's confusing because appending a single element costs O(N)
And you don't really see many languages make doing something like this, so easy (to the point of including syntactic sugar to make it easier)
In languages that are "mostly mutable" like e.g. C++, push_back is an amortized operation and very fast.
In languages that are mostly immutable, your List is backed by a try immutable array implementation.
That doesn't do anything as naive as copy out the whole list.
*try -> true
a

adamd

07/08/2019, 8:57 PM
ok, I get your point, it's valid and interesting
n

Nir

07/08/2019, 8:57 PM
In Kotlin, when you can call += on a List, it doesn't actually know if it's even an immutable list or not. It just knows that it's List, or some subtype of List.
a

adamd

07/08/2019, 8:57 PM
I just still don't get how:
Yeah, this shit confused a new programmer to the language
s

sxtanna

07/08/2019, 8:58 PM
A new programmer would see that an assume it did the same thing as appending an element to a collection
a

adamd

07/08/2019, 8:58 PM
I mean, if I am a
new programmer to the language
, the cost of an adding an element to an array is not the first thing I'm expecting/thinking of
c

camdenorrb

07/08/2019, 8:59 PM
It should be something to note when using collections
s

sxtanna

07/08/2019, 8:59 PM
How would you explain the difference between those @adamd?
n

Nir

07/08/2019, 8:59 PM
Well, thinking about the constants, no,
I agree
but, if you do += on single elements in a loop
on a decent size loop
you will literally get way worse performance than e.g. python
a

adamd

07/08/2019, 9:00 PM
yeah, doing this on an immutable array, probably yes
n

Nir

07/08/2019, 9:00 PM
I guess a lot of these things are this way, because of the JVM and Java.
s

sxtanna

07/08/2019, 9:00 PM
well its java
a

adamd

07/08/2019, 9:01 PM
but it's not suprising for me that an immutable array is not optimized for lots of element adding
n

Nir

07/08/2019, 9:01 PM
But personally, I think languages should either just go full immutable, or they should support
const
a la C++, Rust, D
Anything other than one of these two approaches is a pretty big compromise
a

adamd

07/08/2019, 9:02 PM
it is more suprising that it has
operator fun
supporting adding an element
n

Nir

07/08/2019, 9:02 PM
Well, x += y will desugar to x = x + y
c

camdenorrb

07/08/2019, 9:03 PM
Can't they deprecate the plus assign function?
a

adamd

07/08/2019, 9:03 PM
maybe this is the issue you all are talking about - this convenient method that does not make you call
copy
explicitly
n

Nir

07/08/2019, 9:03 PM
You don't have to call copy explicitly.
x = x + y
is already making copies, and it's pretty explicit about it
a

adamd

07/08/2019, 9:03 PM
@camdenorrb because of less performance?
c

camdenorrb

07/08/2019, 9:03 PM
Yeah
a

adamd

07/08/2019, 9:04 PM
@Nir you're right
n

Nir

07/08/2019, 9:04 PM
So, for me, my Kotlin style guide would say: use MutableList as a local variable, and use List in function signatures and as a member of classes
and that would basically be it
s

sxtanna

07/08/2019, 9:05 PM
I dont think the replacement of the copy function with
+
is explicit enough to show what its actually doing
a

adamd

07/08/2019, 9:05 PM
why?
s

sxtanna

07/08/2019, 9:05 PM
Because its not adding to the array
a

adamd

07/08/2019, 9:05 PM
if it copies whole array
s

sxtanna

07/08/2019, 9:06 PM
Its creating a new array, and throwing the entire old one away
n

Nir

07/08/2019, 9:06 PM
If you use MutableList locally you avoid performance pitfalls, and local variables by their nature are already immune to most of the pitfalls of mutability. If you use List in function signatures and as members, you are protected from most of the pitfalls of mutability.
s

sxtanna

07/08/2019, 9:06 PM
For collections that have the
add
function, it makes sense for them to support the
+
operator
Because semantically, thats already there
c

camdenorrb

07/08/2019, 9:06 PM
It could literally be an IDE intention
To like, not do this
a

adamd

07/08/2019, 9:06 PM
may I ask - what are you programming on a daily basis? 🙂
n

Nir

07/08/2019, 9:07 PM
Who is that directed to?
a

adamd

07/08/2019, 9:07 PM
I guess all discussion participants
n

Nir

07/08/2019, 9:07 PM
Mostly C++, some python.
a

adamd

07/08/2019, 9:07 PM
that is a lot of interesting points I've never thought of
c

camdenorrb

07/08/2019, 9:07 PM
MineCraft plugins lol...
s

sxtanna

07/08/2019, 9:07 PM
Kotlin 90% of the time
And a good sprinkle of trashy Java
c

camdenorrb

07/08/2019, 9:08 PM
Almost 100% Kotlin for me
a

adamd

07/08/2019, 9:09 PM
ok, so you are more from KotlinNative - world than Kotlin-JVM, huh? 😉
c

camdenorrb

07/08/2019, 9:09 PM
Nir?
Please ping the people you're referring to lol
a

adamd

07/08/2019, 9:10 PM
Because I'm one of these JVM-spoiled kids, eating RAM for breakfast 🙂
s

sxtanna

07/08/2019, 9:10 PM
ehhh
Im JVM spoiled too, but I still am constantly concerned with abusive stuff like that
a

adamd

07/08/2019, 9:11 PM
So in my case we do not deal with this kind of memory issues, because we are more concerned not to be killed with our code readers
I get this, maybe this convenient method is, in some ways, confusing, but maybe some things are just left for programmers to think?
I mean, if you are doing a lot of heavy adding on immutable collection because there is a convenience method for this, I don't think that it is a language issue 😉
c

camdenorrb

07/08/2019, 9:13 PM
Still should be an IDE intention
a

adamd

07/08/2019, 9:14 PM
what? warning about copying whole immutable collection?
s

sxtanna

07/08/2019, 9:15 PM
language issue?
Its an extension funtion
...
a

adamd

07/08/2019, 9:16 PM
not really convenient for users not doing this in the loop
@sxtanna my point exactly
s

sxtanna

07/08/2019, 9:16 PM
Who said its a language issue?
a

adamd

07/08/2019, 9:19 PM
I used it as a synonime for
this shit
- maybe it was my misunderstanding, but I tried to understand where it came from 🙂
s

sxtanna

07/08/2019, 9:19 PM
Who said that?
a

adamd

07/08/2019, 9:20 PM
cam, I guess
probably ~100 messages ago 😉
c

camdenorrb

07/08/2019, 9:21 PM
I didn't say it was a language issue
I said it should be an intention in Intellij
a

adamd

07/08/2019, 9:24 PM
like I said, misunderstanding
n

Nir

07/08/2019, 9:24 PM
I'm just investigating Kotlin, not using it really in prod
I'd probably prefer native but don't really care
it's not fast anyway so it's irrelevant to me
(fast by C++ standards)
a

adamd

07/08/2019, 9:25 PM
nevertheless, interesting points was made here 🙂 I will take it into account on my next array-issues
n

Nir

07/08/2019, 9:25 PM
There are lots of things that suck in C++ but const is not one of them.
a

adamd

07/08/2019, 9:25 PM
@Nir that's for sure, but I don't think it was designed to be faster than C++
n

Nir

07/08/2019, 9:25 PM
I think for languages that want to have mutable data structures, and let's face it they are more intuitive for most people
then, those languages (if statically typed) should have
const
At the end of the day, 95% of what I want is just to guarantee that the functions don't modify their arguments
a

adamd

07/08/2019, 9:26 PM
maybe, I'm not big fan of mutability
n

Nir

07/08/2019, 9:26 PM
Well, Kotlin is all about mutability, so... 🙂
List
is not an immutable data structure, it's just a read-only view of some data structure, that may or may not be immutable
a

adamd

07/08/2019, 9:27 PM
well, have you met Java? 😉
n

Nir

07/08/2019, 9:27 PM
Meh fundamentally it's not any different
MutableList
inheriting from
List
, that wasn't invented in Kotlin
val x: MutableList<Int> // populate
foo(x) { x += 5 }
a

adamd

07/08/2019, 9:28 PM
I'm not stating it solves all my problems, but I believe that points me into a good direction
n

Nir

07/08/2019, 9:28 PM
If you look at a situation like this, you can see that if
foo
takes e.g.
List
and a lambda
it clearly can't assume even in the local function scope, that its argument isn't changing
a

adamd

07/08/2019, 9:29 PM
ofcourse, but we are talking about functional programming principles, not language itself
n

Nir

07/08/2019, 9:30 PM
Yeah, I agree, it makes decent compromises. The thing is when you say "I'm not a fan of mutability", if you consider my suggestion with making locals mutable and function interfaces/class members not-mutable
a

adamd

07/08/2019, 9:30 PM
don't blame him for not being not-functional
n

Nir

07/08/2019, 9:30 PM
you will see that you avoid 95% of real world issues with mutability
If not more.
Fully immutable can be good too though, but that works much better if you have proper immutable implementations, which Kotlin doesn't have.
So, in Kotlin I think you have to be pragmatic and use both List and MutableList, hence my advice
If you use List/Map/Set everywhere, especially on a team of varying experience levels, you will likely be running into performance explosions on a semi-regular basis
a

adamd

07/08/2019, 9:32 PM
not in my part of IT-world
like I said before, I'm JVM-spoiled-kid
we have a lot of memory to eat
I'm not proud of that, but that is a fact
if we have performance issues, it is mostly because of some misuse of a framework or library, or a memory leak
or, if somebody fetches really large amounts of data and makes calculations on them
n

Nir

07/08/2019, 9:35 PM
Making something quadratic that should be linear can bring things down very very very fast
No matter the world
a

adamd

07/08/2019, 9:35 PM
still - not my case, but I like to see other points of view - it provides me with constant improvement of my skills
n

Nir

07/08/2019, 9:35 PM
SO quite literally went down for nearly an hour
because they used some regex whose implementation happened to be quadratic in the number of consecutive whitespace characters
someone had a weird post with like 2K consecutive whitespace characters, boom
that's all it took
The problem is actually not really peak memory usage because the JVM is pretty smart about seeing that an object is short lived and recycling it.
It's just literally all the performance cost of copy those elements over and over.
a

adamd

07/08/2019, 9:37 PM
like I said - this is the reason why I took part in this discussion in the first place
this is not something I was thinking of when coding
but it is also important
and very often ignored
n

Nir

07/08/2019, 9:38 PM
Right. So, for me, coming from C++ which is an edge case filled language, we are used to coding guidelines. if you can come up with a good coding guideline that reduces complexity and gives you 99% of the benefit of making an informed decision in each specific case, it's a great thing. especially in a team environment.
If you use MutableList locally you avoid these problems for free, and there's almost no chance of running into other problems.
As long as you keep using List elsewhere.
c

camdenorrb

07/08/2019, 9:42 PM
Jit can make Kotlin and Java faster than C++ at times
Also these are mainly structure problems you're referring to, with the proper setup you won't run into these issues
g

ghedeon

07/08/2019, 9:45 PM
talking about
plus/+
and confusion: https://youtrack.jetbrains.com/issue/KT-9992
how about
plus
and
plusElement
🙂
n

Nir

07/08/2019, 9:47 PM
@camdenorrb this is one of those things that's technically true, yes.
But when you are talking about people using C++ for serious performance requirements it's basically never going to be true.
Before C++11 there were places using Java in my industry, but you really had to fight a lot to get the performance the way you wanted, turn off GC, etc
at that point it's not really idiomatic java and you lose a lot of the benefit. That sort of thing has largely vanished as C++ has improved considerably
c

camdenorrb

07/08/2019, 9:49 PM
What are you making exactly?
n

Nir

07/08/2019, 9:51 PM
HFT
c

camdenorrb

07/08/2019, 9:52 PM
Ah hmmmmm, idk I don't feel like nanosecond differences would be that big of an issue
n

Nir

07/08/2019, 9:56 PM
what do you mean, nanosecond differences?
The differences between well written C++ and well written Java can be a lot more than that
I mean, minecraft was rewritten, right.
Believe me they don't care about nanos there. They barely care about micros.
In C++ you can define a struct and you know that
vector<Foo>
is completely contiguous in memory.
That's a really big deal and not currently possible in Java.
(or kotlin)
c

camdenorrb

07/08/2019, 10:02 PM
Please don't compare anything about Java to Minecraft
Minecraft's code is actual spaghetti
It just really depends on how you write the code for the performance you'll get out of it, I personally haven't experienced any performance issues over the past 5 years besides when I wanted to make a screensharing program and realize the JVM impl for screenshots was hella slow
But I can just use LWJGL for that
n

Nir

07/08/2019, 10:10 PM
My understanding is that in minecraft it because a really difficult issue that some basic type, I think
Vec3
, which was literally 3 doubles, was pointer-indirected (like all Java types other than int, etc)
But anyhow if you are writing anything that only has to be fast enough for humans, you are operating on wildly different timescales.
When a microsecond is a ton of time you can imagine how much of a control freak you need to be.
We're regularly looking at how various C++ abstractions get compiled to assembly to convince ourselves whether the compilers can consistently optimize them out (and therefore whether we can use them freely or not)
etc
t

Toddobryan

07/08/2019, 11:01 PM
Note that for a generic
List
, Kotlin uses
ArrayList
, which is amortized O(1) to add elements, so it's doing the same thing under the hood, but isn't nearly so awful. tl;dr Use
List
instead of
Array
if the size will change.
n

Nir

07/08/2019, 11:11 PM
@Toddobryan that would be if you use
MutableList
When you have a
List
you can't perform any operations that modify the object (unless you downcast first)
x += 1
for a
List
is just syntactic sugar for
x = x.plus(1)
(assuming
x
is a
var
)
t

Toddobryan

07/09/2019, 3:15 PM
@Nir Good point. I guess what I meant to say is that it's more efficient to append a single element at the end of a mutable
List
. This is different than Scala, which uses
Cons
lists, so it's O(1) to add at the beginning, but O(n) to add at the end. Because of the way
ArrayList
works, the reverse is true in Kotlin.