I just discovered that `Iterable<T>.map` returns a `List<R>` not the original data struc...

# announcements

[David Smith]

I just discovered that Iterable<T>.map returns a List<R> not the original data structure (whatever implemented the Iterable. This seems very strange to me, is it expected? I would expect the map function to affect only the contents of the data structure, not the data structure itself.

[Nir]

Well, an iterable doesn't have to actually come from a data structure

[Nir]

so I don't see how that could work, even in principle

[Vampire]

Iterable<T>.map could return Iterable<R>

[Vampire]

But actually it does that as List<R> implements Iterable<R> 🙂

[David Smith]

I’m not sure how to write the code, not there in Kotlin yet, but Iteralble is an interface, can’t I return whatever type implemented the interface?

[Nir]

not necessarily, no

[Nir]

an iterable can be lazy for example

[David Smith]

why does that matter?

[Nir]

well, at that point map would be behaving in a very different way

[Nir]

to discuss that we'd probably need to discuss why Kotlin has both Iterable<T> and Sequence<T>

[Tomasz Krakowiak]

someMap.map{ it.toString()} ... hmm?

[Nir]

FYI, if you're coming from most other languages, what's surprising about map in Kotlin isn't that it doesn't return the same data structure

[Nir]

it's that map isn't just returning a lazy iterator, that you later decide to collect in a data structure

[Nir]

this is probably the most common approach

[Vampire]

Also imagine a MyDatabaseResult that lazily gives you the results from the database when you query them.
How should the result of map create an instance of that class to satisfy your requirement?

[David Smith]

I don’t understand, isn’t Iterable just something that can (possibly) return a next value? Surely I could build a map function on that that works lazily or strictly, depending on the iterator?

[Nir]

so, inKotlin, functions taking lambdas (like map) can be inline, or they can be lazy

[Nir]

they cannot be both

[dmitriy.novozhilov]

Iterable interface have only one method: fun iterator(): Iterator<T>. There is no information how to construct original type on which map was called.

[Nir]

and inline is desirable in this context. So, Kotlin gives you both options. Iterable<T>.map is inline, and isn't lazy, it always processes the iterable immediately. So it has ot put it somewhere. So it needs a data structure.

[Nir]

Sequence<T>.map is not inline, but it is lazy

[dmitriy.novozhilov]

class MySuperIterable : Iterable<String> {
   ...
}

fun test(it: MySuperIterable) {
    val result = it.map { it.length }
}

In this example there is no way to create MySuperIterable which holds Int

[David Smith]

ahhh, @dmitriy.novozhilov I now see the issue with the way it works

[David Smith]

so Iterable is the wrong thing, map would need it’s own interface

[Vampire]

o_O

[Nir]

not really, no

[David Smith]

well that’s a shame, it was quite unexpected to me that map would change the data structure

[Nir]

do you have a specific example?

[Nir]

I guess you wanted to call map on a Map or something?

[David Smith]

yes, I have a Page<T> type which wraps up a list with some other info

[Nir]

okay, and Page<T> implements Iterable<T> I suppose?

[Nir]

Or even implements List<T>?

[David Smith]

well I just added Collection in order to do this, I thought it would work without investigating enough

[Nir]

So, what you'd typically do is to do the map via sequences instead of iterables

[Nir]

myPage.asSequence().map { ... }.toPage();

[Vampire]

Again, how should map know how to instantiate a Page<R>?
And more so if the type bounds for T maybe don't even allow R.

[David Smith]

the problem is that there isn’t a toPage function because there is extra data in the Page object (page number, page size etc)

[Nir]

if you find that verbose, then you could write your own mapPage or something like that. You could also choose to have Page<T> implement Sequence<T> rather than Iterable<T>

[David Smith]

so I’m looking for a way to map over the values inside without changing anything else

[Nir]

Right, that's a good point. but yeah, it would also be unreasonable to think that a generic map could magically know how to do that

[David Smith]

I wrote mapPage but that was a bit of a hack, I thought there would be some interface that I could implement

[Nir]

Probably the best solution is to write a function called transform, which takes a lambda which accepts a List<T> and returns a List<T>

[Nir]

or accepts a sequence<T> and returns a sequence<T>

[David Smith]

yeah, I’m coming from a different world where you have a map interface

[David Smith]

i.e. functor

[Nir]

that would allow you to use the full standard library collection of functions on this

[Nir]

Yeah, I guess you're coming from Scala or Haskell or something

[David Smith]

Haskell/Purescript

[Nir]

Nothing is stopping you from implementing map yourself, just to be clear

[David Smith]

yeah that’s what I’m doing

[Nir]

from my limited knowledge of haskell I'm not sure this concepts really carries over 1-1

[Nir]

The thing is though if you implement map itself, then you'll have the same issue again with say filter

[Nir]

that's why I suggested transform

[David Smith]

well, mapping over the contents of a container

[David Smith]

or mapping a function over the contents of a container

[David Smith]

anyway, no big deal

[Nir]

Yes, I understand, but like I said, you'll need to repeat this for every useful utility function in the stdlib

[Nir]

that you want to use

[Nir]

But, perhaps you only really see yourself using map and prefer the simplicity/convenience of targeting this one use case

[Nir]

it's just fairly common to also use things like filter, mapNotNull, etc

[Nir]

mapIndexed, etc

[David Smith]

yeah, that’s the real shame, with a functor you can implement a lot of other functions

[ephemient]

so this is something that Scala can do with its collection builders, and you can sort of manually do in Kotlin

[Nir]

yeah, but that's easily avoided, if you implement something like transform instead

[ephemient]

e.g. .mapTo(builder) { transform(it) }

[Nir]

Page<T>.transform(f: (List<T>) -> List<T>): Page<T>

[Nir]

that's trivial to implement just once, and now you'll be able to do whatever you want inside the transform

[Nir]

newPage = oldPage.transform { oldList ->
    oldList.map { .... }.filter { ... } etc etc 
}

[David Smith]

in the end I just created

fun <A, B> Page<A>.map(f: (A) -> B): Page<B> {
    val bs = this.values.map(f)
    return Page.slice(bs, this.totalElements, Pageable(this.pageNumber, this.pageSize))
}

[David Smith]

filter is a bit of a weird function compared to map because by it’s nature it changes the data structure

[David Smith]

@Nir this transform function is a bit dodgy because we don’t really want to change the size of the list underneath without changing this.pageSize etc

[David Smith]

thanks for all your help anyway!

[Nir]

Well, transform could of course handle that, since you could ask for the size of the List. better, this.pageSize should probably just be implemented as a property that returns the size of the List anyway

[Nir]

since it saves storage, and better guarantees correctness

[Nir]

np

[Ruckus]

Not sure if it's been mentioned, but if you're coming from a functional world, look into #C5UPMM0A0.

[David Smith]

yeah, I took a look, I’m just trying not to push too much of that type of thing into my workplace (yet…). I was just hoping that some stuff would exist more natively (i.e. without a compiler plugin) and so far I have found lots of great things, map was not one of them though :)

[David Smith]

I sort of took type classes for granted until I started using kotlin and found that it’s very difficult to implement a lot of nice things without them because you need to attach generic stuff to existing classes

[Nir]

Yeah, it's problematic that you cannot implement a trait for a class you don't control

[David Smith]

exactly

[Nir]

it's probably my single biggest issue with kotlin tbh, it's terribly constraining

[Nir]

I come from C++ which is basically the wild west, okay, the generics are totally unconstrained, but at least that means I can implement generic things for anything I want, and have things work.

[David Smith]

lol, an honest way to describe C++ 🙂

[Nir]

I tried to write a class to hold a json data structure in Kotlin, and it was just impossible to make it nice

[Nir]

in C++, yes it's a bit of extra work to have a recursive data structure, but you can do it very nicely

[Nir]

indeed 🙂 we know what we are.

[Tomasz Krakowiak]

@Nir Take a look at kotlinx-serialization-json https://github.com/Kotlin/kotlinx.serialization/blob/master/formats/json/commonMain/src/kotlinx/serialization/json/JsonElement.kt

[Nir]

not really looking for a json library, i was just commenting that Kotlin's type system puts fundamental limitaitons on some of the things you can do

[Nir]

writing an in-memory json representation is a way to see it