https://kotlinlang.org logo
Title
a

addamsson

07/14/2019, 11:07 AM
@raulraja I'm working on a Kotlin MPP library for which I need some FP utilities which is provided by Arrow Core. My problem is that it is not MPP yet and now I'm trying to determine which is a more time-efficient solution: roll my own FP lib including the essentials or help Arrow become MPP. I've heard from @pakoito that you're working on it. Can you give me a rundown on what is needed to get there?
r

raulraja

07/14/2019, 11:44 AM
Hey Adam, for Arrow to become MPP we are developing typeclasses of KEEP-87 as a compiler plugin, it's still in early stages and prototype as we are discovering what the limitations of compiler plugins are and how we overcome it. At the rate we are going I expect to have an implementation mostly ready by September or October. I'm currently investigating and trying to implement quasiquote style tree transformations so we can do metaprogramming similar to Scala meta letting the Kotlin compiler do the codegen based on KElements
As for MPP in general it's in my opinion not ready. The IR implementation is not 100% mpp compatible and mpp is still lacking several areas in terms of dispatchers, etc.
If I were you I would depend and develop the library in terms of the JVM and arrow core for now because it will be mpp automatically by the end of the year. It is our goal Arrow is full MPP before 1.0
We have many more ideas for meta that would work in mpp https://github.com/47deg/arrow-meta-prototype/issues
I'm not sure if vavr uses inheritance in their collections but definetly the features of KEEP-87 can be used in a collections design to separate data and behaviors polymorphically for all Foldable and Traverse
s

simon.vergauwen

07/14/2019, 12:12 PM
for Arrow to become MPP we are developing typeclasses of KEEP-87 as a compiler plugin
Is this blocking core to be MPP?
Monad#fx
and
@higherkind
seem the blockers to me.
@higherkind
could temporarly be replaced by its boilerplate. If we can find a (temp) solution to seperate
Monad#fx
that would open the door to make core MPP already.
a

addamsson

07/14/2019, 12:14 PM
I see
Im not in a hurry
Mypoint is that writing common code is not much more difficult than writing jvm code
But it pays off in the future
Sorry im from phone
s

simon.vergauwen

07/14/2019, 12:16 PM
In that case, it’ll be easier to start from JVM like @raulraja suggested. Currently there is a lot of effort to make Arrow MPP, migrating to that should be trivial.
a

addamsson

07/14/2019, 12:16 PM
One moment brb
Do you have the core arrow interfaces in an mpp library?
Imean module
So i can depend on it, do my implementation of persistent data structures
Then backport it to arrow when it becomes mpp?
s

simon.vergauwen

07/14/2019, 12:37 PM
Work has been done to re-organise those classes but I don’t think it’s been made into a MPP project
a

addamsson

07/14/2019, 12:37 PM
Can i help?
I dont want to roll my own lib if it is possible to do this in arrow
s

simon.vergauwen

07/14/2019, 12:38 PM
I just checked and the core module still contains something that needs to be replaced by compiler plugins.
a

addamsson

07/14/2019, 12:38 PM
Ok
s

simon.vergauwen

07/14/2019, 12:39 PM
In case you’d like to develop MPP you could always clone the core module and rip out everything related to https://github.com/arrow-kt/arrow/blob/master/modules/core/arrow-core-data/src/main/kotlin/arrow/typeclasses/ContinuationUtils.kt
a

addamsson

07/14/2019, 12:39 PM
Then ill do my port (vavr)
Oh
s

simon.vergauwen

07/14/2019, 12:39 PM
Then afterwards, you can just toss the clone and depend on arrow-core directly
a

addamsson

07/14/2019, 12:39 PM
Good idea
Ill check it once im back at mycomputer
s

simon.vergauwen

07/14/2019, 12:40 PM
👍 if you need any help I’d love to help out. Exciting to see this happen
a

addamsson

07/14/2019, 12:40 PM
So you use vanilla kotlin data structures noww?
Cant type sorry
Vavr's option/try have different semantics
s

simon.vergauwen

07/14/2019, 12:41 PM
Yes, almost everything is vanilla Kotlin. We have to rely on some reflection to emulate
for
comphresions/do-notation, for perf and MPP reasons we’ll replace this with compiler plugins.
a

addamsson

07/14/2019, 12:41 PM
So ill use theirs for the port
Then i refactor to arrow
s

simon.vergauwen

07/14/2019, 12:41 PM
I know Vavr has a couple of side-effecting APIs
a

addamsson

07/14/2019, 12:42 PM
I dont mind purging them
s

simon.vergauwen

07/14/2019, 12:42 PM
👍
a

addamsson

07/14/2019, 2:07 PM
This is how vavr looks like
i'm not familiar with Arrow's hierarchy, but I'm almost sure it is not compatible
that seems like a design smell to me is not only
Value
(which they realized they need to remove)
but that all ADTs are
Iterable
this might make sense for
Option
which is an empty
Iterable
if it is
None
and an
Iterable
with 1 element if it is
Some
but how do you iterate an
Either
?
or some custom time where iteration is not sensible
s

simon.vergauwen

07/14/2019, 2:13 PM
Iterable
is the wrong abstraction here.
fold
is what you want to use instead, it's abstracted over by
Foldable
. Which is the polymorphic way of providing this API.
a

addamsson

07/14/2019, 2:16 PM
do you have a set of interfaces which you use for collections / maps?
so I can yank it out
s

simon.vergauwen

07/14/2019, 2:32 PM
Not specifically for collections or maps since those abstractions can be implemented by most data class. `Functor`(map),
Monad
(flatMap), `Foldable`(fold),
Traverse
(traverse),
FunctorFilter
(filter) etc
But those closes are typically used to abstract over the container. I'm not sure this is what you are looking for.
Could you share some signatures? It's probably easier to talk in terms of those.
a

addamsson

07/14/2019, 2:40 PM
oh I got it
so how can we reconcile Kotlin's
Iterable
with these?
what I'd like to be able to do is to have an API which is a no brainer to use for Kotlin devs
for example
myMap.toPersistentMap().map { (key, value) -> ...}.fold { ... }
I wrapped Clojure's collections in a POC which worked really good but there is a problem: the Kotlin stdlib defines a lot of extension functions on
Iterable
so I need to supply specializations for all of them which work with `PersistentCollection`s
If I understand correctly Arrow's `Functor`/`Monad` etc completely decouple the whole thing from the Kotlin stdlib, right?
what's the relationship between this and
?
s

simon.vergauwen

07/14/2019, 2:56 PM
I think both were created for interested people but nothing ended up happening. It should be cleaned up.
Yes, the typeclass hierarchy is meant to decouple the actual types from the behavior. So it also effectively also decoupled everything from the std lib.
To be honest I'd avoid implementing
Iterable
on your persistent collections. This way you can avoid getting tangled into the extensions functions of the std.
With keep-87 or the compiler plugin all these functions implemented by `Functor`/`Monad ` will be automatically available as instance methods.
Having
object PersistedMapFunctor
available is enough for the compiler to know
map
exists for
PersistentMap
.
a

addamsson

07/14/2019, 3:18 PM
This way you can avoid getting tangled into the extensions functions of the std.
This is what I implied, yes!
I've forked
arrow
running
./gradlew clean build
resulted in a wall of text of compilation errors. Am I missing something? A switch maybe?
this is an example:
D:\dev\forks\arrow\modules\validation\arrow-validation\build\generated\source\kaptKotlin\main\extension\arrow\validation\refinedTypes\numeric\either\nonZero\EitherNonZero.kt: (18, 47): Type argument is not within its bounds: should be subtype of 'Number'
r

raulraja

07/14/2019, 9:29 PM
what JDK are you running this with? I’ve never seen that come up
a

addamsson

07/15/2019, 1:07 PM
I'm not at my other computer right now but I think it is the latest JDK8
(Oracle)
i'll check when I'm back at home
i'm trying it on my mac now
It is still running (15m) so I think this is a windows problem
on my mac I have OpenJDK
Is Oracle's JDK unsupported?
Confirmed, it works on my mac
r

raulraja

07/15/2019, 3:16 PM
no idea what the issue is on your windows but AFAIK others have built it on Windows before
a

addamsson

07/15/2019, 4:12 PM
don't worry I'll figure this out
btw is it normal that a
./gradlew clean build
takes 32m?
p

pakoito

07/15/2019, 4:15 PM
yes
use assemble for compilation
and test for festing
build does both, plus builds the docs
a

addamsson

07/15/2019, 4:16 PM
got it 👍
I just wanted a full build to verify that it works
@raulraja I'm going to create an implementation for
PersistentMap
using Arrow with the plain old JVM target (no MPP) just to see how it works, although I have one question:
There is no typeclass for removal AFAIK. We should agree how it should work
r

raulraja

07/15/2019, 5:51 PM
What is the type signature of
remove
?
a

addamsson

07/15/2019, 5:51 PM
I think the simplest solution is to add a
fun remove(element: E): PersistentMap<E>
what do you think?
r

raulraja

07/15/2019, 5:51 PM
that does not mutate the map right?
a

addamsson

07/15/2019, 5:55 PM
it does not
it returns a new version which doesn't have
element
now that I think of it, we can create a typeclass for this
something similar to
FunctorFilter
r

raulraja

07/15/2019, 5:58 PM
well this probably already exists for either all Foldables or all Traverse
a

addamsson

07/15/2019, 5:58 PM
checking
r

raulraja

07/15/2019, 5:58 PM
so we can just add the
remove
operation to wherever it fits
I mean is not there today but it’s an operation that can be probably implemented in Foldable
a

addamsson

07/15/2019, 6:00 PM
where do we have
filter
?
remove
is a specialized version of
filter
it filters a single element using an equality check
it is like
list.filter { it != element}
it is just very ineffective:
O(n)
vs
O(1)
r

raulraja

07/15/2019, 6:02 PM
well data types can specialise it to make it fast
but we can provide a default implementation
a

addamsson

07/15/2019, 6:02 PM
we need something which defines
eq
for
E
and which has
filter
makes sense
so it is a combination of
FunctorFilter
and
Eq
so we can add
remove
to
FunctorFilter
and have the default implementation which is
O(n)
and the specialized version which implements the
O(1)
version for
PersistentMap
how's that sound?
r

raulraja

07/15/2019, 6:04 PM
fun <F, A> Kind<F, A>.remove(a: A): Kind<F, A> =
  if (this.contains(a)) this.filter { it == a }
  else this
a

addamsson

07/15/2019, 6:04 PM
although I'm not 100% sure how
Eq
comes into the picture
r

raulraja

07/15/2019, 6:04 PM
is that conceptually the impl ^^^ ?
a

addamsson

07/15/2019, 6:05 PM
yes!
so if I understand the concept properly
you define this on
Kind<F, A>
and provide an implementation for
Kind<ForPeristentMap, A>
?
s

simon.vergauwen

07/15/2019, 6:10 PM
Yes, so a default impl
O(n)
impl would exist in
FunctorFilter
and a fast impl would override the default in
FunctorFilter<PersistentMapOf<Keys>>
and
FunctorFilter<ForPersistentSet>
☝️🏼 1
r

raulraja

07/15/2019, 6:11 PM
right
a

addamsson

07/15/2019, 6:11 PM
great!
i'll be back
i'm going to read the docs so I can quit asking stupid questions
i also need to port the unit tests
and the rest of the data structures
thanks for the input this was really insightful! 👍
👍 2
r

raulraja

07/15/2019, 6:12 PM
fun <A> Kind<F, A>.remove(a: A): Kind<F, A> =
    mapFilter {
      if (a == it) Some(it)
      else None
    }
conceptually if this ends up inside FunctorFilter which is already in core master
the
==
is where the equality plays a role
Also if you plan to add this to FunctorFilter or something similar we should consider others like
removeIf
etc…
that resemble the apis in the standard library but without mutation
s

simon.vergauwen

07/15/2019, 6:15 PM
Yes, I've missed this set of functions a couple of times already and I always ended with slow
fold
code instead
r

raulraja

07/15/2019, 6:16 PM
Also this may be also implemented in Foldable which in my opinion it would be more versatile
In Foldable we already have things like
contains
etc.
so you may be able to fold and accumulate into the new collection ignoring items that match the predicate
p

pakoito

07/15/2019, 6:17 PM
if (a == it) Some(it)
==> one thing to mention, all maps in Haskell use Eq and Ord for keys
🔝 1
maybe if we're building our own libs we could leverage them too
a

addamsson

07/15/2019, 6:22 PM
Foldable
seems like a reasonable choice if we care about utility
but conceptually it would be best suited in
FunctorFilter
1
I also agree with @pakoito that
Eq
should be used instead of
==
1
since we already have a lot of utitlity in
Eq
where should I put the data types?
arrow-core-data
?
are persistent data structures a
core
concept?
s

simon.vergauwen

07/15/2019, 6:28 PM
Core concept absolutely. Does it belong in
core
not perse, if the module is small enough I'd say yes.
We've moved away from an as small as possible core module to one that makes more sense to users.
r

raulraja

07/15/2019, 6:34 PM
remove
would go in FunctorFilter or Foldable in core and the actual collection implementations datatypes and extensions a
arrow-persistent-collections
or similar named module
a

addamsson

07/15/2019, 7:55 PM
which one would you guys prefer
FunctorFilter
or
Foldable
?
I'm not that familiar with the code yet to be able to make this decision on my own
so what I'm thinking is this: 1. I'll finish my port of persistent data structures to Kotlin in my own project 2. I'll properly read through the docs of Arrow and tinker with it so I can fully grasp how these concepts work 3. I'll integrate
PersistentMap
on a branch in my Arrow fork and open a PR you can review
how's that sound?
I think it would make sense to start small and go from there (by not implementing
PersistentSet
+
PersistentList
together with
PersistentMap
in one go)
p

pakoito

07/15/2019, 8:08 PM
if it’s implementable with Foldable I’d use it, it’d be funny to use delete on Option hahaha
1
1. I’ll finish my port of persistent data structures to Kotlin in my own project
what’s missing/necessary?
a

addamsson

07/15/2019, 8:10 PM
i want to port the unit tests to see that my port is up to snuff
then I have to refactor the type hierarchy to align it with Arrow
the original implementation is full of `null`s so I have to double-check everything
good thing is that the code is in a
common
project so retrofitting to MPP will be a no-op 😄
p

pakoito

07/15/2019, 8:18 PM
null is okay in internal implementation for speed
just make sure that collections can store nullables
then I have to refactor the type hierarchy to align it with Arrow
I’m curious about this, if the code is shared through inheritance
a

addamsson

07/15/2019, 8:20 PM
i've ported VAVR code as-is
next step is to make it Kotlin idiomatic
then make it Arrow-idiomatic 😄
p

pakoito

07/15/2019, 8:38 PM
fair 😄
let us know where we can help
a

addamsson

07/15/2019, 8:39 PM
will do! 👍
👍 1
I think the docs is fairly understandable
Do you think that
null
keys should be allowed?
VAVR is allowing it 😮
p

pakoito

07/15/2019, 9:08 PM
keys will be restricted to Eq and Ord
which are the typeclasses that define equality and order
so if someone has a typeclass that does string nullability comparison I don’t see why not
it’s just an additional value
one trick that Clojure did is that keys and values were treated as Object/Any? and only safe at the edges
a

addamsson

07/15/2019, 9:10 PM
hm
r

raulraja

07/15/2019, 11:12 PM
null in Kotlin is isomorphic to None. It's a Singleton empty value and it's a valid value in the monoid of nullable types as the identity element
Kotlin already prevents you from referencing direct null types and allows you to pass null as a value so imo it should be a valid key
If the user wants a map that does not accept nullable types it could constrain the key with : Any which will not allow you pass null as a key but that in Kotlin is a specialization since Any? is more generic
a

addamsson

07/16/2019, 7:10 AM
That't true
i'm also using
<K, V>
so
null
is theoretically accepted
This is the
interface
for HAMT right now:
interface HashArrayMappedTrie<K, V> : Iterable<Tuple2<K, V>> {

    val isEmpty: Boolean
    val size: Int

    operator fun get(key: K): Option<out V>

    fun containsKey(key: K): Boolean

    fun put(key: K, value: V): HashArrayMappedTrie<K, V>

    fun remove(key: K): HashArrayMappedTrie<K, V>

    fun keysIterator(): Iterator<K>

    fun valuesIterator(): Iterator<V>
}
how do you think this should be modified to enable Arrow integration?
apart from using the Arrow
Option
instead of the VAVR one
r

raulraja

07/16/2019, 12:28 PM
You would just provide similar instances as the ones MapK provides for whatever type classes that datatype can conform to.
You’d need to determine if that can implement Foldable, Traverse, Functor etc… based on whether it can guarantee order or not.
Currently MapK can provide instances for Eq, Foldable, Functor, Hash, Monoid, Semigroup, Show, Traverse
a

addamsson

07/16/2019, 12:56 PM
okay
I'll take a look at
MapK
i also fixed nullable keys
👏 1
A bit of an update: I've ported all the tests for the HAMT implementation and they passed out of the box so I'm happy about this
🎉 2
Now I'm going to make it a bit more Kotlin-idiomatic
👏🏼 2
@raulraja @simon.vergauwen @pakoito what do you think about
core/arrow-persistent-collections
as a module for the implementations?
r

raulraja

07/17/2019, 11:09 AM
Should be top level outside core like fx is now in master
a

addamsson

07/17/2019, 11:09 AM
okay
should I branch from
master
?
r

raulraja

07/17/2019, 11:10 AM
Yes, thanks!
Happy to see this initiative reborn
a

addamsson

07/17/2019, 11:12 AM
me too!
so I'll add a top level module
arrow-persistent-collections
in the
modules
folder, I'll branch from
master
and I'll augment
Foldable
. If that's not gonna work I'll use
FunctorFilter
I'll use
MapK
as an example
a

addamsson

07/17/2019, 11:13 AM
what's the
k()
function btw which creates a
ListK
form a
List
?
there are no docs
p

pakoito

07/17/2019, 11:13 AM
there’s also the collections repo if you prefer
k()
is how you wrap stuff
List -> ListK
Map -> MapK
only ListK and MapKa and the others have typeclass instances
a

addamsson

07/17/2019, 11:14 AM
I got it, I just don't understand what the name
k
and the prefix
K
means?
p

pakoito

07/17/2019, 11:14 AM
Kind/Kinded
a

addamsson

07/17/2019, 11:15 AM
oh!
p

pakoito

07/17/2019, 11:15 AM
because they implement Kind
r

raulraja

07/17/2019, 11:17 AM
When we do the newtype compiler plugin there will be no need to k()
Since we can create instances on the fly for foreign types
a

addamsson

07/17/2019, 11:18 AM
got it
r

raulraja

07/17/2019, 11:18 AM
For your own types you don't need k() in what you are doing since you can make then @highkind
a

addamsson

07/17/2019, 11:18 AM
i'm wondering how KEEP-87 will work with MPP
r

raulraja

07/17/2019, 11:18 AM
ListK is for cases where we can't modify the hierarchy
It does already using the IR Backend
Which arrow meta enables by default
a

addamsson

07/17/2019, 11:18 AM
IR?
r

raulraja

07/17/2019, 11:19 AM
It's the bytecode format of the Kotlin compiler Backend
a

addamsson

07/17/2019, 11:19 AM
oh I see
r

raulraja

07/17/2019, 11:19 AM
It's an intermediate format for all backends
a

addamsson

07/17/2019, 11:19 AM
i haven't done any bytecode manipulation since my days in OSGi
r

raulraja

07/17/2019, 11:20 AM
That's all I'm doing these days 😓
a

addamsson

07/17/2019, 11:20 AM
ever since I used LISP it feels like a hack 😄
oh 😓
r

raulraja

07/17/2019, 11:20 AM
More tree manipulation because meta is now functional to the user
It's a function from Tree to List of trees
a

addamsson

07/17/2019, 11:21 AM
got it
s

simon.vergauwen

07/17/2019, 11:23 AM
But Kotlin compiler plugins isn't bytecode manipulation. It's much higher level.
A modern kapt... 🤔
r

raulraja

07/17/2019, 11:25 AM
Yeah though some times you write the IR classes yourself to produce just bytecode without descriptors but still much higher level than working with ASM libs
Meta is already a modern kapt because it can spit any new trees in new actual files so it can do codegen with the KElements which are more complete than the JavaElements and you have access to the bindingtrace which you can use to typecheck expressions
So more complete than kapt for metaprogramming
Even with a simpler compiler plugin and just the analysys phase of a compiler plugin you can do all that kapt does faster and better
a

addamsson

07/17/2019, 11:29 AM
so
a compiler plugin is run before Kotlin is compiled to java bytecode / javascript / etc?
i've never worked on these things 😞
s

simon.vergauwen

07/17/2019, 11:30 AM
A plugin can register extensions in different phases of the compiler.
a

addamsson

07/17/2019, 11:30 AM
what's the case with Arrow?
i'm just curious about the compatibility with MPP
s

simon.vergauwen

07/17/2019, 11:30 AM
So you could also use compiler plugins to write linters and create a IDEA plugin from the compiler plugin.
We plug into multiple faces. Type checking and code gen for higher kinded types. But @raulraja knows all the details
r

raulraja

07/17/2019, 11:32 AM
Some of the things we are working on https://github.com/47deg/arrow-meta-prototype/issues
a

addamsson

07/17/2019, 11:32 AM
thanks
r

raulraja

07/17/2019, 11:32 AM
the README has some context
a

addamsson

07/17/2019, 11:34 AM
oh
so this happens before MPP compilation
good
r

raulraja

07/17/2019, 11:35 AM
yes
a

addamsson

07/17/2019, 11:35 AM
👍
r

raulraja

07/17/2019, 11:36 AM
this is just contributing to the compiler with new trees that get automatically compiled to whatever target
a

addamsson

07/17/2019, 11:36 AM
i wonder how this relates to MPS
if it relates at all
r

raulraja

07/17/2019, 11:36 AM
what is MPS?
a

addamsson

07/17/2019, 11:36 AM
r

raulraja

07/17/2019, 11:38 AM
I think the only relation is that both use the IDEA apis for the PsiElements
a

addamsson

07/17/2019, 11:38 AM
@raulraja i ended up leaving most of the ported code of the HAMT intact
I don't want to introduce subtle bugs with code transformation
r

raulraja

07/17/2019, 11:38 AM
At the PsiElement levels is where the AST is
a

addamsson

07/17/2019, 11:38 AM
I intend this implementation to be a placeholder until I get CHAMP working
all these things would be unnecessary if we were using a homoiconic language like LISP 😄
r

raulraja

07/17/2019, 11:41 AM
true that
a

addamsson

07/17/2019, 11:45 AM
nevertheless
you can expect a PR within 2 weeks
i'm going to be on vacation so I'm not sure about the timeline
r

raulraja

07/17/2019, 11:46 AM
no worries and enjoy your vacation, thanks so much for this contribution. Hope we get to meet in person at LW or KotlinConf
☝️ 2
👍 1
a

addamsson

07/17/2019, 11:47 AM
I hope so
I've submitted a talk to KotlinConf
although it is still being evaluated
i've looked at the implementation of
MapK
and I think that we shouldn't implement Kotlin's
Map
in our
PersistentMap
because the stdlib has a lot of extension functions on
Map
which just don't make sense in our implementation
i've added a module under
modules/persistent/arrow-persistent-data-structures
because it is better aligned with the naming I see there
if that's OK
r

raulraja

07/18/2019, 11:03 AM
sounds good
a

addamsson

07/18/2019, 11:32 AM
Do you think that folding a map makes sense with a
Tuple2
?
@extension
interface PersistentMapFoldable<K, V> : Foldable<Tuple2<K, V>> {

  override fun <A, B> Kind<Tuple2<K, V>, A>.foldLeft(b: B, f: (B, A) -> B): B {
    TODO("not implemented")
  }

  override fun <A, B> Kind<Tuple2<K, V>, A>.foldRight(lb: Eval<B>, f: (A, Eval<B>) -> Eval<B>): Eval<B> {
    TODO("not implemented")
  }
}
i've seen that
MapK
uses the partially applied type and only folds keys
I've just seen how you do unit tests
with(intFold) {

      "Folding a list of ints" {
        forAll(Gen.list(<http://Gen.int|Gen.int>())) { ints ->
          fold(Int.monoid(), ints.k()) == ints.sum()
        }
      }
    }
👌
r

raulraja

07/18/2019, 11:51 AM
The Foldable instance for this map should be consistent with how MapK does it
even if this isn’t a Map
a

addamsson

07/18/2019, 11:51 AM
it doesn't work this way I just found out
this code is generated
@JvmName("orEmpty")
@Suppress(
        "UNCHECKED_CAST",
        "USELESS_CAST",
        "EXTENSION_SHADOWED_BY_MEMBER",
        "UNUSED_PARAMETER"
)
fun <K, V, A> orEmpty(arg0: Applicative<Tuple2<K, V>>, arg1: Monoid<A>): Tuple2<K, V, A> =
        arrow.core.Tuple2
   .foldable<K, V>()
   .orEmpty<A>(arg0, arg1) as arrow.core.Tuple2<K, V, A>
r

raulraja

07/18/2019, 11:51 AM
The type of fold based on tuples is for the isomorphism between Map<K, V> and List<Tuple2<K, V>>
a

addamsson

07/18/2019, 11:52 AM
okay
r

raulraja

07/18/2019, 11:52 AM
that fold already exists for List
The code is generated because Tuple2 is not a valid generic there
a

addamsson

07/18/2019, 11:53 AM
why is that?
r

raulraja

07/18/2019, 11:53 AM
try implementing Foldable as Mapk does
a

addamsson

07/18/2019, 11:53 AM
ok
i'm just trying to understand why
Foldable<Tuple2<K, V>>
is not working
r

raulraja

07/18/2019, 12:00 PM
Because Foldable is a higher kinded interface that expects a one type argument hole for any F
The F representation of Tuple2 is Tuple2PartialOf<A> ForTuple2
depending on how many type args you are applying
Foldable is like Functor, it expects types in one whole in them and you are giving it a concrete applied type but it wants a type constructor
a

addamsson

07/18/2019, 12:05 PM
oh I see
how do I assert for equality
?
i've checked some tests but in all of them the
forAll {}
constuct is used
r

raulraja

07/18/2019, 12:06 PM
what are you trying to compare?
==
or
Eq
usually is the way
a

addamsson

07/18/2019, 12:07 PM
i'm sure i'm doing it the wrong way
but this test passes
"Should be able to use fold as a filter" {

    val result = listOf(1, 2, 3, 4).k().foldLeft(listOf<Int>()) { list, next ->
        if (next == 2) {
            list
        } else list.plus(next)
    }

    with(result) {
        forAll {
            result == listOf(0, 3, 4)
        }
    }

}
result
is
list(1, 3, 4)
i just copied this from
FoldTest
i was trying to determine whether
Foldable
is sufficient to implement
remove
what we've been talking about yesterday
and it works!
i just can't figure out how to do a simple assertEquals
😄
s

simon.vergauwen

07/18/2019, 12:13 PM
You're not doing any now. Weird that
forAll
passed but not sure how it work without arguments.
It's meant for properly based testing.
Here I'd rewrite everything starting from
with(result)
, to
result shouldBe listOf(0, 3, 4)
. Does it fail then?
a

addamsson

07/18/2019, 12:18 PM
i'll try
good
failed
so the only question now is how to implement remove
because
Foldable
is not a
Monoid
we don't have
compose
nor
identity
fun <A> Kind<F, A>.remove(element: A): Kind<F, A> = foldLeft(???) { foldable, next ->
    if(element == next) {
      foldable
    } else foldable ??? next
  }
s

simon.vergauwen

07/18/2019, 12:41 PM
Right, so basically we could only have
fun <A> Kind<F, A>.remove(element: A, MA: Monoid<A>): Kind<F, A>
in
Foldable
.
And the more desired
fun <A> Kind<F, A>.remove(element: A): Kind<F, A>
can only live in
FunctorFilter
.
a

addamsson

07/18/2019, 12:42 PM
that's what I thought
but i'm not sure what goes here
fun <A> Kind<F, A>.remove(element: A, monoid: Monoid<A>): Kind<F, A> = foldLeft(monoid.empty()) { foldable, next ->
    if(element == next) {
      foldable
    } else monoid.???
  }
I'm a bit lost here 😞
s

simon.vergauwen

07/18/2019, 12:46 PM
I’m still wondering if it’s possible 😄
a

addamsson

07/18/2019, 12:46 PM
well
you need two monoids
s

simon.vergauwen

07/18/2019, 12:47 PM
Ye that’s what I was going to say
a

addamsson

07/18/2019, 12:47 PM
one for Kind<F, A> and one for Kind<A>
s

simon.vergauwen

07/18/2019, 12:47 PM
You need
MonoidK
for
F
.
a

addamsson

07/18/2019, 12:47 PM
yes
s

simon.vergauwen

07/18/2019, 12:48 PM
Not sure how much sense it still makes at that point. I mean it’s definitely possible but to what extend would it still be user friendly instead of using
FunctorFilter
.
a

addamsson

07/18/2019, 12:49 PM
i'd say user friendliness factor is around
0.2%
😂 1
s

simon.vergauwen

07/18/2019, 12:52 PM
Grouping
remove
with
filter
makes more sense to me than grouping it with
fold
.
a

addamsson

07/18/2019, 12:52 PM
yes
i'm also not sure how I can implement it with fold as
combine
is an extension function
and there is no function to create a monoid and
add
the next element to it
or is there?
fun <A> Kind<F, A>.remove(element: A, MF: MonoidK<F>, MA: Monoid<A>): Kind<F, A> = MF.run {
    foldLeft(empty()) { acc, a ->
      if (a == element) {
        acc
      } else ???
    }
  }
you'd need some kind of monoid combinator
Monoid<MonoidK<F>>
?
😄
s

simon.vergauwen

07/18/2019, 12:58 PM
fun <A> Kind<F, A>.remove(element: A,
                          monoid: Monoid<A>,
                          monoidK: MonoidK<F>,
                          AF: Applicative<F>): Kind<F, A> = AF.run {
  foldLeft(AF.just(monoid.empty())) { foldable: Kind<F, A>, next: A ->
    if (next == element) foldable
    else monoidK.run { foldable.combineK(AF.just(next)) }
  }
}
a

addamsson

07/18/2019, 12:59 PM
I still don't understand the use of
Applicative
😞
s

simon.vergauwen

07/18/2019, 12:59 PM
MonoidK
is what you’re looking for, it’s meant to combine `F`s,
* -> *
or constructors with a typehole.
Applicative is needed here because
monoid.empty()
returns
A
but the return type of our
foldLeft
needs to be
Kind<F, A>
so we need to lift it.
a

addamsson

07/18/2019, 12:59 PM
yep I use it in my example above
I was just not aware of the functionality
Applicative
provides
s

simon.vergauwen

07/18/2019, 1:00 PM
Ah, it offers
just
which can
(A) -> Kind<F, A>
a

addamsson

07/18/2019, 1:01 PM
can you explain in a few words how
Applicative
works? I'm kinda lost here
s

simon.vergauwen

07/18/2019, 1:01 PM
So we can lift
A
into
F
, and then we can use
MonoidK
to combine
F
. You can overload this function in
Traverse
and default supply the
Applicative
instance.
Let me add some intermediate steps with type info
a

addamsson

07/18/2019, 1:01 PM
👍
s

simon.vergauwen

07/18/2019, 1:02 PM
fun <A> Kind<F, A>.remove(element: A,
                          monoid: Monoid<A>,
                          monoidK: MonoidK<F>,
                          AF: Applicative<F>): Kind<F, A> {
  val emptyValue: A = monoid.empty()
  val liftedEmpty: Kind<F, A> = AF.just(emptyValue) 
  
  return foldLeft(liftedEmpty) { foldable: Kind<F, A>, next: A ->
    if (next == element) foldable
    else monoidK.run {
      val liftedNext = AF.just(next)
      val newlyAccumulated = foldable.combineK(liftedNext)
      newlyAccumulated
    }
  }
}
a

addamsson

07/18/2019, 1:03 PM
so what
just
does is it wraps a value in a context?
s

simon.vergauwen

07/18/2019, 1:03 PM
So Applicative is simply
fun <A> just(a: A): List<A> = listOf(a)
a

addamsson

07/18/2019, 1:03 PM
which supplies the behavior
s

simon.vergauwen

07/18/2019, 1:03 PM
Exactly
a

addamsson

07/18/2019, 1:03 PM
just like
Option.just(1)
s

simon.vergauwen

07/18/2019, 1:04 PM
Yep, and we need to stay in
F
so we can accumulate the results in our collections.
MonoidK.combineK
is append or
+
for
List
.
a

addamsson

07/18/2019, 1:05 PM
so that's why we need to keep lifting the
next
values?
in whatever actual foldable the user provides
so this whole thing evaluates to
listOf()
+
listOf(1)
+
listOf(3)
+
listOf(4)
?
s

simon.vergauwen

07/18/2019, 1:06 PM
While explaining I noticed there is a mistake and the solution is actually easier.
Give me a sec so I can properly write it down to now confuse you 😜
a

addamsson

07/18/2019, 1:07 PM
ok
you don't need AF.just
s

simon.vergauwen

07/18/2019, 1:09 PM
So here we
foldLeft
over our original
Kind<F, A>
, and as an initial value we pass an “empty context”. I thought the empty context could be constructed by using
Monoid
+
Applicative
, but that’s inaccurate.
Monoid<Int>
returns
0
on
empty
so wrapping it with
Applicative#just
results in
listOf(0)
instead of
emptyList()
So what we want instead is simply
MonoidK
, which it’s
empty
returns
emptyList()
.
a

addamsson

07/18/2019, 1:09 PM
because
monoidK
has
empty
s

simon.vergauwen

07/18/2019, 1:09 PM
Yes exactly
No need for
Monoid
either.
You still need Applicative.
a

addamsson

07/18/2019, 1:10 PM
fun <A> Kind<F, A>.remove(element: A,
                            monoidK: MonoidK<F>,
                            AF: Applicative<F>): Kind<F, A> {
    val emptyValue = monoidK.run {
      empty<A>()
    }

    return foldLeft(emptyValue) { foldable: Kind<F, A>, next: A ->
      if (next == element) foldable
      else monoidK.run {
        foldable.combineK(AF.just(next))
      }
    }
  }
s

simon.vergauwen

07/18/2019, 1:10 PM
Yes that’s it.
a

addamsson

07/18/2019, 1:11 PM
wait a sec
s

simon.vergauwen

07/18/2019, 1:11 PM
val emptyValue = monoidK.run {
      empty<A>()
    }
Can simply be
monoidK.empty<A>()
So this boils down to
emptyList() + listOf(1) + listOf(3) + listOf(4)
for
ListK.foldable().run { listOf(1, 2, 3, 4).k().remove(2) }
emptyList().combineK(listOf(1).combineK(listOf(3).combineK(listOf(4))))
=>
monoidK.empty().combineK(just(1).combineK(just(3).combineK(just(4))))
If that still makes any sense 😄
a

addamsson

07/18/2019, 1:15 PM
can you explain in layman's terms what an
Applicative
is?
yes it recursively yanks out the elements from the foldable and combines them in a new one
so
Applicative
is for yanking?
s

simon.vergauwen

07/18/2019, 1:18 PM
Hmm I am thinking of a good description but it’s not easy for
Applicative
😅 In Arrow it only adds the power to lift values into
F
on top of
Functor
.
a

addamsson

07/18/2019, 1:18 PM
seems like we're on the right track:
"Should be able to remove from Foldable" {

    val result = ListK.foldable().run { listOf(1, 2, 3, 4).k().remove(2, ListK.monoidK(), ListK.applicative()) }

    result shouldBe listOf(1, 3, 4)
}
this passes
so it adds
just
btw shouldn't we use
Eq
instead of
==
?
s

simon.vergauwen

07/18/2019, 1:23 PM
Exactly, https://github.com/arrow-kt/arrow/blob/master/modules/core/arrow-core-data/src/main/kotlin/arrow/typeclasses/Applicative.kt However it’s better know for
ap
, which is not that useful in Kotlin and split off in
Apply
. What is more interesting there is the derived functions which allow for combining N types wrapped in
F
.
ListK.applicative(listOf(1, 2), listOf(3, 4)) { (a, b) -> Tuple2(a, b) }
//ListK(list=[Tuple2(a=1, b=3), Tuple2(a=1, b=4), Tuple2(a=2, b=3), Tuple2(a=2, b=4)])
Ye we should use
Eq
instead of
==
but most of the tests that aren’t using
Eq
have been there for a long time. We haven’t actively gone back to tests to rewrite them.
a

addamsson

07/18/2019, 1:25 PM
when i started my journey into FP I thought that monads will be the hardest concept to grasp
but it is applicatives
😄
I'm flabbergasted
s

simon.vergauwen

07/18/2019, 1:28 PM
Yes but functions wrapped in contexts aren’t very nice in Kotlin. It requires a bunch of manual currying and partial application which quickly becomes hard to read.
a

addamsson

07/18/2019, 1:28 PM
yep I saw it in VAVR.....
s

simon.vergauwen

07/18/2019, 1:28 PM
That’s why I mentioned that the derived functionality is much more interesting in Kotlin and probably VAVR.
a

addamsson

07/18/2019, 1:29 PM
take a look
man I need to learn Haskell now
but back to our topic
s

simon.vergauwen

07/18/2019, 1:32 PM
For example, let’s say we write a simple function that combines 2
F
and turns them into a tuple.
fun <F, A, B> Applicative<F>.tupled(fa: Kind<F, A>, fb: Kind<F, B>) : Kind<F, Tuple2<A, B>> {
  val ff: Kind<F, (B) -> Tuple2<A, B> = fa.map { a -> { b ->   Tuple2(a, b) } }
  return fb.ap(ff)
}
a

addamsson

07/18/2019, 1:32 PM
this is not very ergonomic:
listOf(1, 2, 3, 4).k().remove(2, ListK.monoidK(), ListK.applicative())
s

simon.vergauwen

07/18/2019, 1:32 PM
We could’ve done the same with
Monad
and
flatMap
but that means it’ll always be sequential.
a

addamsson

07/18/2019, 1:33 PM
i'm not following
s

simon.vergauwen

07/18/2019, 1:33 PM
But for
IO
we could write an
Applicative
that uses
parMapN
to combine 2 values in parallel.
This is ~~ similar to Either vs Validation with error accumulation.
a

addamsson

07/18/2019, 1:34 PM
the parallel part is a bit of a dark area for me
never done that with Arrow
s

simon.vergauwen

07/18/2019, 1:34 PM
It would be more ergonomic when the compiler can inject the instances, but we can also provide a more ergonomic
FunctorFilter
alternative
Nobody has since the PR is not merged 😄
In the next version you can ask every
Concurrent
typeclass for a
parApplicative()
instance.
This’ll mean that you can run programs written with
Applicative
in a concurrent/parallel manner with guarantees about cancelation, error handling and resource safety etc.
a

addamsson

07/18/2019, 1:36 PM
so i still not get it why an
Applicative
is capable of doing that
In Haskell Applicative and Apply are 1. I am not sure about other langs but in Scala it’s also split.
a

addamsson

07/18/2019, 1:38 PM
i'm trying to make sense of this
oh I see
Apply
has no kdocs 😢
s

simon.vergauwen

07/18/2019, 1:39 PM
😢
a

addamsson

07/18/2019, 1:40 PM
but now I understand this part:
fun <A, B> Kind<F, A>.ap(ff: Kind<F, (A) -> B>): Kind<F, B>
i'm going to improve the kdocs in a future pr
s

simon.vergauwen

07/18/2019, 1:40 PM
🙏 That would be awesome!
a

addamsson

07/18/2019, 1:40 PM
so what
Apply
does is that it applies a function wrapped in a context to a value wrapped in a context
☝️ 1
👍
what i'm not 100% sure about is what is the utility of this?
man, Arrow kills IDEA on my crappy mac
s

simon.vergauwen

07/18/2019, 1:41 PM
hahaha. It can also kill the high-end model if you try 😜
a

addamsson

07/18/2019, 1:42 PM
i don't like macs to be honest
s

simon.vergauwen

07/18/2019, 1:42 PM
It’s a super popular operator in Haskell due to currying, it’s not used very often in Scala either.
a

addamsson

07/18/2019, 1:43 PM
so how do I get parallel decomposition with this?
s

simon.vergauwen

07/18/2019, 1:44 PM
?
a

addamsson

07/18/2019, 1:48 PM
oh wait a second I think I get it
because both the function and the value is wrapped we can exert fine-grained control over the actual value on both sides, right?
by "both sides" i mean input and output
s

simon.vergauwen

07/18/2019, 1:49 PM
Exactly
a

addamsson

07/18/2019, 1:49 PM
wow
thanks for the explanation
s

simon.vergauwen

07/18/2019, 1:51 PM
Glad I could help you out 👍
a

addamsson

07/18/2019, 1:51 PM
👍
how do you thing we could mix
Eq
into this implementation of
remove
?
s

simon.vergauwen

07/18/2019, 1:54 PM
I’d probably do something like this.
Eq.any()
is
==
.
fun <A> Kind<F, A>.remove(element: A,
                          monoidK: MonoidK<F>,
                          AF: Applicative<F>,
                          eq: Eq<A> = Eq.any()): Kind<F, A> =
  foldLeft(monoidK.empty()) { foldable: Kind<F, A>, next: A ->
    if (eq.run { next.eqv(element) }) foldable
    else monoidK.run { foldable.combineK(AF.just(next)) }
  }
cc\ @raulraja we need KEEP-87 xD
a

addamsson

07/18/2019, 1:55 PM
😄
r

raulraja

07/18/2019, 1:58 PM
I’m working on it!
😘 2
as we speak!
val MetaComponentRegistrar.typeClasses: List<ExtensionPhase>
  get() =
    meta(
      classOrObject(::isExtension) { ktClass ->
        val typeClass = ktClass.typeClassName()
        val typeArgs = ktClass.typeArgumentNames()
        val factoryName = typeClass.decapitalize()
        println("intercepted ${ktClass.name}")
        listOf(
          ktClass.text,
          "fun $factoryName(): $typeClass<${typeArgs..","}> = TODO()"
        )
      },
      IrGeneration { compilerContext, file, backendContext, bindingContext ->
        file.transformChildren(object: IrElementTransformer<Unit> {

        }, Unit)
      }
    )
xD
check out the collection of declarations spreads for templates
typeArgs..","
where
","
can be any other delimiter
s

simon.vergauwen

07/18/2019, 2:00 PM
😲
Is that range?
r

raulraja

07/18/2019, 2:00 PM
yes
s

simon.vergauwen

07/18/2019, 2:00 PM
love it
cool find
That was the worst thing ever in kapt
r

raulraja

07/18/2019, 2:01 PM
private operator fun <A> List<A>.rangeTo(s: String): String =
  joinToString(s)
👌 1
a

addamsson

07/18/2019, 2:15 PM
where would you put this extension function?
fun <A> Kind<ForListK, A>.remove(element: A) = remove(element, ListK.monoidK(), ListK.applicative())
i've put it here:
@extension
interface ListKFoldable : Foldable<ForListK> {
  override fun <A, B> Kind<ForListK, A>.foldLeft(b: B, f: (B, A) -> B): B =
    fix().foldLeft(b, f)

  override fun <A, B> Kind<ForListK, A>.foldRight(lb: Eval<B>, f: (A, Eval<B>) -> Eval<B>): Eval<B> =
    fix().foldRight(lb, f)

  override fun <A> Kind<ForListK, A>.isEmpty(): kotlin.Boolean =
    fix().isEmpty()

  fun <A> Kind<ForListK, A>.remove(element: A) = remove(element, ListK.monoidK(), ListK.applicative())
}
as part of adding
remove
to
Foldable
I'm going to augment all implementations with this to keep it ergonomic
s

simon.vergauwen

07/18/2019, 2:16 PM
That function won’t be accessible since it’s only known in the
ListKFoldable
interface.
a

addamsson

07/18/2019, 2:17 PM
my test passes
"Should be able to remove from Foldable" {

  val result = ListK.foldable().run { listOf(1, 2, 3, 4).k().remove(2) }

  println(result)

  result shouldBe listOf(1, 3, 4)
}
oh wait a sec this is because it is int the context of
Foldable
what is the best practice in this case?
I thought that
@extension
will create an unboxed extension function
s

simon.vergauwen

07/18/2019, 2:19 PM
We typically put these either in the class of type itself as an instance method. Reason is that people mostly work either with polymoprhic with typeclasses or with concrete types.
a

addamsson

07/18/2019, 2:20 PM
this goes to
ListK
then, thanks
s

simon.vergauwen

07/18/2019, 2:20 PM
It does but only for the functions defined in the typeclass, not for additional defined functions in the impl.
a

addamsson

07/18/2019, 2:21 PM
remove
would make
ListK
invariant 😞
right now it is
<out A>
s

simon.vergauwen

07/18/2019, 2:22 PM
Right, extension function below
ListK
it is. The only way to walk around that problem now since Kotlin doesn’t allow for variance in functions.
p

pakoito

07/18/2019, 2:23 PM
jesus this thread
worth moving to a private room?
s

simon.vergauwen

07/18/2019, 2:23 PM
probably yes
1
a

addamsson

07/18/2019, 2:32 PM
this might be a stupid question
but I have trouble finding the
remove
function we imlpemented
fun <A> ListK<A>.remove(element: A): ListK<A> = this.??? // can't find it
fix()
doesn't help either
i've checked the other extension functions which are defined and they are using the functions defined in
ListK
itself
is this because the
Foldable
functions are implemented in
ListKFoldable
?
s

simon.vergauwen

07/18/2019, 2:34 PM
Right, that’s not a stupid question. That’s what we’re trying to solve by implementing the KEEP-87 in compiler plugins. It should be under
import arrow.core.extensions.listk.foldable.remove
We currently expose those functions with as global extension functions which resulted in horrible IDEA support and performance. So we compared a couple solutions and with compiler plugins we’ll be able to inject those functions directly in the concrete types.
a

addamsson

07/18/2019, 2:36 PM
I see
s

simon.vergauwen

07/18/2019, 2:36 PM
So with KEEP-87 and HKT plugin you’ll be able to implement a typeclass’s abstract behavior and you’ll also get all the derived behavior available in the concrete types on top of having it available using typeclasses.
a

addamsson

07/18/2019, 2:36 PM
btw to implement
remove
for
ListK
we could just use the wrapped `List`'s
remove
ok I'll skip this for now then
s

simon.vergauwen

07/18/2019, 2:37 PM
Yes, that would be the efficient overload for that function.
a

addamsson

07/18/2019, 2:37 PM
as long as this works I'm fine for this preliminary implementation:
"Should be able to remove from Foldable" {

  val result = ListK.foldable()
    .run { listOf(1, 2, 3, 4).k() }
    .remove(2, ListK.monoidK(), ListK.applicative())

  result shouldBe listOf(1, 3, 4)
}
I see no test class for
Foldable
though
s

simon.vergauwen

07/18/2019, 2:39 PM
Those are in
arrow-test
they’re implemented as an abstract test suite. Called
FoldableLaws.laws(...)
a

addamsson

07/18/2019, 2:39 PM
I see
s

simon.vergauwen

07/18/2019, 2:39 PM
1 test suite for n implementations 🙂
a

addamsson

07/18/2019, 2:40 PM
👍
i'll try to implement a test for remove
what do you think about this
PersistentMap
implementation?
/**
 * A [PersistentMap] is an immutable [Map] which implements structural sharing.
 */
@higherkind
data class PersistentMap<K, V>(private val map: Map<K, V>) : PersistentMapOf<K, V> {

  val isEmpty: Boolean = map.isEmpty()
  val size: Int = map.size

  operator fun get(key: K): Option<V> = Option.fromNullable(map[key])

  fun getOrElse(key: K, defaultValue: V): V = map.getOrElse(key) { defaultValue }

  fun containsKey(key: K): Boolean = map.containsKey(key)

  fun put(key: K, value: V): PersistentMap<K, V> {
    TODO()
  }

  fun remove(key: K): PersistentMap<K, V> {
    TODO()
  }
}
p

pakoito

07/18/2019, 2:54 PM
getOrElse is implicit by using Option for get
get(1).orElse { 1 }
a

addamsson

07/18/2019, 3:05 PM
this is how it looks like now:
/**
 * A [PersistentMap] is an immutable [Map] which implements structural sharing.
 */
@higherkind
data class PersistentMap<K, V>(private val map: HashArrayMappedTrie<K, V>) : PersistentMapOf<K, V> {

  val isEmpty: Boolean = map.isEmpty
  val size: Int = map.size

  operator fun get(key: K): Option<V> = map[key]

  fun containsKey(key: K): Boolean = map.containsKey(key)

  fun put(key: K, value: V): PersistentMap<K, V> {
    return PersistentMap(map.put(key, value))
  }

  fun remove(key: K): PersistentMap<K, V> {
    return PersistentMap(map.remove(key))
  }
}
I'm so psyched
i almost have it working
p

pakoito

07/18/2019, 3:55 PM
yay 😄
a

addamsson

07/18/2019, 4:18 PM
i've learned more in this last week about FP than in the last year combined
p

pakoito

07/18/2019, 4:20 PM
that's what you happen when you implement stuff hahaha
a

addamsson

07/18/2019, 4:22 PM
😄
so do we move this convo into a room?
just to reiterate:
the implementations I see in
MapK
are necessary because we don't have KEEP-87 yet, right?
so I'll need to replicate them in
PersistentMapK
s

simon.vergauwen

07/18/2019, 5:22 PM
No, that's because we cannot make the std Map extend
Kind
.
Which you already do for
PersistentMap
a

addamsson

07/18/2019, 5:39 PM
got it
so I only have to add the implementations for the typeclasses?
like in
mapk.kt
although I'm not sure how that should work?
what do I have to do to make
PersistentMapK
a
Foldable
apart from this?
@extension
interface MapKFoldable<K> : Foldable<PersistentMapKPartialOf<K>>
how do I know what I need to override?
s

simon.vergauwen

07/18/2019, 5:54 PM
Same as you'd do in oop, the onces that provide better performance than the default impl.
a

addamsson

07/18/2019, 5:56 PM
so just by adding this I've made it a
Functor
?
@extension
@undocumented
interface PersistentMapKFunctor<K> : Functor<PersistentMapKPartialOf<K>>
s

simon.vergauwen

07/18/2019, 5:57 PM
Ah sorry. There you should implement `Functor`'s abstract functions so
PersistentMapKFunctor
becomes an interface with no abstract functions.
a

addamsson

07/18/2019, 5:58 PM
i'm a bit lost
s

simon.vergauwen

07/18/2019, 5:58 PM
For
Functor
that's only
map
if I'm not mistaken.
a

addamsson

07/18/2019, 5:58 PM
i can't call
map
on my
PersistentMapK<Int, Int>()
s

simon.vergauwen

07/18/2019, 5:58 PM
Sec
a

addamsson

07/18/2019, 5:58 PM
even though I have
@extension
@undocumented
interface PersistentMapKFunctor<K> : Functor<PersistentMapKPartialOf<K>>
which I copied from
mapk.kt
i tried the official docs but I didn't find a guide for implementing my own typeclasses
r

raulraja

07/18/2019, 6:00 PM
fix().map
s

simon.vergauwen

07/18/2019, 6:00 PM
@extension
@undocumented
interface PersistentMapKFunctor<K> : Functor<PersistentMapKPartialOf<K>> {
   override fun <A, B> PersistentMapKOf<A>.map(f: (A) -> B): PersistentMapK<B> =
     fix().map(f)
}
Now that you’ve implemented all abstract methods of
interface Functor<F>
, we can no generate
fun <K> PersistentMapK.Companion.functor(): Functor<PersistentMapKPartialOf<K>> = object : PersistentMapKFunctor<K> { }
r

raulraja

07/18/2019, 6:01 PM
Damm Simon beat me
a

addamsson

07/18/2019, 6:01 PM
i'll try thanks
s

simon.vergauwen

07/18/2019, 6:01 PM
haha 😄 You can also overload other functions that
Functor
provides with faster implementations, but that’s optional.
Did you implement
map
on
PersistentMapK
?
a

addamsson

07/18/2019, 6:02 PM
no
you said that I don't need it
r

raulraja

07/18/2019, 6:05 PM
You are gonna need roughly the same api MapK specializes map, fold etc
To implement Foldable, FunctorFilter etc
Each type class instance will delegate to the actual function via fix. This will change with compiler plugins and arrow meta at the end of the year
s

simon.vergauwen

07/18/2019, 6:06 PM
With KEEP-87 😕 Sorry if I was unclear. Our current strategy is to implement all abstract typeclass functions in the data types directly so that in the interface impl we can delegate to them by moving between kinded and concrete world using
fix
. I.e.
fix().map(f)
. So if we look at
Option
it looks like this with only the Functor impl.
@higherkind sealed class Option<out A> : OptionOf<A> {
  object None: Option<Nothing>()
  data class Just<A>(val a: A): Option<A>()
  fun <B> map(f: (A) -> B): Option<B> = ...
}

interface OptionFunctor : Functor<ForOption> {
   fun <A, B> OptionOf<A>.map(f: (A) -> B): Option<B> =
     fix() //returns Option<A>
       .map(f)
}
a

addamsson

07/18/2019, 6:07 PM
so then I do have to implement these functions just like in
MapK
s

simon.vergauwen

07/18/2019, 6:08 PM
Sorry for the confusion
a

addamsson

07/18/2019, 6:08 PM
even though we can implement
PersistentMapKOf
s

simon.vergauwen

07/18/2019, 6:08 PM
PersistentMapKOf
is simply
typealias PersistentMapKOf<K, V> = Kind2<ForPersistentMapK, K, V>
What we can do with the compiler plugins is tell the compiler that
PersistentMapK<K, V>
is equal to
PersistentMapKOf<K, V>
and additionally we can tell the compiler that
PersistentMapK
has all the functions that are implemented by the typeclasses for
ForPersistentMapK
.
a

addamsson

07/18/2019, 6:10 PM
I mean you said
s

simon.vergauwen

07/18/2019, 6:10 PM
With that last bit there won’t be the need to manually delegate
a

addamsson

07/18/2019, 6:11 PM
that I don't have to replicate them
but as it turns out I do have to?
s

simon.vergauwen

07/18/2019, 6:11 PM
Oh, I miss-understood you there.
a

addamsson

07/18/2019, 6:11 PM
😄
ok so is it enough if I implement
map
like this:
@extension
@undocumented
interface PersistentMapKFunctor<K> : Functor<PersistentMapKPartialOf<K>> {
  override fun <A, B> Kind<PersistentMapKPartialOf<K>, A>.map(f: (A) -> B): Kind<PersistentMapKPartialOf<K>, B> {
    TODO("not implemented")
  }
}
s

simon.vergauwen

07/18/2019, 6:12 PM
Yes, that is the minimum definition of a
Functor
a

addamsson

07/18/2019, 6:12 PM
if I understood you correctly then yes, because
PersistentMapK
is already a kind and kapt can work its magic
👍
why am I getting
> Task :arrow-persistent-data-structures:kaptKotlin FAILED
e: error: Arrow's annotations can only be used on Kotlin classes. Not valid for error.NonExistentClass
after each build?
I need to manually remove
PersistentMapKOf
from
PersistentMapK
then rebuild then re-add it
s

simon.vergauwen

07/18/2019, 6:23 PM
Ai, you have
@higherkind
and
@extension
in the same module?
This is a kapt issue 😕 I’d suggest to just put the
@higherkind
boilerplate in the module for now.
a

addamsson

07/18/2019, 6:24 PM
ok
i can't wait to get rid of kapt
1
I need to add a
companion object
to
PersistentMapK
just realized
do you have a doc somewhere for prospective typeclass implementors?
btw
remove
is not going to work since
Foldable
folds the values
but
remove
works with the
key
, not the value 😞
maybe I'm implementing
Foldable
the wrong way
anyway my brain just melted for today so I'm gonna shelve this for now
s

simon.vergauwen

07/18/2019, 7:14 PM
Hmm... Actually for complex immutable structures I can also recommend looking
Optics
.
You're not implementing
Foldable
wrong. We don't know the type of the
key
inside
Foldable
.
I need to know the type of
key
in order to implement an efficient
remove
operation
s

simon.vergauwen

07/18/2019, 7:20 PM
We'll have the same problem with
key
for
FunctorFilter
.
a

addamsson

07/18/2019, 7:20 PM
although I already have a
remove
function in
PersistentMap
this is where i'm at right now with the fork
it works but it is a bit rough around the edges
s

simon.vergauwen

07/18/2019, 7:21 PM
And we don't know the type there. We'd have to create an
IndexedFoldable
or
IndexedFunctorFilter
a

addamsson

07/18/2019, 7:21 PM
most folks will use the
remove(element)
function anyway
s

simon.vergauwen

07/18/2019, 7:22 PM
Yes, maps are indexed structures so they need indexed typeclasses but we don't have any of those.
Having this for other immutable structures would be great. Remove is much nicer when you would filter on a constant.
Nice work!
a

addamsson

07/18/2019, 7:27 PM
should I open a PR so you can review it?
s

simon.vergauwen

07/18/2019, 7:30 PM
That's probably easiest to make comments and talk code.
a

addamsson

07/18/2019, 7:30 PM
👍
is it possible that the build failed because of KTlint?
Missing newline before ")"
I feel physical pain if I can't put the
)
at the end of the line (just like in Clojure)
also why is this a build failure?
s

simon.vergauwen

07/18/2019, 7:56 PM
We rather wanted to have a CI controlled styled than a user controlled one.
Consistency or OCD. Pick one ;p
a

addamsson

07/18/2019, 8:09 PM
hehe
i'll fix this
although i've been pressing the format command
so there's something amyss here
the build succeeds locally but fails remotely 😞
@Imran/Malic
i

Imran/Malic

07/19/2019, 11:52 AM
@addamsson did you fix your formating issue, if not it may lay here:
fun <A> Kind<F, A>.remove(element: A,
	                            monoidK: MonoidK<F>,
	                            AF: Applicative<F>): Kind<F, A> {
I would rather inline it
s

simon.vergauwen

07/19/2019, 11:56 AM
./gradlew ktlintFormat
should fix all style isssues, and the onces it cannot like
*
imports will get linked in the console.
a

addamsson

07/19/2019, 12:54 PM
I didn't have time since I opened the PR, but I'll fix it today evening!
s

simon.vergauwen

07/19/2019, 12:54 PM
No rush 🙂 Whenever you have time is great 👍
a

addamsson

07/19/2019, 1:36 PM
@Imran/Malic is there something specific which you wanted to talk about?
I'm not familiar with the
Traverse
typeclass, but we discussed
Foldable
with @simon.vergauwen
s

simon.vergauwen

07/19/2019, 1:48 PM
Traverse
inherits from
Foldable
and
Functor
and is a very popular and powerful typeclass to do operations.
fun getUserById(id: Int): IO<User> = ...
val operations: List<IO<User>> = listOf(1, 2, 3).k().map(::getUserById)
val sequenced: IO<List<User>> = operations.sequence(IO.applicative())

val traversed: IO<List<User>> = listOf(1, 2, 3).k().traverse(IO.applicative()) { id -> getUserById(id) }
Having
List<IO<User>>
is useless to work with so you need to be able to swap the order of the contexts.
☝️ using
IO.parApplicative()
which we discussed earlier will automatically make this work in parallel. Which is exactly what we I did in that PR I shared,
parTraverse
is just an alias for
traverse
with
parApplicative()
.
a

addamsson

07/19/2019, 2:06 PM
what is the use of
IO
?
i'm not sure I understand the purpose of
Traverse
when you do things in parallel how does it work under the hood?
s

simon.vergauwen

07/19/2019, 2:08 PM
It’s an effect type similar to
io.reactivex.Single
,
reactor.Mono
which are both pure and fairly principled. You can also see it as a FP version of
kotlinx.coroutines.Deferred
, or Scala Future.
a

addamsson

07/19/2019, 2:08 PM
coroutines?
s

simon.vergauwen

07/19/2019, 2:08 PM
IO
uses coroutines for threading under the hood yes
a

addamsson

07/19/2019, 2:08 PM
@simon.vergauwen which one,
IO
or
Traverse
?
i

Imran/Malic

07/19/2019, 2:08 PM
Traverse is the typeclass
a

addamsson

07/19/2019, 2:09 PM
I mean
IO
~
Deferred
or
Traverse
~
Deferred
?
s

simon.vergauwen

07/19/2019, 2:09 PM
IO
~
Deferred
i

Imran/Malic

07/19/2019, 2:09 PM
IO is our main datatype in effects aka Fx
a

addamsson

07/19/2019, 2:09 PM
what's an effect?
i

Imran/Malic

07/19/2019, 2:10 PM
Are you familiar with cats-effects
The one from scala
a

addamsson

07/19/2019, 2:10 PM
I have
0
Scala knowledge
s

simon.vergauwen

07/19/2019, 2:10 PM
effect is just short for side-effect.
a

addamsson

07/19/2019, 2:10 PM
I came from a Clojure/Java background
s

simon.vergauwen

07/19/2019, 2:11 PM
Oh Clojure cool
a

addamsson

07/19/2019, 2:11 PM
there is a reason why I contribute persistent data structures 😄
s

simon.vergauwen

07/19/2019, 2:12 PM
Anyway, Traverse is useful when you are in context
F
and you want to run functions that return
G
. So in our example our original context was
List
and we wanted to run functions that return
IO
. But a simpler example maybe.
fun getOptionalUser(id: Int): Option<User> = ..
listOf(1, 2, 3).k().traverse { id -> getOptionalUser(id) }
If we use map here then you’d end up with
List<Option<User>>
but lets say I am not interested in having empty values (
None
) in my list. So I want to short-circuit just like `map`/`flatMap` for
Option
and have an
Option<List<User>>
which either contains all users or
None
.
a

addamsson

07/19/2019, 2:14 PM
what will
listOf(1, 2, 3).k().traverse { id -> getOptionalUser(id) }
return?
s

simon.vergauwen

07/19/2019, 2:14 PM
Option<List<User>>
a

addamsson

07/19/2019, 2:15 PM
so it not only maps (
Functor
) but also folds
Foldable
?
s

simon.vergauwen

07/19/2019, 2:16 PM
Exactly, hence
interface Traverse<F> : Functor<F>, Foldable<F>
a

addamsson

07/19/2019, 2:16 PM
hm
how does this relate to
Applicative
?
or rather
Apply
s

simon.vergauwen

07/19/2019, 2:17 PM
You need to supply
Applicative<G>
so that you can use
map2
to combine 2
G
elements.
a

addamsson

07/19/2019, 2:17 PM
as it seems that this takes a wrapped value and returns a wrapped value
s

simon.vergauwen

07/19/2019, 2:19 PM
yes but it swap containers.
fun Kind<G, Kind<F, A>>.sequence(AF: Applicative<F>): Kind<F, Kind<G, A>> = traverse(AF, :identity)

val listOfOptions: List<Option<Int>> = listOf(1.some(), 2.some())

listOfOptions.sequence() //Some([1, 2])
listOf(1.some(), None).sequence() // None
☝️ this is what made it click for me. Since
listOf(1, 2).map { Some(it) }.sequence()
is the same as
listOf(1, 2).traverse { Some(it) }
a

addamsson

07/19/2019, 2:23 PM
oh I see
👍
so can we move this convo to its own channel?
I've fixed the issues in the review
i'm going on a vacation for 2 weeks
i'll be back! 👍
s

simon.vergauwen

07/19/2019, 10:43 PM
Enjoy your vacation!
r

raulraja

07/23/2019, 8:58 AM
What are the laws of remove?
s

simon.vergauwen

07/23/2019, 8:59 AM
I'd imagine something like
remove(a).find { a } == false
but currently we haven't defined any yet
r

raulraja

07/23/2019, 9:00 AM
Makes sense
@Imran/Malic I believe 'remove' belongs in Foldable and FunctorFilter if we can express a valid law for their type class
I think that is true for any other combinator unless it's overly specialized. Remove has clear semantics even if it can be expressed with just folds
And we are the ones making the law and shaping the typeclasses to how are best used.
i

Imran/Malic

07/23/2019, 9:18 AM
@raulraja makes sense. Maybe there is a way to reconstruct some of the Foldable FunctorFilter Laws in away where remove is also tested. I post something here if I find something