Hello Sorry for the dumb question but is there a w...

# arrow

[Alexander Schell]

Hello
Sorry for the dumb question but is there a way to transform a SequenceK<Either<Throwable, Iterable<T>>> to a SequenceK<Either<Throwable, T>> while preservering the original potential throwable?
I'm tinkering around with this problem but without getting anywhere... =(
Thanks!

[Patrick Louis]

Hey,
.sequence(Either.applicative) should be able to convert your Either<Throwable<Iterable<T>> into Iterable<Either<Throwable,T>>. I'm not sure if you can flatten over nested Sequences directly but I guess you'd be able to figure something out from there.

[Alexander Schell]

Hello Patrick
Thanks for your Answer! I'll give it a try.

[pakoito]

As Patrick said, sequence first, then flatten should do the trick!

[Alexander Schell]

Oi, Thanks Paco

Well, I tried to weasel my way around the problem... 😛

fun fetchEventsByStatusSafe(eventStatus: EventStatus,
                                fetchBatchSize: Int): SequenceK<Either<Throwable, Event>> {
        LOGGER.debug("Fetching Events (eventStatus = $eventStatus)")
        return generateSequence(fetchPageSafe(eventStatus, PageRequest.of(0, fetchBatchSize))) {
            either -> when (either) {
                is Either.Right -> {
                    when (either.b.hasNext()) {
                        true -> fetchPageSafe(eventStatus, either.b.nextPageable())
                        else -> null
                    }
                }
                else -> either
            }
        }.flatten().k()
    }

    private fun fetchPageSafe(eventStatus: EventStatus,
                              pageable: Pageable): Either<Throwable, Page<Event>> {
        LOGGER.debug("Fetching Page<Event> (eventStatus = $eventStatus, pageNumber = ${pageable.pageNumber}, pageSize = ${pageable.pageSize})")
        return Either.fx {
            eventRepository.findAllByStatus(eventStatus.eventStatusId, pageable, EVENT_WITH_PARAMETERS_GRAPH)
        }
    }

    private fun Sequence<Either<Throwable, Page<Event>>>.flatten(): Sequence<Either<Throwable, Event>>
            = this.map { either -> either.flatten() }.flatten()

    private fun Either<Throwable, Page<Event>>.flatten(): Sequence<Either<Throwable, Event>>
            = when (this) {
                is Either.Right -> {
                    this.b.asSequence().map { event -> Either.right(event) }
                }
                is Either.Left -> sequenceOf(this)
            }

[Alexander Schell]

But I don't know if this is an elegant way to solve it... 😕

[pakoito]

return Either.fx {
            eventRepository.findAllByStatus(eventStatus.eventStatusId, pageable, EVENT_WITH_PARAMETERS_GRAPH)
        }

[pakoito]

why do you need fx there?

[pakoito]

exceptions?

[Alexander Schell]

Oi, I wanted to attach possible exceptions to the elements in the sequence so I'm able to process them in a stream-like fashion.
The eventRepository-object is a spring EntityGraphJpaRepository and have I have to deal with all the possible failures using it.
Paco, Patrick: Thanks for all your help! :)

[Patrick Louis]

Hey Alexander,
I'd suggest something like this

fun fetchEventsByStatusSafe(eventStatus: EventStatus,
                                fetchBatchSize: Int): SequenceK<Either<Throwable, Event>> {
        LOGGER.debug("Fetching Events (eventStatus = $eventStatus)")
        return generateSequence(fetchPageSafe(eventStatus, PageRequest.of(0, fetchBatchSize))) {
            either -> either.map { if (it.hasNext()) fetchPageSafe(eventStatus, either.b.nextPageable()) else null }
        }.flatMap { it.sequence(Either.applicative())<<<possibly '.toSequence()'>>> }.fix()
    }

    private fun fetchPageSafe(eventStatus: EventStatus,
                              pageable: Pageable): Either<Throwable, Page<Event>> {
        LOGGER.debug("Fetching Page<Event> (eventStatus = $eventStatus, pageNumber = ${pageable.pageNumber}, pageSize = ${pageable.pageSize})")
        return eventRepository.findAllByStatus(eventStatus.eventStatusId, pageable, EVENT_WITH_PARAMETERS_GRAPH).right()
        // if you want exceptions to be caught here you should probably do something like 
        // return IO { eventRepository.findAllByStatus(eventStatus.eventStatusId, pageable, EVENT_WITH_PARAMETERS_GRAPH) }.attempt().unsafeRunSync()
    }

[Alexander Schell]

Hello Patrick
Thank you! I'm stumbling over the part around 'possibly'. I've to confess that I'm rather tired now; started today at 0600 and I can hear my sixpack kilkenny's and my balcony calling for me... 😄
I'll try to wrap my head around your suggestion tomorrow.
A nice day to the both of you!
Best regards
Alex

[Alexander Schell]

Moin! 🙂
Well, it seems that I'm as blind as I'm dumb. Just can't find an extension-function 'sequence'/'toSequence' for Either accepting an EitherApplicative.
I guess I'm missing a dependency.
Thanks for the hint to use IO!
Best regards
Alex

[Patrick Louis]

Hey Alexander,
the toSequence extension function should exist on your nested iterable.
Given the following

val thing: Sequence<Either<Throwable,Iterable<XXX>>>
thing.map { it.sequence(Either.applicative()) } // Sequence<Iterable<Either<Throwable,XXX>>>

in order to flatten this to a Sequence<Either<Throwable,XXX>> you'd need to turn the Iterable into a Sequence.
That's where the toSequence() extension function would come into play, given that it's provider by the Iterable.

Also in order for this to work, the Iterable - you're using Page iirc - must implement the Traverse TypeClass: https://arrow-kt.io/docs/next/apidocs/arrow-core-data/arrow.typeclasses/-traverse/

[Alexander Schell]

Hello Patrick
Sorry, I was buried in work and didn't find the time to take look at our conversation.
Again: Thanks for your help! 🙂
So it seems I have to extend Page. Okay, let's do this... 😄
Best regards

[Alexander Schell]

Alex

[Alexander Schell]

OjWej, that's not as easy as I thought...

[Patrick Louis]

you probably have to create a separate module for the extension as well as one for the higherkind definition for Page. This is because the annotation processing for the extension-annotation requires the generated output of the @higherkind annotation. if you need any help with that let me know 🙂

[Alexander Schell]

Hello Patrick
I'm afraid that I've made a mistake... 🤔
As far as I've seen now one can use Traverse to do something like this:
Kind<A, Kind<B, C>> → Kind<B, Kind<A, C>>
I hope I got this right... 😄

[Alexander Schell]

What I want to do is slightly different. In my case 'C' is an Iterable - The Spring-Page-Class.
So I want to do something like this:
Kind<A, Kind<B, Kind<C, D>>> → Kind<A, Kind<B, D>>
Where A is of SequenceK<Either<Throwable, Page<D>> and Page is an Iterable over D.
So: SequenceK<Either<Throwable, Page<D>> → SequenceK<Either<Throwable, D>>.
So the 'container' C hast to be replaced with the collection of it's contents, than forming up another SequenceK I can iterate - Maybe in parallel - over...

[Alexander Schell]

I'm sorre for the confusion 😕
Best regards
Alex

[Patrick Louis]

So if I get this right: You basically want to resolve the Page<D> objects to its content D ? If so that can be achieved with

val items: SequenceK<Either<Throwable,Page<D>>>
items.map{ either -> either.map { page -> page.whateverExtractsTheContent() } } // SequenceK<Either<Throwable,D>>

This would not evaluate the pages in parallel however.

[Patrick Louis]

So looking back at your snippet, I've cooked something up that should work for your case:

fun fetchEventsByStatusSafe(
    eventStatus: EventStatus,
    fetchBatchSize: Int
): SequenceK<Either<Throwable, Event?>> {
    return generateSequence(fetchPageSafe(eventStatus, PageRequest.of(0, fetchBatchSize))) {
        IO.fx {
            val current = !it
            if (current?.hasNext() == true) !fetchPageSafe(eventStatus, current.nextPageable()) else null // Page<Event>?
        }
    }.k().parTraverse(SequenceK.traverse()) {
        it.map { current ->
            current?.whateverGetsTheContent()
        }.attempt()
    }.unsafeRunSync().fix()
}

private fun fetchPageSafe(
    eventStatus: EventStatus,
    pageable: Pageable
): IO<Page<Event>?> {
    return IO {
        eventRepository.findAllByStatus(eventStatus.eventStatusId, pageable, EVENT_WITH_PARAMETERS_GRAPH).right()
    }
}

This will also run the current?.whateverGetsTheContent() in parallel.

[Alexander Schell]

Hello Patrick
Thanks a lot! I'll use your first suggestion for the selection of the events and your second one for processing them.
Maybe it was not such a good idea to learn Kotlin and programming with Arrow-Kt at the same time but I'm stubborn as hell, so here we go... 😄
Best regards
Alex

[Patrick Louis]

Hey Alex,
you certainly haven't chosen the easy path but I think it's worth it in the end.
A good resource when you're looking for certain arrow-functions by their signature is hoogle (haskell-google).
i.e. if youre looking for a function that inverts two containers like List<Option<A>> to Option<List<A>> you can search for it like this:
https://hoogle.haskell.org/?hoogle=f%20t%20a%20-%3E%20t%20f%20a
the results will most likely also be found in arrow under a similar name.
Cheers,
Patrick

[Alexander Schell]

Hello Patrick
That's why I've chosen one of our most simple batch-processes to migrate it to Kotlin. I thought that would by a good approach to get familiar with Kotlin and Arrow. It's mechanics are quite straight and there is room to try out some concepts from Kotlin/Arrow.
Problem is, I don't have the time to do this during my normal work hours. So I'm tinkering around with it in after-hours, weekends and - Like now - during my holidays.
Thanks for the link to Hoogle; completely forgot this one. 😛
It's more than twenty-five years ago that I had to use Haskell. And all I can remember is my struggling with the monad concept.
Well, we'll see how this will work out...
Best regards
Alex