https://kotlinlang.org logo
Join Slack
Powered by
I have upgraded to arrow 1.1.2, and switched from ...
# arrow
i
I have upgraded to arrow 1.1.2, and switched from 
either
to 
Effect
. I like it BTW, it is a nice abstraction IMHO. But I have some issues. It is a multiplatform project and issue might not be related to Arrow, but it emerged once I switched to 
Effect
. It looks related to 
native
target(s) only. More in the thread …
Copy code
Details: Internal error in body lowering: kotlin.NotImplementedError: An operation is not implemented: Unknown file

 * Source files: 
 * Compiler version info: Konan: 1.6.21 / Kotlin: 1.6.21
 * Output kind: PROGRAM

e: org.jetbrains.kotlin.backend.common.CompilationException: Back-end: Please report this problem <https://kotl.in/issue>
/home/runner/work/fmodel/fmodel/application-arrow/src/commonMain/kotlin/com/fraktalio/fmodel/application/MaterializedViewArrowExtension.kt:36:9
Problem with `suspend fun <S : Any?, E : Any?> MaterializedView<S, E>.handleEither(event: E): Effect<Error, S> {
  local fun S?.eitherComputeNewStateOrFail(event: E): Effect<Error, S> {
    return effect<Error, S>(f = local suspend fun EffectScope<Error>.<anonymous>(): S {
      return try { // BLOCK
        (<this>, <this>).computeNewState(event = event)
      }
      catch (t: Throwable){ // BLOCK
        $this$effect.shift<S>(r = CalculatingNewViewStateFailed<S?, E>(state = <this>, event = event, throwable = t.nonFatalOrThrow()))
      }

    }
It is an open-source project, so I will share a link to the build https://github.com/fraktalio/fmodel/runs/6147738743?check_suite_focus=true#step:4:361
…and the PR 
<https://github.com/fraktalio/fmodel/pull/95/files>
s
Hey @Ivan Dugalic, Thanks for sharing. On first sight it looks like a compiler issue we didn't ran into in the Arrow repo, or subsequent projects.
Lets try to reproduce in a smaller example
seems like it might be related to 
inline suspend
. There are a couple issues with that still, are you using that a lot in your code?
r
Seems like is hitting one of these two, maybe if we can look at the actual declaration failing with the debugger we can workaround it and submit an issue
Copy code
val IrDeclaration.file: IrFile
    get() = fileOrNull ?: TODO("Unknown file")
s
Seems like is hitting one of these two, maybe if we can look at the actual declaration failing with the debugger we can workaround it and submit an issue
That'd be great! Inline suspension seems to be stabilizing in the compiler relatively fast, if we can support them with good issues that'd be awesome Nice find!
i
Ohh, good catch. I keep forgetting that TODO is a function 🙂 Thanks @simon.vergauwen and @raulraja!!! I am not sure if I can submit a 
good
issue 😞 TBH, I would need time to understand it at first.
r
does 
suspend fun <S : Any?, E : Any?> MaterializedView<S, E>.handleEither(event: E): Effect<Error, S>
have local function on its body?
if so, what happens if you move those out?
for example: 
local fun S?.eitherComputeNewStateOrFail(event: E): Effect<Error, S>
local funtions needs special treatment as backends in the compiler and their targets would have to desugar them out.
I’m guessing they are not correctly reparented while transformed in the IR lowering phase
i
I tried that without much luck, and the same issue occurred. I also tried to simplify the case by removing local functions.
PR (
<https://github.com/fraktalio/fmodel/pull/95/files>
) is not doing much. Only switching to 
efffect
It was working with 
either
. Everything else is the same.
Without local functions
Copy code
e: org.jetbrains.kotlin.backend.common.CompilationException: Back-end: Please report this problem <https://kotl.in/issue>
/Users/idugalic/work/fraktalio/fmodel/application-arrow/src/commonMain/kotlin/com/fraktalio/fmodel/application/MaterializedViewArrowExtension.kt:36:9
Problem with `suspend fun <S : Any?, E : Any?> MaterializedView<S, E>.handleEither(event: E): Effect<Error, S> {
  return effect<Error, S>(f = local suspend fun EffectScope<Error>.<anonymous>(): S {
    return (<this>, (<this>, (<this>, event).fetchState()).computeNewState(event = event)).save()
  }
)
}

`
Details: Internal error in body lowering: kotlin.NotImplementedError: An operation is not implemented: Unknown file
        at org.jetbrains.kotlin.ir.util.IrUtilsKt.getFile(IrUtils.kt)
        at org.jetbrains.kotlin.backend.common.lower.LocalDeclarationsLowering$LocalDeclarationsTransformer.collectLocalDeclarations(LocalDeclarationsLowering.kt:893)
 ...
Please notice, that it compiles without 
native
targets.
Copy code
//    val hostOs = System.getProperty("os.name")
//    val isMingwX64 = hostOs.startsWith("Windows")
//    val nativeTarget = when {
//        hostOs == "Mac OS X" -> macosX64()
//        hostOs == "Linux" -> linuxX64()
//        isMingwX64 -> mingwX64()
//        else -> throw GradleException("Host OS is not supported in Kotlin/Native.")
//    }
//    nativeTarget.compilations.all {
//        kotlinOptions.verbose = true
//    }
BTW, no pressure with this. I will try to reproduce this in a smaller example in the following week. Thank you for your help!!! I will keep you posted.
r
Copy code
local suspend fun EffectScope<Error>.<anonymous>()
seems there is still a local inside
Copy code
LocalDeclarationsLowering$LocalDeclarationsTransformer.collectLocalDeclarations(LocalDeclarationsLowering.kt:893)
i
Not sure from where 
local
is coming. I believe I removed all local/inner scope functions. I have further simplified the case (handlerEither function)
Copy code
fun <S, E> handleEither(event: E): Effect<Throwable, S> = effect {
    shift(RuntimeException("OUPS"))
}
and the test:
Copy code
class MaterializedViewTest : FunSpec({
    test("Materialized view - even number added") {
           handleEither<EvenNumberState, NumberEvent.EvenNumberEvent>(
               EvenNumberAdded(
                   Description("2"),
                   NumberValue(2)
               )
           )
    }
})
and still have the same exception:
Copy code
e: org.jetbrains.kotlin.backend.common.CompilationException: Back-end: Please report this problem <https://kotl.in/issue>
/Users/idugalic/work/fraktalio/fmodel/application-arrow/src/commonMain/kotlin/com/fraktalio/fmodel/application/MaterializedViewArrowExtension.kt:31:1
Problem with `fun <S : Any?, E : Any?> handleEither(event: E): Effect<Throwable, S> {
  return effect<Throwable, S>(f = local suspend fun EffectScope<Throwable>.<anonymous>(): S {
    return $this$effect.shift<S>(r = RuntimeException(message = "OUPS"))
  }
)
}

`
Details: Internal error in body lowering: kotlin.NotImplementedError: An operation is not implemented: Unknown file
        at org.jetbrains.kotlin.ir.util.IrUtilsKt.getFile(IrUtils.kt)
        at org.jetbrains.kotlin.backend.common.lower.LocalDeclarationsLowering$LocalDeclarationsTransformer.collectLocalDeclarations(LocalDeclarationsLowering.kt:893)
        at org.jetbrains.kotlin.backend.common.lower.LocalDeclarationsLowering$LocalDeclarationsTransformer.lowerLocalDeclarations(LocalDeclarationsLowering.kt:248)
        at org.jetbrains.kotlin.backend.common.lower.LocalDeclarationsLowering.lower(LocalDeclarationsLowering.kt:102)
        at org.jetbrains.kotlin.backend.common.lower.inline.LocalClassesInInlineLambdasLowering$lower$1.visitCall(LocalClasses.kt:59)
        at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitCall(IrElementTransformerVoid.kt:215)
        at org.jetbrains.kotlin.ir.visitors.IrElementTransformerVoid.visitCall(IrElementTransformerVoid.kt:24)
        at org.jetbrains.kotlin.ir.expressions.impl.IrCallImpl.accept(IrCallImpl.kt:47)
        at org.jetbrains.kotlin.ir.expressions.IrExpression.transform(IrExpression.kt:33)
Using KoTest 5.2.3, BTW.
s
Does the 
local
come from the fact that the 
EffectScope
is an anonymous implementation? https://github.com/arrow-kt/arrow/blob/eeac854dec3677fc8fd42871a7df136f3ba0d02f/ar[…]w-core/src/commonMain/kotlin/arrow/core/continuations/Effect.kt Weird that it's only failing for native though, since it seems to be failing in 
common
code 🤔
Oh, wait no it starts at 
IR
so more likely during code-gen phase.
i
Does the local come from the fact that the EffectScope is an anonymous implementation?
It could be you are on the right track.
r
most likely, and that inlines the object crossinlining 
f
may be reproable with a custom version of the 
effect
function that uses a class instead of the object to implement 
Effect
the issue is a bug in IR local lowering, but the workaround may be not implementing Effect like that so it just inlines a constructor call for the reference of 
f
. Also not sure inline here buys us much, it may be creating bigger binaries because of the inlined impl
✅ 1
i
I will be able to try that later in the day as an experiment (implementing Effect)
👍 1
I can confirm that by using the Effect class (not object) the problem is solved 🙌
Copy code
fun <R, A> myEffect(f: suspend EffectScope<R>.() -> A): Effect<R, A> = MyEffect(f)
class MyEffect<R, A>(val f: suspend EffectScope<R>.() -> A) : Effect<R, A> { // copy-pasted the implementation }
I prefer an Object rather than a Class, but it is a workaround. I wonder if it can be a workaround just for the Native target (Expect/Actual)?
r
it’s not an object at the end but a local instance, same as a lamba or the class and same alloc cost, but higher bytecode footprint because its inlined
I personally think arrow should change it to a class
beside the IR bug which would be nice to isolate in a smaller case and report it
i
ohh, right. It is a good day today. I learned something new. Thanks a lot for debugging this with me!
🙌 1
r
likewise, lets wait for @simon.vergauwen input and see if we can change this in arrow for the next version
i
I can open an ‘issue’ on Kotlin, and try and isolate it in a smaller case. This can take up to a week, unfortunately. So, if you do this before me, please let me know in this chat.
Copy code
inline fun <R, A> myEffect(crossinline f: suspend EffectScope<R>.() -> A): Effect<R, A> = object : Effect<R, A> { Copy pasting Arrow code here, overriding the methods. Implementing Token, Suspend and FoldContinuation internal classes }
What is surprising, is that this approach ☝️also works once compiled within my project.
s
Yes, we can change it to a class @raulraja. It's internal anyway so doesn't affect the user in any way.
i
@simon.vergauwen it is very strange. I have copy-pasted your current Effect implementation (anonymous class and inline included) into my own source code and used it under the test. It is also working for me. Take a look at 
myEffect
☝️
This works:
Copy code
fun <S, E> handleEither(event: E): Effect<Throwable, S> = myEffect {
    shift(RuntimeException("OUPS"))
}
This does not work:
Copy code
fun <S, E> handleEither(event: E): Effect<Throwable, S> = effect {
    shift(RuntimeException("OUPS"))
}
s
Oh, that is very weird 🤔
So just to verify. An object from Arrow fails, but an object from your own code works? 🤔
i
Yes
r
in a diff module or same?
i
I am using new hierarchical project structure (by default from 1.6.20). 
Effect
is being placed in the same module under 
comonMain
source set/compilation.
Copy code
@PublishedApi
internal class Token {
    override fun toString(): String = "Token(${hashCode().toString(16)})"
}
@PublishedApi
internal class Suspend(val token: Token, val shifted: Any?, val recover: suspend (Any?) -> Any?) : ShiftCancellationException() {
    override fun toString(): String = "ShiftCancellationException($message)"
}

@PublishedApi
internal class FoldContinuation<B>(
    private val token: Token,
    override val context: CoroutineContext,
    private val parent: Continuation<B>
) : Continuation<B> {
    override fun resumeWith(result: Result<B>) {
        result.fold(parent::resume) { throwable ->
            if (throwable is Suspend && token == throwable.token) {
                val f: suspend () -> B = { throwable.recover(throwable.shifted) as B }
                when (val res = f.startCoroutineUninterceptedOrReturn(parent)) {
                    COROUTINE_SUSPENDED -> Unit
                    else -> parent.resume(res as B)
                }
            } else parent.resumeWith(result)
        }
    }
}

sealed class ShiftCancellationException : CancellationException("Shifted Continuation")
inline fun <R, A> myEffect(crossinline f: suspend EffectScope<R>.() -> A): Effect<R, A> = object : Effect<R, A> {

    override suspend fun <B> fold(recover: suspend (R) -> B, transform: suspend (A) -> B): B =
        suspendCoroutineUninterceptedOrReturn { cont ->
            val token = Token()
            val effectScope =
                object : EffectScope<R> {
                    override suspend fun <B> shift(r: R): B =
                        throw Suspend(token, r, recover as suspend (Any?) -> Any?)
                }

            try {
                suspend { transform(f(effectScope)) }
                    .startCoroutineUninterceptedOrReturn(FoldContinuation(token, cont.context, cont))
            } catch (e: Suspend) {
                if (token == e.token) {
                    val f: suspend () -> B = { e.recover(e.shifted) as B }
                    f.startCoroutineUninterceptedOrReturn(cont)
                } else throw e
            }
        }
}

fun <S, E> handleEither(event: E): Effect<Throwable, S> = myEffect {
    shift(RuntimeException("OUPS"))
}
r
try diff module, it may be an issue on compiled jars only
i
I also used @PublishedApi and internal for Token, Suspend… to exclude this as potential issue.
🤔 1
I will try to put it in another module now.
Raul, it works if I put my Effect implementation in another module 
commonMain
I have no clue. Could it be some multiplatform hierarchical structure non-compatibility issue?
r
no idea LOL XD
s
🤯
i
haha, right!
no pressure, I will find it. Just, the multiplatform is still in the adoption phase, for me. Thanks again for your time.
👍 1
@raulraja @simon.vergauwen The issue is fixed now. The 
build
is green (https://github.com/fraktalio/fmodel/actions/runs/2436185398) and I am going to continue with adopting 
Effect
. You can find more here https://youtrack.jetbrains.com/issue/KT-52540/Internal-error-in-body-lowering-kotlinNotImplementedError (in case someone else runs into a similar issue).
👍 1
🙌 1
22 Views
Open in Slack
PreviousNext
Powered by