are there any docs or examples of the IR transform...

# compiler

[toadzky]

are there any docs or examples of the IR transform process for converting a suspend function into a regular function that launches a coroutine? i'm trying to build a plugin to let a java framework that doesn't know about coroutines invoke suspend functions auto-magically without the developer having to deal with the disconnect, but i'm not really understanding why i'm getting weird issues. i'm hoping there's something that can point me in the right direction

[dmitriy.novozhilov]

You can check the implementation here and here

[toadzky]

i'm still fairly new at the compiler level stuff, but it seems like the k2 plugin backend IR generation would create essentially different kotlin code and then the lowering stuff would create bytecode? or is lowering just de-sugaring

[dmitriy.novozhilov]

Lowerings are passes which take the more high-level IR and transform to more low-level
IR plugins are applied before any lowerings, so you effectively generate plain Kotlin code but in form of IR tree instead of sources
Lowerings will be applied to the generated code together with the original one

[toadzky]

thanks for the explanation

[toadzky]

i'm doing backend IR stuff and i'm not able to generate functional kotlin code as IR - it looks right but then it throws errors when it tries to run

[udalov]

The main part of coroutine transformations for JVM is happening on the bytecode level in codegen, not on IR level. There's a doc here: https://github.com/JetBrains/kotlin/blob/d039714146350a1fd7b8d6d4756d62ffad89e1e2/[…]c/org/jetbrains/kotlin/codegen/coroutines/coroutines-codegen.md

[toadzky]

i'm trying to use the IR to write a coroutine launch block so the rest of the compiler process will do the bytecode generation and transforms

[udalov]

Lowering them on the IR level (and I assume, doing something similar from plugin code) would be impossible because of inlining, which also happens on the bytecode level in the codegen.

[toadzky]

maybe some specifics will help people understand what i'm trying to do. i like use jax-rs for api backends. it's standard, has multiple implementations, and is mature. however, it's not compatible with coroutines without each endpoint having to launch a coroutine - either as a runBlocking or using the jax-rs AsyncResponse pattern. i'm trying to write a compiler plugin that will automatically transform annotated suspend jax-rs methods to be non-suspend methods that have an additional parameter and wraps the entire function body contents with a coroutine async block.

[toadzky]

i'm able to do the first half: remove the suspend keyword and add the new parameter. what i'm having trouble with is correctly creating the async launch with the trailing lambda block that contains all the statements from the original function

[udalov]

Yeah, lowering suspend function to a non-suspend will involve building the state machine and all related stuff, including tail call optimization (maybe you don't need it as much though). But you will not be able to reuse anything from the JVM compiler pipeline for that, since that transformation for JVM happens after IR has already been compiled to JVM bytecode.

However, you can take a look at the suspend lowering code in JS for example, which happens on IR, but there might be some JS specifics in there which I don't know about (AbstractSuspendFunctionsLowering.kt, JsSuspendFunctionsLowering.kt).

[toadzky]

why would it involve all that? don't the backend IR plugins get applied before all that happens?

[toadzky]

if i can transform the function to just look like a non-suspend function that invokes a coroutine block, wouldn't the rest of the compiler pipeline treat it like any developer written function of the same structure?

[udalov]

Yes, but how exactly are you planning to launch that coroutine block? Maybe you can show a source code transformation example?

[toadzky]

CoroutineScope(Dispatchers.Default).async

[toadzky]

that's what i'm trying to generate, and then have a trailing lambda block containing the original function body

[toadzky]

so something like

@GET
suspend fun myRouteHandler(): MyResponseType {
  // handle the rest request
}

becomes

@GET
fun myRouteHandler(response: AsyncResponse): Unit {
  CoroutineScope(Dispatchers.Default).async {
    // handle the rest request
    // take the original response and pipe it into the `AsyncResponse`
  }
}

[udalov]

Yeah, I thought you wanted some kind of a generic solution, but for this you basically need to add calls to a few functions. Which issues are you getting?

[toadzky]

i'm at my work laptop at the moment, so i can't copy the errors, but i'm struggling to add the trailing lambda to the async call and have the original statements inside it

[toadzky]

i can get those errors during my lunch break in like an hour

[toadzky]

i was hoping there was some kind of example or docs on how to do it - it's confusing whether it should be an IrBlockBody or an anonymous function or a few other things

[udalov]

My main practical advice would be to write this source code (calling async) by yourself, dump IR via -Xphases-to-dump-before=IrLowering, and then do the same with your plugin's code, and compare them.

[toadzky]

ooo - didn't know i could do that

[toadzky]

thanks - i'll give that a shot and see if that can get me what i need

[toadzky]

that has been a great suggestion and it seems like i'm making much better progress because of it - or at least i'm understanding what i'm doing a lot more (which is a low bar to clear). one thing i'm having trouble figuring out... where does this line come from and how would i generate it?

TYPE_OP type=kotlin.Unit origin=IMPLICIT_COERCION_TO_UNIT typeOperand=kotlin.Unit

[toadzky]

the other one that's stumping me is this one:

block: FUN_EXPR type=@[ExtensionFunctionType] kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope, kotlin.Unit> origin=LAMBDA

i've managed to get to look like this:

block: FUN_EXPR type=kotlin.coroutines.SuspendFunction0<kotlin.Unit> origin=LAMBDA

but that's as close as i've managed to come

[toadzky]

however, those 2 things don't appear to actually be blocking the functionality, i finally got a successful run of my test app 🎉

[udalov]

TYPE_OP type=kotlin.Unit origin=IMPLICIT_COERCION_TO_UNIT typeOperand=kotlin.Unit

It's an IrTypeOperatorCall with IrTypeOperator.IMPLICIT_COERCION_TO_UNIT

[udalov]

Presence of @[ExtensionFunctionType] is not important, however it seems weird that SuspendFunction0 works for you where SuspendFunction1 is needed

[toadzky]

i'll double check my IR output and see if it's still SuspendFunction0 or if i switched it. i probably switched it because you are right, it shouldn't work with a 0 arity

[toadzky]

yep - it was changed to SuspendFunction1 to make it work.

[toadzky]

like i mentioned, i got the basic form working, but right now i'm stumped because my IR looks basically the same as the IR generated by the regular compiler, but i'm getting an error i don't get on the compiler. here's the diff in the IR (official is left, mine is right)

[toadzky]

the error i'm getting when i run the app using my generated IR is

! java.lang.NullPointerException: null
! at kotlin.coroutines.jvm.internal.ContinuationImpl.getContext(ContinuationImpl.kt:105)
! at kotlin.coroutines.jvm.internal.ContinuationImpl.intercepted(ContinuationImpl.kt:112)
! at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.intercepted(IntrinsicsJvm.kt:182)
! at kotlinx.coroutines.DelayKt.delay(Delay.kt:172)
! at dev.skytag.jaxrs.suspend.test.Api.doTheThing(Api.kt:31)

[toadzky]

the handler function being rewritten is

@GET
suspend fun doTheThing(): String {
    delay(50) // this is line 31 in the source
    return getResponse()
}

[toadzky]

the function i'm trying to turn it into look like this and works fine

@GET
    fun doTheThing(@Suspended asyncResponse: AsyncResponse) {
        CoroutineScope(Dispatchers.Default).launch {
            try {
                delay(50)
                asyncResponse.resume(getResponse())
            } catch (ex: Exception) {
                asyncResponse.resume(ex)
            }
        }
    }