Hm. still the same error. Both with the V1 handler...

# http4k

[snowe]

Hm. still the same error. Both with the V1 handler and the routes:

"java.lang.IllegalStateException: method is invalid\n\tat org.http4k.serverless.ApiGatewayV1AwsHttpAdapter.toHttp4kRequest(ApiGatewayV1.kt:54)\n\tat org.http4k.serverless.ApiGatewayV1AwsHttpAdapter.invoke(ApiGatewayV1.kt:61)\n\tat org.http4k.serverless.ApiGatewayV1AwsHttpAdapter.invoke(ApiGatewayV1.kt:51)\n\tat org.http4k.serverless.ApiGatewayFnLoader.invoke$lambda$0(ApiGatewayFnLoader.kt:24)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:44)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:40)\n\tat kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)\n"

val http4kApp = routes(
    "/echo/{message:.*}" bind GET to {
        Response(OK).body(
            it.path("message") ?: "(nothing to echo, use /echo/<message>)"
        )
    },
    "/" bind GET to { Response(OK).body("ok") }
)
fun main() {
    ApiGatewayV1FnLoader(http4kApp).asServer(AwsLambdaRuntime()).start()
}

[s4nchez]

I will not be setting up my lambda with routes through apigateway, it will just be direct lambda calls

How are you invoking the lambda?

By adding an ApiGateway adapter, the function will expect a JSON payload with the request details and will translate that into an http4k Request. If you're invoking the function outside the ApiGateway, you have to produce the payload yourself

[s4nchez]

An alternative way to skip ApiGateway is to expose function as a Lambda Function URL (example here)

[snowe]

you can invoke lambda functions directly through aws sdk without going through apigateway or lambda function urls. That’s currently how our entire stack works, we’re using kotlin graalvm quarkus lambdas that all execute just from sdk calls, no gateway.

[s4nchez]

That's fine, but in that case, you need to build the same payload as ApiGateway would generate if you want the http4k lambda to use it as an http request

[snowe]

I don’t really care about using it as an http request. I just want it deserialized as an object. For example, testing the lambda through the AWS Console we shouldn’t have to build out the entire payload with the underlying aws data, as it’s already passed in that form.

[snowe]

quarkus’s version of this looks like this

@Named("test")
public class TestLambda implements RequestHandler<InputObject, OutputObject> {
}

@Named("stream")
public class StreamLambda implements RequestStreamHandler {
}

where RequestHandler automatically deserializes using jackson (🤮 ) and RequestStreamHandler just hands you the object stream for you to deserialize yourself.

[snowe]

trying to do

val http4kApp = routes(
    "/echo/{message:.*}" bind GET to {
        Response(OK).body(
            it.path("message") ?: "(nothing to echo, use /echo/<message>)"
        )
    },
    "/external" bind GET to {
        Response(OK).body(JavaHttpClient()(Request(GET, "<http://httpbin.org/get>")).bodyString())
    },
    "/" bind GET to { Response(OK).body("Hello from Lambda URL!").header("content-type", "text/html; charset=utf-8") }
)

@Suppress("unused")
class HelloServerlessHttp4k : ApiGatewayV2LambdaFunction(http4kApp)

gives me

Error: Main method not found in class com.sunrun.pricing.lambda.HelloServerlessHttp4k, please define the main method as:
public static void main(String[] args)
or a JavaFX application class must extend javafx.application.Application
INIT_REPORT Init Duration: 470.93 ms	Phase: init	Status: error	Error Type: Runtime.ExitError

I actually did expect this one to work so not sure why it’s not.

[dave]

The reason that doesn't work is because you need to wrap the adapter inside the AWS runtime "main" which you were doing before.

[dave]

There are other adapters which give you the various fornats here .

[dave]

It may help you to check out the Aws custom runtime that we built - I suspect the difference is that quarkus is doing that wrapping for you under the covers (and hence Jackson etc because it's wiring at compile time): https://github.com/http4k/http4k/tree/master/http4k-serverless/lambda-runtime/src/main/kotlin/org/http4k/serverless )

[dave]

Http4k uses Moshi under the covers and you can also combine that with kotshi to avoid any reflection at all but also get generated adapters

[Andrew O'Hara]

If you're only ever going to invoke your function with the lambda invoke operation, then is it really necessary to run a web application in your function?

Why not use the quarkus handlers, define your own input and output DTO, and treat it like RPC? If you don't want to be using Quarkus's Jackson to perform the (de)serialization for you, then you can always hook the InputStream and OutputStream into your JSON marshaller.

[snowe]

It may help you to check out the Aws custom runtime that we built

I forgot to update this yesterday but I did do that. This does pretty much exactly what I want:

object TestLambdaAdapter : AwsHttpAdapter<Map<String, Any>, Map<String, Any>> {
    private fun Map<String, Any>.toHttp4kRequest(): Request {
        return Request(POST, Uri.of("")).body(toBody())
    }

    override fun invoke(req: Map<String, Any>, ctx: Context): Request = req.toHttp4kRequest()

    override fun invoke(resp: Response): Map<String, Any> {
        val nonCookies = resp.headers.filterNot { it.first.lowercase(Locale.getDefault()) == "set-cookie" }
        return Json.parseToJsonElement(resp.body.payload.asString()).jsonObject.toMap()
    }
}

Http4k uses Moshi under the covers and you can also combine that with kotshi to avoid any reflection at all but also get generated adapters

interesting. I’ll have to look at that.

Why not use the quarkus handlers, define your own input and output DTO, and treat it like RPC? If you don’t want to be using Quarkus’s Jackson to perform the (de)serialization for you, then you can always hook the InputStream and OutputStream into your JSON marshaller.

I’m trying to see if http4k is faster than quarkus due to having a smaller final function size. Smaller package should result in better cold start times.

[snowe]

with graalvm native of course

[Andrew O'Hara]

I’m trying to see if http4k is faster than quarkus due to having a smaller final function size. Smaller package should result in better cold start times.

Ah, I see! So this still doesn't mean you need to use Http4k as a web application. See how to use http4k-serverless as an event function. You're still taking advantage of the lighter environment, but don't need to build a whole web application to do it. I don't see an example that returns a synchronous response, but you should be able to deduce how to accomplish it.

[snowe]

@Andrew O'Hara gotcha. that’s originally close to how I was trying to do it, I think? https://kotlinlang.slack.com/archives/C5AL3AKUY/p1701813441310939

[snowe]

I’ll take a look at that and see what I can get running.

[dave]

Yes, you can do that with http4k as well. We do have the concept of a fn handler which deals just with the streams 😉 . All the rest are built on top of that.

[snowe]

Hm. So I’ve been trying to get this working, still unsuccessfully. The examples in the tutorials state to use

// This class is the entry-point for the Lambda function call - configure it when deploying
class EventFunction : AwsLambdaEventFunction(EventFnLoader(JavaHttpClient()))

as the entrypoint, but doing so results in errors stating I’m missing a main method. Not doing so results in only the main method being run (and subsequently blowing up, not sure why).

[dave]

The examples are not using Graal

[dave]

When you are using graal you need to provide a runtime which gives you the main

[dave]

That's the point of the custom runtime

[dave]

The function wrappers like this: AwsLambdaEventFunction

[snowe]

hm. thought you could still provide the class, like quarkus.

[dave]

no magic please!

[dave]

🙂

[snowe]

so it still must always call main()?

[dave]

package guide.tutorials.going_native_with_graal_on_aws_lambda

import org.http4k.core.Method.GET
import org.http4k.core.Response
import org.http4k.core.Status.Companion.OK
import org.http4k.routing.bind
import org.http4k.routing.path
import org.http4k.routing.routes
import org.http4k.serverless.ApiGatewayV2FnLoader
import org.http4k.serverless.AwsLambdaRuntime
import org.http4k.serverless.asServer

val http4kApp = routes(
    "/echo/{message:.*}" bind GET to {
        Response(OK).body(
            it.path("message") ?: "(nothing to echo, use /echo/<message>)"
        )
    },
    "/" bind GET to { Response(OK).body("ok") }
)

fun main() {
    ApiGatewayV2FnLoader(http4kApp).asServer(AwsLambdaRuntime()).start()
}

[dave]

yes - the docker container is JUST calling the main function

[snowe]

then where does the inputStream come from for the handleRequest method?

[dave]

Look at AwsLambdaRuntime

[dave]

thread {
                do {
                    runtime.nextInvocation().use { nextInvocation ->
                        try {
                            lambda(nextInvocation.body.stream, LambdaEnvironmentContext(nextInvocation, env)).use {
                                runtime.success(nextInvocation, it)
                            }
                        } catch (e: Exception) {
                            runtime.error(nextInvocation, e)
                        }
                    }
                } while (!done.get())
            }

[dave]

This code:

lambda(nextInvocation.body.stream, LambdaEnvironmentContext(nextInvocation, env))

[dave]

is the call to the lambda function which passes the stream loaded from the Lambda HTTP endpoint

[snowe]

i see.

[snowe]

so it is still starting up a webserver under the hood

[dave]

nope

[dave]

the main calls out to an HTTP endpoint to load the next lambda call

[dave]

and then passes that to the function (in memory)

[dave]

it's just a big loop

[snowe]

oh strange.

[dave]

not really if you think about it

[dave]

Lambda isnt' magic

[dave]

it needs to get the data from somewhere:

[dave]

1. it starts up

[dave]

2. it starts a loop

[dave]

3. inside the loop, it calls the HTTP endpoint to get the next call, passes it to the lambda and then posts the result back to another HTTP endpoint

[dave]

until AWS kills it

[snowe]

I see. That way it doesn’t have to do cold starts every time, which it would if it worked the way I thought it did.

[snowe]

good to know.

[dave]

trust me - you're overthinking it! 😉

[dave]

and the lambda runtime main is just a docker container which contains java or whatever is used to run the lambda

[dave]

I think you want to use: InvocationFnLoader

[snowe]

so I’m at the point where I’m able to run in aws, but it’s throwing errors about exiting without a reason. I’m guessing my main is wrong, but not sure why.

INIT_REPORT Init Duration: 2871.69 ms	Phase: invoke	Status: error	Error Type: Runtime.ExitError
START RequestId: 6cd6ba0a-8de3-4b36-ae17-4dcc9d4eb848 Version: $LATEST
RequestId: 6cd6ba0a-8de3-4b36-ae17-4dcc9d4eb848 Error: Runtime exited without providing a reason
Runtime.ExitError

[snowe]

inline fun <reified In : Any, Out : Any> FnLoader(
    autoMarshalling: AutoMarshalling = AwsLambdaMoshi,
    noinline makeHandler: (Map<String, String>) -> FnHandler<In, Context, Out>
) = AutoMarshallingFnLoader(autoMarshalling, In::class, makeHandler)

// This is the handler for the incoming AWS SQS event. It's just a function so you can call it without any infrastructure
fun EventFnHandler(http: HttpHandler) =
    FnHandler { e: String, _: Context ->
        println("In the FnHandler EventFnHandler")
        "processed ${e} messages"
    }

// We can add filters to the FnHandler if we want to - in this case print the transaction (with the letency).
val loggingFunction = ReportFnTransaction<String, Context, String> { tx ->
    println(tx)
}

// The FnLoader is responsible for constructing the handler and for handling the serialisation of the request and response
fun EventFnLoader(http: HttpHandler) = FnLoader { env: Map<String, String> ->
    loggingFunction.then(EventFnHandler(http))
}

// This class is the entry-point for the Lambda function call - configure it when deploying
class EventFunction : AwsLambdaEventFunction(EventFnLoader(JavaHttpClient()))

fun main() {
    // this server receives the reversed event
    val receivingServer = { req: Request ->
        println(req.bodyString())
        Response(OK)
    }.asServer(SunHttp(8080)).start()
    println("i'm here. dfdlkjdf")

    fun runLambdaInMemoryOrForTesting() {
        println("RUNNING In memory:")
        val app = EventFnHandler(JavaHttpClient())
        app("testing?", mock())
    }

    fun runLambdaAsAwsWould() {
        println("RUNNING as AWS would invoke the function:")

        val out = ByteArrayOutputStream()
        EventFunction().handleRequest(AwsLambdaMoshi.asInputStream("""{"event": "value"}"""), out, mock())
        // the response is empty b
        println(out.toString())
    }

    runLambdaInMemoryOrForTesting()
    println("I'm here. ")
    runLambdaAsAwsWould()
    println("run lambda as aws would?")

    receivingServer.stop()
}

[snowe]

I customized Awslambdamoshi to be

object AwsLambdaMoshi : ConfigurableMoshi(
    Moshi.Builder()
        .addLast(MapAdapter)
        .addLast(ThrowableAdapter)
        .addLast(ListAdapter)
        .addLast(EventAdapter)
        .asConfigurable(MetadataKotlinJsonAdapterFactory())
        .withStandardMappings()
        .done()
)

[snowe]

oh, let me look into InvocationFnLoader

[dave]

fun MyLoader() = FnLoader {
    FnHandler { i: InputStream, c: Context ->
        "helloworld".byteInputStream()
    }
}

fun main() {
    MyLoader().asServer(AwsLambdaRuntime())
}

[dave]

That is the most basic thing to give you your inputstream

[dave]

you can use Moshi to convert the input stream to your object (via JSON)

[snowe]

Ok, thank you very much. I was almost there with EventFnLoader, but didn’t realize I could call asServer on that. And now AwsLambdaRuntime makes more sense. Thanks for all the help!

[snowe]

Hm. looks like my function still isn’t getting called.

Http4k - Starting Lambda Runtime
INIT_REPORT Init Duration: 203.51 ms	Phase: init	Status: error	Error Type: Runtime.ExitError
Http4k - Starting Lambda Runtime
INIT_REPORT Init Duration: 2456.91 ms	Phase: invoke	Status: error	Error Type: Runtime.ExitError
START RequestId: 2119a44d-41ff-4293-a6fd-8d4020a0434b Version: $LATEST
RequestId: 2119a44d-41ff-4293-a6fd-8d4020a0434b Error: Runtime exited without providing a reason
Runtime.ExitError
END RequestId: 2119a44d-41ff-4293-a6fd-8d4020a0434b
REPORT RequestId: 2119a44d-41ff-4293-a6fd-8d4020a0434b	Duration: 2477.70 ms	Billed Duration: 2478 ms	Memory Size: 128 MB	Max Memory Used: 31 MB	
XRAY TraceId: 1-65777453-773dc56472227f34768bb21d	SegmentId: 0cf810c11e5ead88	Sampled: true

enum class Color{
    black,grey,white,brown,splotchy;
}
data class MoshiCat(val color: Color, val name: String)
fun MyLoader() = FnLoader {
    FnHandler { i: InputStream, c: Context ->
        println("Http4k - Running the function")
        try {
            val out = MoshiXLite.asA<MoshiCat>(i)
                .also(::println)
        } catch (e: Exception){
            println(e)
        }
    }
}
fun main() {
    println("Http4k - Starting Lambda Runtime")
    MyLoader().asServer(AwsLambdaRuntime())
}

[dave]

need to call start()

[dave]

sorry - my bad

[snowe]

nah that’s my bad. I am not remembering what I read in the documentation 😭

[snowe]

ty!

[dave]

for giggles you can also do

fun main() {
    MyLoader().asServer(AwsLambdaRuntime(http = JavaHttpClient().debug()))
}

[dave]

then you'll see the calls to the lambda service to get and post the lambda function call

[snowe]

oh I’ll definitely do that. thank you!

[snowe]

Now I’m getting

"java.lang.IllegalArgumentException: Platform class java.io.InputStream requires explicit JsonAdapter to be registered\n\tat com.squareup.moshi.ClassJsonAdapter$1.create(ClassJsonAdapter.java:76)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:146)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:106)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:80)\n\tat org.http4k.format.ConfigurableMoshi.adapterFor(ConfigurableMoshi.kt:93)\n\tat org.http4k.format.ConfigurableMoshi.asA(ConfigurableMoshi.kt:97)\n\tat org.http4k.serverless.AutoMarshallingFnLoader.invoke$lambda$0(AutoMarshallingFnLoader.kt:19)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:44)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:40)\n\tat kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)\n\tat

indicating that I don’t have an adaptor for this, but I would expect that to be built in somewhere. I also tried just marshalling straight to an object but that results in a different error message, indicating I should have set up KotlinJsonAdapterFactory even though I have done so.

"java.lang.IllegalArgumentException: Cannot serialize Kotlin type com.sunrun.pricing.lambda.MoshiCat. Reflective serialization of Kotlin classes without using kotlin-reflect has undefined and unexpected behavior. Please use KotlinJsonAdapterFactory from the moshi-kotlin artifact or use code gen from the moshi-kotlin-codegen artifact.\n\tat com.squareup.moshi.ClassJsonAdapter$1.create(ClassJsonAdapter.java:98)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:146)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:106)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:80)\n\tat org.http4k.format.ConfigurableMoshi.adapterFor(ConfigurableMoshi.kt:93)\n\tat org.http4k.format.ConfigurableMoshi.asA(ConfigurableMoshi.kt:97)\n\tat org.http4k.serverless.AutoMarshallingFnLoader.invoke$lambda$0(AutoMarshallingFnLoader.kt:19)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:44)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:40)\n\tat kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)\n\tat

[dave]

you can't just pass an inputstream to Moshi

override fun <T : Any> asA(input: InputStream, target: KClass<T>): T = adapterFor(target).fromJson(
        input.source().buffer()
    )!!

[dave]

Moshi.asA(input, MyThang::class)

[snowe]

sorry, wasn’t clear there. I’ve commented out any deserialization I’m doing and the errors are coming from internal to http4k.

fun MyLoader() = MoshiFnLoader {
    FnHandler { i: MoshiCat, c: Context ->
        println("Http4k - Running the function")
        "asldkfj"
    }
}

[snowe]

or InputStream instead of MoshiCat

[snowe]

both throw errors. so it never even makes it to my deserialization.

[dave]

ah - you haven't got reflection at all

[dave]

you need to generate adapters manually

[dave]

(suggest kotshi)

[snowe]

oh is it using reflection to retrieve the adapters?

[snowe]

oh yeah kotshi. I went back and looked for that in the docs and couldn’t find it, went with moshix.

[dave]

Moshi falls back to using reflection if it can't find a registered adapter

[snowe]

ah. I could specify the adapter in the reflection config if I knew what it was called. let me go try that.

[snowe]

and then try kotshi as well.

[dave]

I'd really try kotshi - it's pretty easy.

[dave]

you just annotate with @JsonSerializable and create a factory then add it to your config

[dave]

stay away from reflect-config.json

[dave]

(IMHO)

[snowe]

alright I’ll try that. though I wouldn’t be able to add it to inputstream right? I’d need to serialize using the automarshalling

[dave]

well you'd need to use that code above

[dave]

http4k-connect makes heavy use of kotshi to avoid reflection: https://github.com/http4k/http4k-connect/blob/master/amazon/eventbridge/client/src[…]otlin/org/http4k/connect/amazon/eventbridge/EventBridgeMoshi.kt

[dave]

1. annotate your class
2. run ksp(<kotshi>) in your gradle
3. Add the generated factory to your Moshi instance

[snowe]

👍🏽

[snowe]

image.png

[snowe]

thank you for all the help ❤️

[dave]

🎉

[dave]

I think we're both glad you've got it working 😉

[snowe]

haha yeah, I bothered you quite a lot.

[dave]

Np. It's what we're here for 🙃 .

[snowe]

Lenses don’t use reflection right?

[dave]

The body lenses simply use whatever the underlying tech does - so if you've got Moshi with kotshi then no they don't.

[dave]

None of the query/header etc lenses use reflection

[snowe]

yeah that’s what I thought from looking through the code.