https://kotlinlang.org logo
#http4k
Title
# http4k
s

snowe

12/05/2023, 10:22 PM
Hm. still the same error. Both with the V1 handler and the routes:
Copy code
"java.lang.IllegalStateException: method is invalid\n\tat org.http4k.serverless.ApiGatewayV1AwsHttpAdapter.toHttp4kRequest(ApiGatewayV1.kt:54)\n\tat org.http4k.serverless.ApiGatewayV1AwsHttpAdapter.invoke(ApiGatewayV1.kt:61)\n\tat org.http4k.serverless.ApiGatewayV1AwsHttpAdapter.invoke(ApiGatewayV1.kt:51)\n\tat org.http4k.serverless.ApiGatewayFnLoader.invoke$lambda$0(ApiGatewayFnLoader.kt:24)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:44)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:40)\n\tat kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)\n"
Copy code
val http4kApp = routes(
    "/echo/{message:.*}" bind GET to {
        Response(OK).body(
            it.path("message") ?: "(nothing to echo, use /echo/<message>)"
        )
    },
    "/" bind GET to { Response(OK).body("ok") }
)
fun main() {
    ApiGatewayV1FnLoader(http4kApp).asServer(AwsLambdaRuntime()).start()
}
s

s4nchez

12/05/2023, 10:31 PM
I will not be setting up my lambda with routes through apigateway, it will just be direct lambda calls
How are you invoking the lambda? By adding an ApiGateway adapter, the function will expect a JSON payload with the request details and will translate that into an http4k Request. If you're invoking the function outside the ApiGateway, you have to produce the payload yourself
An alternative way to skip ApiGateway is to expose function as a Lambda Function URL (example here)
s

snowe

12/05/2023, 10:43 PM
you can invoke lambda functions directly through aws sdk without going through apigateway or lambda function urls. That’s currently how our entire stack works, we’re using kotlin graalvm quarkus lambdas that all execute just from sdk calls, no gateway.
s

s4nchez

12/05/2023, 10:54 PM
That's fine, but in that case, you need to build the same payload as ApiGateway would generate if you want the http4k lambda to use it as an http request
s

snowe

12/05/2023, 10:56 PM
I don’t really care about using it as an http request. I just want it deserialized as an object. For example, testing the lambda through the AWS Console we shouldn’t have to build out the entire payload with the underlying aws data, as it’s already passed in that form.
quarkus’s version of this looks like this
Copy code
@Named("test")
public class TestLambda implements RequestHandler<InputObject, OutputObject> {
}

@Named("stream")
public class StreamLambda implements RequestStreamHandler {
}
where
RequestHandler
automatically deserializes using jackson (🤮 ) and
RequestStreamHandler
just hands you the object stream for you to deserialize yourself.
trying to do
Copy code
val http4kApp = routes(
    "/echo/{message:.*}" bind GET to {
        Response(OK).body(
            it.path("message") ?: "(nothing to echo, use /echo/<message>)"
        )
    },
    "/external" bind GET to {
        Response(OK).body(JavaHttpClient()(Request(GET, "<http://httpbin.org/get>")).bodyString())
    },
    "/" bind GET to { Response(OK).body("Hello from Lambda URL!").header("content-type", "text/html; charset=utf-8") }
)

@Suppress("unused")
class HelloServerlessHttp4k : ApiGatewayV2LambdaFunction(http4kApp)
gives me
Copy code
Error: Main method not found in class com.sunrun.pricing.lambda.HelloServerlessHttp4k, please define the main method as:
public static void main(String[] args)
or a JavaFX application class must extend javafx.application.Application
INIT_REPORT Init Duration: 470.93 ms	Phase: init	Status: error	Error Type: Runtime.ExitError
I actually did expect this one to work so not sure why it’s not.
d

dave

12/06/2023, 4:29 AM
The reason that doesn't work is because you need to wrap the adapter inside the AWS runtime "main" which you were doing before.
There are other adapters which give you the various fornats here .
It may help you to check out the Aws custom runtime that we built - I suspect the difference is that quarkus is doing that wrapping for you under the covers (and hence Jackson etc because it's wiring at compile time): https://github.com/http4k/http4k/tree/master/http4k-serverless/lambda-runtime/src/main/kotlin/org/http4k/serverless )
Http4k uses Moshi under the covers and you can also combine that with kotshi to avoid any reflection at all but also get generated adapters
a

Andrew O'Hara

12/06/2023, 4:33 PM
If you're only ever going to invoke your function with the lambda
invoke
operation, then is it really necessary to run a web application in your function? Why not use the quarkus handlers, define your own input and output DTO, and treat it like RPC? If you don't want to be using Quarkus's Jackson to perform the (de)serialization for you, then you can always hook the InputStream and OutputStream into your JSON marshaller.
s

snowe

12/06/2023, 5:33 PM
It may help you to check out the Aws custom runtime that we built
I forgot to update this yesterday but I did do that. This does pretty much exactly what I want:
Copy code
object TestLambdaAdapter : AwsHttpAdapter<Map<String, Any>, Map<String, Any>> {
    private fun Map<String, Any>.toHttp4kRequest(): Request {
        return Request(POST, Uri.of("")).body(toBody())
    }

    override fun invoke(req: Map<String, Any>, ctx: Context): Request = req.toHttp4kRequest()

    override fun invoke(resp: Response): Map<String, Any> {
        val nonCookies = resp.headers.filterNot { it.first.lowercase(Locale.getDefault()) == "set-cookie" }
        return Json.parseToJsonElement(resp.body.payload.asString()).jsonObject.toMap()
    }
}
Http4k uses Moshi under the covers and you can also combine that with kotshi to avoid any reflection at all but also get generated adapters
interesting. I’ll have to look at that.
Why not use the quarkus handlers, define your own input and output DTO, and treat it like RPC? If you don’t want to be using Quarkus’s Jackson to perform the (de)serialization for you, then you can always hook the InputStream and OutputStream into your JSON marshaller.
I’m trying to see if http4k is faster than quarkus due to having a smaller final function size. Smaller package should result in better cold start times.
with graalvm native of course
a

Andrew O'Hara

12/06/2023, 7:57 PM
I’m trying to see if http4k is faster than quarkus due to having a smaller final function size. Smaller package should result in better cold start times.
Ah, I see! So this still doesn't mean you need to use Http4k as a web application. See how to use http4k-serverless as an event function. You're still taking advantage of the lighter environment, but don't need to build a whole web application to do it. I don't see an example that returns a synchronous response, but you should be able to deduce how to accomplish it.
s

snowe

12/06/2023, 7:58 PM
@Andrew O'Hara gotcha. that’s originally close to how I was trying to do it, I think? https://kotlinlang.slack.com/archives/C5AL3AKUY/p1701813441310939
I’ll take a look at that and see what I can get running.
d

dave

12/06/2023, 8:09 PM
Yes, you can do that with http4k as well. We do have the concept of a fn handler which deals just with the streams 😉 . All the rest are built on top of that.
s

snowe

12/11/2023, 7:50 PM
Hm. So I’ve been trying to get this working, still unsuccessfully. The examples in the tutorials state to use
Copy code
// This class is the entry-point for the Lambda function call - configure it when deploying
class EventFunction : AwsLambdaEventFunction(EventFnLoader(JavaHttpClient()))
as the entrypoint, but doing so results in errors stating I’m missing a main method. Not doing so results in only the main method being run (and subsequently blowing up, not sure why).
d

dave

12/11/2023, 7:51 PM
The examples are not using Graal
When you are using graal you need to provide a runtime which gives you the main
That's the point of the custom runtime
The function wrappers like this: AwsLambdaEventFunction
s

snowe

12/11/2023, 7:52 PM
hm. thought you could still provide the class, like quarkus.
d

dave

12/11/2023, 7:52 PM
no magic please!
😂 1
🙂
s

snowe

12/11/2023, 7:52 PM
so it still must always call
main()
?
d

dave

12/11/2023, 7:52 PM
Copy code
package guide.tutorials.going_native_with_graal_on_aws_lambda

import org.http4k.core.Method.GET
import org.http4k.core.Response
import org.http4k.core.Status.Companion.OK
import org.http4k.routing.bind
import org.http4k.routing.path
import org.http4k.routing.routes
import org.http4k.serverless.ApiGatewayV2FnLoader
import org.http4k.serverless.AwsLambdaRuntime
import org.http4k.serverless.asServer

val http4kApp = routes(
    "/echo/{message:.*}" bind GET to {
        Response(OK).body(
            it.path("message") ?: "(nothing to echo, use /echo/<message>)"
        )
    },
    "/" bind GET to { Response(OK).body("ok") }
)

fun main() {
    ApiGatewayV2FnLoader(http4kApp).asServer(AwsLambdaRuntime()).start()
}
yes - the docker container is JUST calling the main function
s

snowe

12/11/2023, 7:53 PM
then where does the inputStream come from for the handleRequest method?
d

dave

12/11/2023, 7:53 PM
Look at AwsLambdaRuntime
Copy code
thread {
                do {
                    runtime.nextInvocation().use { nextInvocation ->
                        try {
                            lambda(nextInvocation.body.stream, LambdaEnvironmentContext(nextInvocation, env)).use {
                                runtime.success(nextInvocation, it)
                            }
                        } catch (e: Exception) {
                            runtime.error(nextInvocation, e)
                        }
                    }
                } while (!done.get())
            }
This code:
Copy code
lambda(nextInvocation.body.stream, LambdaEnvironmentContext(nextInvocation, env))
is the call to the lambda function which passes the stream loaded from the Lambda HTTP endpoint
s

snowe

12/11/2023, 7:54 PM
i see.
so it is still starting up a webserver under the hood
d

dave

12/11/2023, 7:55 PM
nope
the main calls out to an HTTP endpoint to load the next lambda call
and then passes that to the function (in memory)
it's just a big loop
s

snowe

12/11/2023, 7:56 PM
oh strange.
d

dave

12/11/2023, 7:56 PM
not really if you think about it
Lambda isnt' magic
it needs to get the data from somewhere:
1. it starts up
2. it starts a loop
3. inside the loop, it calls the HTTP endpoint to get the next call, passes it to the lambda and then posts the result back to another HTTP endpoint
until AWS kills it
s

snowe

12/11/2023, 7:57 PM
I see. That way it doesn’t have to do cold starts every time, which it would if it worked the way I thought it did.
good to know.
d

dave

12/11/2023, 7:58 PM
trust me - you're overthinking it! 😉
and the lambda runtime main is just a docker container which contains java or whatever is used to run the lambda
I think you want to use: InvocationFnLoader
s

snowe

12/11/2023, 7:59 PM
so I’m at the point where I’m able to run in aws, but it’s throwing errors about exiting without a reason. I’m guessing my main is wrong, but not sure why.
Copy code
INIT_REPORT Init Duration: 2871.69 ms	Phase: invoke	Status: error	Error Type: Runtime.ExitError
START RequestId: 6cd6ba0a-8de3-4b36-ae17-4dcc9d4eb848 Version: $LATEST
RequestId: 6cd6ba0a-8de3-4b36-ae17-4dcc9d4eb848 Error: Runtime exited without providing a reason
Runtime.ExitError
Copy code
inline fun <reified In : Any, Out : Any> FnLoader(
    autoMarshalling: AutoMarshalling = AwsLambdaMoshi,
    noinline makeHandler: (Map<String, String>) -> FnHandler<In, Context, Out>
) = AutoMarshallingFnLoader(autoMarshalling, In::class, makeHandler)

// This is the handler for the incoming AWS SQS event. It's just a function so you can call it without any infrastructure
fun EventFnHandler(http: HttpHandler) =
    FnHandler { e: String, _: Context ->
        println("In the FnHandler EventFnHandler")
        "processed ${e} messages"
    }

// We can add filters to the FnHandler if we want to - in this case print the transaction (with the letency).
val loggingFunction = ReportFnTransaction<String, Context, String> { tx ->
    println(tx)
}

// The FnLoader is responsible for constructing the handler and for handling the serialisation of the request and response
fun EventFnLoader(http: HttpHandler) = FnLoader { env: Map<String, String> ->
    loggingFunction.then(EventFnHandler(http))
}

// This class is the entry-point for the Lambda function call - configure it when deploying
class EventFunction : AwsLambdaEventFunction(EventFnLoader(JavaHttpClient()))

fun main() {
    // this server receives the reversed event
    val receivingServer = { req: Request ->
        println(req.bodyString())
        Response(OK)
    }.asServer(SunHttp(8080)).start()
    println("i'm here. dfdlkjdf")

    fun runLambdaInMemoryOrForTesting() {
        println("RUNNING In memory:")
        val app = EventFnHandler(JavaHttpClient())
        app("testing?", mock())
    }

    fun runLambdaAsAwsWould() {
        println("RUNNING as AWS would invoke the function:")

        val out = ByteArrayOutputStream()
        EventFunction().handleRequest(AwsLambdaMoshi.asInputStream("""{"event": "value"}"""), out, mock())
        // the response is empty b
        println(out.toString())
    }

    runLambdaInMemoryOrForTesting()
    println("I'm here. ")
    runLambdaAsAwsWould()
    println("run lambda as aws would?")

    receivingServer.stop()
}
I customized Awslambdamoshi to be
Copy code
object AwsLambdaMoshi : ConfigurableMoshi(
    Moshi.Builder()
        .addLast(MapAdapter)
        .addLast(ThrowableAdapter)
        .addLast(ListAdapter)
        .addLast(EventAdapter)
        .asConfigurable(MetadataKotlinJsonAdapterFactory())
        .withStandardMappings()
        .done()
)
oh, let me look into InvocationFnLoader
d

dave

12/11/2023, 8:06 PM
Copy code
fun MyLoader() = FnLoader {
    FnHandler { i: InputStream, c: Context ->
        "helloworld".byteInputStream()
    }
}

fun main() {
    MyLoader().asServer(AwsLambdaRuntime())
}
That is the most basic thing to give you your inputstream
you can use Moshi to convert the input stream to your object (via JSON)
s

snowe

12/11/2023, 8:10 PM
Ok, thank you very much. I was almost there with EventFnLoader, but didn’t realize I could call
asServer
on that. And now
AwsLambdaRuntime
makes more sense. Thanks for all the help!
Hm. looks like my function still isn’t getting called.
Copy code
Http4k - Starting Lambda Runtime
INIT_REPORT Init Duration: 203.51 ms	Phase: init	Status: error	Error Type: Runtime.ExitError
Http4k - Starting Lambda Runtime
INIT_REPORT Init Duration: 2456.91 ms	Phase: invoke	Status: error	Error Type: Runtime.ExitError
START RequestId: 2119a44d-41ff-4293-a6fd-8d4020a0434b Version: $LATEST
RequestId: 2119a44d-41ff-4293-a6fd-8d4020a0434b Error: Runtime exited without providing a reason
Runtime.ExitError
END RequestId: 2119a44d-41ff-4293-a6fd-8d4020a0434b
REPORT RequestId: 2119a44d-41ff-4293-a6fd-8d4020a0434b	Duration: 2477.70 ms	Billed Duration: 2478 ms	Memory Size: 128 MB	Max Memory Used: 31 MB	
XRAY TraceId: 1-65777453-773dc56472227f34768bb21d	SegmentId: 0cf810c11e5ead88	Sampled: true
Copy code
enum class Color{
    black,grey,white,brown,splotchy;
}
data class MoshiCat(val color: Color, val name: String)
fun MyLoader() = FnLoader {
    FnHandler { i: InputStream, c: Context ->
        println("Http4k - Running the function")
        try {
            val out = MoshiXLite.asA<MoshiCat>(i)
                .also(::println)
        } catch (e: Exception){
            println(e)
        }
    }
}
fun main() {
    println("Http4k - Starting Lambda Runtime")
    MyLoader().asServer(AwsLambdaRuntime())
}
d

dave

12/11/2023, 8:45 PM
need to call start()
sorry - my bad
s

snowe

12/11/2023, 8:46 PM
nah that’s my bad. I am not remembering what I read in the documentation 😭
ty!
d

dave

12/11/2023, 8:46 PM
for giggles you can also do
Copy code
fun main() {
    MyLoader().asServer(AwsLambdaRuntime(http = JavaHttpClient().debug()))
}
then you'll see the calls to the lambda service to get and post the lambda function call
s

snowe

12/11/2023, 8:48 PM
oh I’ll definitely do that. thank you!
Now I’m getting
Copy code
"java.lang.IllegalArgumentException: Platform class java.io.InputStream requires explicit JsonAdapter to be registered\n\tat com.squareup.moshi.ClassJsonAdapter$1.create(ClassJsonAdapter.java:76)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:146)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:106)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:80)\n\tat org.http4k.format.ConfigurableMoshi.adapterFor(ConfigurableMoshi.kt:93)\n\tat org.http4k.format.ConfigurableMoshi.asA(ConfigurableMoshi.kt:97)\n\tat org.http4k.serverless.AutoMarshallingFnLoader.invoke$lambda$0(AutoMarshallingFnLoader.kt:19)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:44)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:40)\n\tat kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)\n\tat
indicating that I don’t have an adaptor for this, but I would expect that to be built in somewhere. I also tried just marshalling straight to an object but that results in a different error message, indicating I should have set up
KotlinJsonAdapterFactory
even though I have done so.
Copy code
"java.lang.IllegalArgumentException: Cannot serialize Kotlin type com.sunrun.pricing.lambda.MoshiCat. Reflective serialization of Kotlin classes without using kotlin-reflect has undefined and unexpected behavior. Please use KotlinJsonAdapterFactory from the moshi-kotlin artifact or use code gen from the moshi-kotlin-codegen artifact.\n\tat com.squareup.moshi.ClassJsonAdapter$1.create(ClassJsonAdapter.java:98)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:146)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:106)\n\tat com.squareup.moshi.Moshi.adapter(Moshi.java:80)\n\tat org.http4k.format.ConfigurableMoshi.adapterFor(ConfigurableMoshi.kt:93)\n\tat org.http4k.format.ConfigurableMoshi.asA(ConfigurableMoshi.kt:97)\n\tat org.http4k.serverless.AutoMarshallingFnLoader.invoke$lambda$0(AutoMarshallingFnLoader.kt:19)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:44)\n\tat org.http4k.serverless.AwsLambdaRuntime$asServer$1$start$1$1.invoke(AwsLambdaRuntime.kt:40)\n\tat kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)\n\tat
d

dave

12/11/2023, 9:31 PM
you can't just pass an inputstream to Moshi
Copy code
override fun <T : Any> asA(input: InputStream, target: KClass<T>): T = adapterFor(target).fromJson(
        input.source().buffer()
    )!!
Moshi.asA(input, MyThang::class)
s

snowe

12/11/2023, 9:33 PM
sorry, wasn’t clear there. I’ve commented out any deserialization I’m doing and the errors are coming from internal to http4k.
Copy code
fun MyLoader() = MoshiFnLoader {
    FnHandler { i: MoshiCat, c: Context ->
        println("Http4k - Running the function")
        "asldkfj"
    }
}
or InputStream instead of MoshiCat
both throw errors. so it never even makes it to my deserialization.
d

dave

12/11/2023, 9:34 PM
ah - you haven't got reflection at all
you need to generate adapters manually
(suggest kotshi)
s

snowe

12/11/2023, 9:35 PM
oh is it using reflection to retrieve the adapters?
oh yeah kotshi. I went back and looked for that in the docs and couldn’t find it, went with moshix.
d

dave

12/11/2023, 9:35 PM
Moshi falls back to using reflection if it can't find a registered adapter
s

snowe

12/11/2023, 9:36 PM
ah. I could specify the adapter in the reflection config if I knew what it was called. let me go try that.
and then try kotshi as well.
d

dave

12/11/2023, 9:36 PM
I'd really try kotshi - it's pretty easy.
you just annotate with @JsonSerializable and create a factory then add it to your config
stay away from reflect-config.json
(IMHO)
s

snowe

12/11/2023, 9:37 PM
alright I’ll try that. though I wouldn’t be able to add it to inputstream right? I’d need to serialize using the automarshalling
d

dave

12/11/2023, 9:37 PM
well you'd need to use that code above
1. annotate your class 2. run ksp(<kotshi>) in your gradle 3. Add the generated factory to your Moshi instance
s

snowe

12/11/2023, 9:52 PM
👍🏽
image.png
thank you for all the help ❤️
d

dave

12/11/2023, 9:57 PM
🎉
I think we're both glad you've got it working 😉
s

snowe

12/11/2023, 9:58 PM
haha yeah, I bothered you quite a lot.
d

dave

12/11/2023, 10:18 PM
Np. It's what we're here for 🙃 .
s

snowe

12/11/2023, 10:23 PM
Lenses don’t use reflection right?
d

dave

12/11/2023, 10:26 PM
The body lenses simply use whatever the underlying tech does - so if you've got Moshi with kotshi then no they don't.
None of the query/header etc lenses use reflection
s

snowe

12/11/2023, 10:26 PM
yeah that’s what I thought from looking through the code.
8 Views