https://kotlinlang.org logo
#http4k
Title
# http4k
p

Philipp Mayer

10/19/2023, 8:08 PM
How do I know if I use the V1 or V2 payload format when invoking a lambda function? I created a lambda with
ApiGatewayV2LambdaFunction
, which didn’t work, but it worked with
ApiGatewayV1LambdaFunction
(http trigger). What do I have to configure to use v2, and is this recommended? 🙂
s

s4nchez

10/19/2023, 8:11 PM
The version is dictated by the type of API (REST vs HTTP) and the what kind of integration is defined in API Gateway.
👍 1
I can’t remember what versions are supported by each type, but in practice there’s not much difference as the goal of all adapters is to just translate aws events to http4k messages.
p

Philipp Mayer

10/19/2023, 8:17 PM
Alright, thank you! I’ll just stick to V1 then. I thought at first that I need to upgrade and wasn’t sure how.
d

dave

10/19/2023, 8:21 PM
actually - we always default to V2.
V1 used to not support multi-value headers (but actually it does now). And all the examples we base around V2. In practice you want the http instead of the rest version of the lambdas.
p

Philipp Mayer

10/19/2023, 8:23 PM
It’s unclear to me what I have to change to make it work. I habe a basic lambda that reacts to http calls. It works with v1, but doesn’t with v2. Let me have a look inside my pulumi config
d

dave

10/19/2023, 8:24 PM
It's the "payload" format which makes the difference
p

Philipp Mayer

10/19/2023, 8:24 PM
The API gateway config follows the http4k example.
Found it
Thanks!!
d

dave

10/19/2023, 8:24 PM
Check out the tutorial or example repo 🙃
p

Philipp Mayer

10/19/2023, 8:25 PM
I did! 😛
Copy code
const lambdaIntegration = new aws.apigatewayv2.Integration("hello-http4k-api-lambda-integration", {
    apiId: api.id,
    integrationType: "AWS_PROXY",
    integrationUri: lambdaFunction.arn,
    payloadFormatVersion: "1.0"
});
Thanks for the help you two!
d

dave

10/19/2023, 8:25 PM
Np 🙃
Actually the examples repo is v1... We should update that really
p

Philipp Mayer

10/19/2023, 8:26 PM
Let me take care of it
One thing that is quite a pain when working with DynamoDB is kotshi. It’s unclear from the outside why it throws an exception when trying to run some kind of dynamo integration for the first time in prod/aws, esp. since it’s not happening when running the (in-memory) tests locally. It’s confusing, since I also replaced Moshi with Jackson in the setup, but still get the exception:
Copy code
private val dynamoDb = DynamoDb.Http(
    region = Region.EU_CENTRAL_1,
    credentialsProvider = { config.credentials },
    http = config.http
  )

  private val table: DynamoDbTableMapper<Artist, String, Unit> =
    dynamoDb.tableMapper<Artist, String, Unit>(
      tableName = TableName.of("pedro-artists"),
      hashKeyAttribute = Attribute.string().required("id"),
      autoMarshalling = Jackson
    )
When going with Kotshi, one than has to dive into KSP/gradle/code generation, which is quite a burden. I know it’s nothing you can change as it’s rather a general JSON problem on the JVM and I really respect what you’re doing, but might be good to hear about some speedbumps from an “outsider” 🙂
d

dave

10/19/2023, 8:56 PM
what exceptions do you get? Maybe @Andrew O'Hara can help as he's the daddy of that table mapper (if it's related to that 🙃 )
p

Philipp Mayer

10/19/2023, 9:03 PM
I’m trying to debug it right now:
Copy code
"java.base/java.lang.ClassLoader.defineClass1(Native Method)",
            "java.base/java.lang.ClassLoader.defineClass(Unknown Source)",
            "java.base/java.security.SecureClassLoader.defineClass(Unknown Source)",
            "java.base/java.net.URLClassLoader.defineClass(Unknown Source)",
            "java.base/java.net.URLClassLoader$1.run(Unknown Source)",
            "java.base/java.net.URLClassLoader$1.run(Unknown Source)",
            "java.base/java.security.AccessController.doPrivileged(Unknown Source)",
            "java.base/java.net.URLClassLoader.findClass(Unknown Source)",
            "java.base/java.lang.ClassLoader.loadClass(Unknown Source)",
            "java.base/java.lang.ClassLoader.loadClass(Unknown Source)",
            "org.http4k.connect.amazon.dynamodb.KotshiDynamoDbJsonAdapterFactory.create(KotshiDynamoDbJsonAdapterFactory.kt:364)",
>>            HERE IS THE FACTORY "org.http4k.connect.amazon.dynamodb.DynamoDbJsonAdapterFactory.create(DynamoDbMoshi.kt)",
            "com.squareup.moshi.Moshi.adapter(Moshi.java:146)",
            "com.squareup.moshi.Moshi.adapter(Moshi.java:106)",
            "com.squareup.moshi.Moshi.adapter(Moshi.java:80)",
            "org.http4k.format.ConfigurableMoshi.asFormatString(ConfigurableMoshi.kt:84)",
            "org.http4k.connect.amazon.AwsJsonAction.toRequest(AwsJsonAction.kt:27)",
            "org.http4k.connect.amazon.dynamodb.HttpDynamoDbKt$Http$1.invoke(HttpDynamoDb.kt:25)",
            "org.http4k.connect.amazon.dynamodb.DynamodbExtensionsKt$scanPaginated$1.invoke(dynamodbExtensions.kt:327)",
            "org.http4k.connect.amazon.dynamodb.DynamodbExtensionsKt$scanPaginated$1.invoke(dynamodbExtensions.kt:327)",
            "org.http4k.connect.PagedKt$paginated$1.invoke(Paged.kt:42)",
            "org.http4k.connect.PagedKt$paginated$1.invoke(Paged.kt:39)",
            "kotlin.sequences.GeneratorSequence$iterator$1.calcNext(Sequences.kt:591)",
            "kotlin.sequences.GeneratorSequence$iterator$1.hasNext(Sequences.kt:609)",
            "kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)",
            "kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:311)",
            "kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:303)",
            "kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)",
            "kotlin.sequences.SequencesKt___SequencesKt.toList(_Sequences.kt:809)",
>>>>            HERE IS THE SAVE TO THE DB "de.p10r.adapters.driven.db.ArtistRepository.save(ArtistRepository.kt:53)",
            "de.p10r.domain.UserCommandHub.process(UserCommandHub.kt:24)",
            "de.p10r.AppKt$App$1.invoke(App.kt:32)",
            "de.p10r.AppKt$App$1.invoke(App.kt:32)",
            "de.p10r.adapters.driving.http.ApiRoutesKt$ApiRoutes$2.invoke(ApiRoutes.kt:46)",
            "de.p10r.adapters.driving.http.ApiRoutesKt$ApiRoutes$2.invoke(ApiRoutes.kt:45)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:284)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:282)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.filter.ResponseFilters$ReportHttpTransaction$invoke$2$1.invoke(ResponseFilters.kt:48)",
            "org.http4k.filter.ResponseFilters$ReportHttpTransaction$invoke$2$1.invoke(ResponseFilters.kt:46)",
            "org.http4k.filter.ServerFilters$RequestTracing$invoke$3$1$1.invoke(ServerFilters.kt:96)",
            "org.http4k.filter.ServerFilters$RequestTracing$invoke$3$1$1.invoke(ServerFilters.kt:92)",
            "org.http4k.filter.ZipkinTracesKt.ensureCurrentSpan(ZipkinTraces.kt:105)",
            "org.http4k.filter.ServerFilters$RequestTracing$invoke$3$1.invoke(ServerFilters.kt:92)",
            "org.http4k.filter.ServerFilters$RequestTracing$invoke$3$1.invoke(ServerFilters.kt:91)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:284)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:282)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.routing.TemplateRouter$match$1.invoke(Router.kt:154)",
            "org.http4k.routing.TemplateRouter$match$1.invoke(Router.kt:154)",
            "org.http4k.routing.RouterMatch$MatchingHandler.invoke(Router.kt:54)",
            "org.http4k.routing.RouterMatch$MatchingHandler.invoke(Router.kt:53)",
            "org.http4k.routing.RouterBasedHttpHandler.invoke(RouterBasedHttpHandler.kt:19)",
            "org.http4k.routing.RouterBasedHttpHandler.invoke(RouterBasedHttpHandler.kt:13)",
            "org.http4k.serverless.CommonKt$AddLambdaContextAndRequest$1$1.invoke(common.kt:14)",
            "org.http4k.serverless.CommonKt$AddLambdaContextAndRequest$1$1.invoke(common.kt:11)",
            "org.http4k.filter.ServerFilters$InitialiseRequestContext$invoke$1$1.invoke(ServerFilters.kt:345)",
            "org.http4k.filter.ServerFilters$InitialiseRequestContext$invoke$1$1.invoke(ServerFilters.kt:342)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:284)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:282)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.serverless.ApiGatewayFnLoader.invoke$lambda$0(ApiGatewayFnLoader.kt:27)",
            "org.http4k.serverless.AwsLambdaEventFunction.handleRequest(AwsLambdaEventFunction.kt:15)",
            "com.amazonaws.services.lambda.runtime.api.client.EventHandlerLoader$2.call(EventHandlerLoader.java:905)",
            "com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:245)",
            "com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:197)",
            "com.amazonaws.services.lambda.runtime.api.client.AWSLambda.main(AWSLambda.java:187)"
        ]
    }
But why does that only happen in prod, but not in local tests? 🤔 Just to clarify: Setting up Kotshi, annotating the class that will be saved and rerunning fixes it. It’s just not transparent from the outside. I’ll think about how to add this to the documentation.
d

dave

10/19/2023, 9:37 PM
Hmm. Do me a favour and just update to the latest version of http4k and connect? We had some clqsspath nonsense over the last couple of releases and want to rule that out.
👍 1
(Latest versions might not be present on maven website yet - check the changelog)
p

Philipp Mayer

10/19/2023, 9:44 PM
I’ll try to push an example for this tomorrow. Thanks a ton for the help. ❤️ It’s working with Kotshi, so everything is fine. I was just curious why the error didn’t appear when running the tests locally. The last bit I’m battling with is that
Copy code
fun DynamoDb.createPedroTables() =
  createTable(
    TableName = TableName.of("pedro-artists"),
    KeySchema = KeySchema.compound(AttributeName.of("id")),
    AttributeDefinitions = listOf(Attribute.string().required("id").asAttributeDefinition()),
    ProvisionedThroughput = ProvisionedThroughput(
      ReadCapacityUnits = 10,
      WriteCapacityUnits = 10
    )
  )
The creation of tables works, but using it in the tableMapper doesn’t:
Copy code
private val table: DynamoDbTableMapper<Artist, String, Unit> =
    dynamoDb.tableMapper<Artist, String, Unit>(
      tableName = TableName.of("pedro-artists"),
      hashKeyAttribute = Attribute.string().required("id"),
    )
Maybe also something for @Andrew O'Hara. I’ll look into it more closely tomorrow.
a

Andrew O'Hara

10/19/2023, 10:21 PM
I can look closer at this later, but my kneejerk reaction is that kotlin-reflect is available in your local environment, but not in prod. If you build a lens with kotshi, it will happily fall back to reflection if there are no kotshi mappers defined for the class. So it will work locally, but not on prod. So make sure you have serializers properly set up for the class you're serializing? However, as you noticed, you get the same issue with Jackson, so probably not related.
👀 1
d

dave

10/20/2023, 6:14 AM
Yes - that makes sense. Connect doesn't have reflection available generally on purpose.
p

Philipp Mayer

10/20/2023, 6:17 AM
Good callout Andrew, thanks. But if I use Jackson, I would have reflect on the classpath 🤔
a

Andrew O'Hara

10/20/2023, 1:32 PM
So what's the issue with using the tableMapper? Do you get the same error as previously mentioned?
p

Philipp Mayer

10/20/2023, 1:37 PM
I think my confusion lies in the following two problems: 1. I was getting a kotshi error in AWS that I was not getting when I was running the tests locally. You pointed out that this might be because of
kotlin-reflect
, which makes sense, but for the MVP I used Jackson in the rest of the app, so Jackson (and as sub dependency kotlin-reflect) was on the classpath, also in production. 2. I was getting a kotshi error even though I was providing
autoMarshalling = Jackson
. Just adding Kotshi + annotating the class fixed it for me, so I didn’t bother diving deeper. My suggestion is that I will create a PR that calls out that kotshi + ksp + the correct annotation at the entity to save need to be in place to make everything work. Wdyt?
d

dave

10/20/2023, 1:41 PM
I'm still a bit confused as to the overall problem. Under what circumstances do things blow up - as in - what is the code trying to do? Kotshi will be. used for all standard dynamo serialisation. I wonder if this is a wrong import (like with Body.auto<>)?
p

Philipp Mayer

10/20/2023, 1:42 PM
I will try to provide an example later 👍