How do I know if I use the V1 or V2 payload format...

# http4k

[Philipp Mayer]

How do I know if I use the V1 or V2 payload format when invoking a lambda function?
I created a lambda with ApiGatewayV2LambdaFunction, which didn’t work, but it worked with ApiGatewayV1LambdaFunction (http trigger). What do I have to configure to use v2, and is this recommended? 🙂

[s4nchez]

The version is dictated by the type of API (REST vs HTTP) and the what kind of integration is defined in API Gateway.

[s4nchez]

I can’t remember what versions are supported by each type, but in practice there’s not much difference as the goal of all adapters is to just translate aws events to http4k messages.

[Philipp Mayer]

Alright, thank you! I’ll just stick to V1 then. I thought at first that I need to upgrade and wasn’t sure how.

[dave]

actually - we always default to V2.

[dave]

V1 used to not support multi-value headers (but actually it does now). And all the examples we base around V2. In practice you want the http instead of the rest version of the lambdas.

[Philipp Mayer]

It’s unclear to me what I have to change to make it work.
I habe a basic lambda that reacts to http calls. It works with v1, but doesn’t with v2. Let me have a look inside my pulumi config

[dave]

It's the "payload" format which makes the difference

[Philipp Mayer]

The API gateway config follows the http4k example.

[Philipp Mayer]

Found it

[Philipp Mayer]

Thanks!!

[dave]

Check out the tutorial or example repo 🙃

[Philipp Mayer]

I did! 😛

const lambdaIntegration = new aws.apigatewayv2.Integration("hello-http4k-api-lambda-integration", {
    apiId: api.id,
    integrationType: "AWS_PROXY",
    integrationUri: lambdaFunction.arn,
    payloadFormatVersion: "1.0"
});

[Philipp Mayer]

Thanks for the help you two!

[dave]

Np 🙃

[dave]

Actually the examples repo is v1... We should update that really

[Philipp Mayer]

Let me take care of it

[Philipp Mayer]

https://github.com/http4k/http4k/pull/989

[Philipp Mayer]

One thing that is quite a pain when working with DynamoDB is kotshi. It’s unclear from the outside why it throws an exception when trying to run some kind of dynamo integration for the first time in prod/aws, esp. since it’s not happening when running the (in-memory) tests locally.
It’s confusing, since I also replaced Moshi with Jackson in the setup, but still get the exception:

private val dynamoDb = DynamoDb.Http(
    region = Region.EU_CENTRAL_1,
    credentialsProvider = { config.credentials },
    http = config.http
  )

  private val table: DynamoDbTableMapper<Artist, String, Unit> =
    dynamoDb.tableMapper<Artist, String, Unit>(
      tableName = TableName.of("pedro-artists"),
      hashKeyAttribute = Attribute.string().required("id"),
      autoMarshalling = Jackson
    )

When going with Kotshi, one than has to dive into KSP/gradle/code generation, which is quite a burden.

I know it’s nothing you can change as it’s rather a general JSON problem on the JVM and I really respect what you’re doing, but might be good to hear about some speedbumps from an “outsider” 🙂

[dave]

what exceptions do you get? Maybe @Andrew O'Hara can help as he's the daddy of that table mapper (if it's related to that 🙃 )

[Philipp Mayer]

I’m trying to debug it right now:

"java.base/java.lang.ClassLoader.defineClass1(Native Method)",
            "java.base/java.lang.ClassLoader.defineClass(Unknown Source)",
            "java.base/java.security.SecureClassLoader.defineClass(Unknown Source)",
            "java.base/java.net.URLClassLoader.defineClass(Unknown Source)",
            "java.base/java.net.URLClassLoader$1.run(Unknown Source)",
            "java.base/java.net.URLClassLoader$1.run(Unknown Source)",
            "java.base/java.security.AccessController.doPrivileged(Unknown Source)",
            "java.base/java.net.URLClassLoader.findClass(Unknown Source)",
            "java.base/java.lang.ClassLoader.loadClass(Unknown Source)",
            "java.base/java.lang.ClassLoader.loadClass(Unknown Source)",
            "org.http4k.connect.amazon.dynamodb.KotshiDynamoDbJsonAdapterFactory.create(KotshiDynamoDbJsonAdapterFactory.kt:364)",
>>            HERE IS THE FACTORY "org.http4k.connect.amazon.dynamodb.DynamoDbJsonAdapterFactory.create(DynamoDbMoshi.kt)",
            "com.squareup.moshi.Moshi.adapter(Moshi.java:146)",
            "com.squareup.moshi.Moshi.adapter(Moshi.java:106)",
            "com.squareup.moshi.Moshi.adapter(Moshi.java:80)",
            "org.http4k.format.ConfigurableMoshi.asFormatString(ConfigurableMoshi.kt:84)",
            "org.http4k.connect.amazon.AwsJsonAction.toRequest(AwsJsonAction.kt:27)",
            "org.http4k.connect.amazon.dynamodb.HttpDynamoDbKt$Http$1.invoke(HttpDynamoDb.kt:25)",
            "org.http4k.connect.amazon.dynamodb.DynamodbExtensionsKt$scanPaginated$1.invoke(dynamodbExtensions.kt:327)",
            "org.http4k.connect.amazon.dynamodb.DynamodbExtensionsKt$scanPaginated$1.invoke(dynamodbExtensions.kt:327)",
            "org.http4k.connect.PagedKt$paginated$1.invoke(Paged.kt:42)",
            "org.http4k.connect.PagedKt$paginated$1.invoke(Paged.kt:39)",
            "kotlin.sequences.GeneratorSequence$iterator$1.calcNext(Sequences.kt:591)",
            "kotlin.sequences.GeneratorSequence$iterator$1.hasNext(Sequences.kt:609)",
            "kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)",
            "kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:311)",
            "kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:303)",
            "kotlin.sequences.TransformingSequence$iterator$1.hasNext(Sequences.kt:214)",
            "kotlin.sequences.SequencesKt___SequencesKt.toList(_Sequences.kt:809)",
>>>>            HERE IS THE SAVE TO THE DB "de.p10r.adapters.driven.db.ArtistRepository.save(ArtistRepository.kt:53)",
            "de.p10r.domain.UserCommandHub.process(UserCommandHub.kt:24)",
            "de.p10r.AppKt$App$1.invoke(App.kt:32)",
            "de.p10r.AppKt$App$1.invoke(App.kt:32)",
            "de.p10r.adapters.driving.http.ApiRoutesKt$ApiRoutes$2.invoke(ApiRoutes.kt:46)",
            "de.p10r.adapters.driving.http.ApiRoutesKt$ApiRoutes$2.invoke(ApiRoutes.kt:45)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:284)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:282)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.filter.ResponseFilters$ReportHttpTransaction$invoke$2$1.invoke(ResponseFilters.kt:48)",
            "org.http4k.filter.ResponseFilters$ReportHttpTransaction$invoke$2$1.invoke(ResponseFilters.kt:46)",
            "org.http4k.filter.ServerFilters$RequestTracing$invoke$3$1$1.invoke(ServerFilters.kt:96)",
            "org.http4k.filter.ServerFilters$RequestTracing$invoke$3$1$1.invoke(ServerFilters.kt:92)",
            "org.http4k.filter.ZipkinTracesKt.ensureCurrentSpan(ZipkinTraces.kt:105)",
            "org.http4k.filter.ServerFilters$RequestTracing$invoke$3$1.invoke(ServerFilters.kt:92)",
            "org.http4k.filter.ServerFilters$RequestTracing$invoke$3$1.invoke(ServerFilters.kt:91)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:284)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:282)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.routing.TemplateRouter$match$1.invoke(Router.kt:154)",
            "org.http4k.routing.TemplateRouter$match$1.invoke(Router.kt:154)",
            "org.http4k.routing.RouterMatch$MatchingHandler.invoke(Router.kt:54)",
            "org.http4k.routing.RouterMatch$MatchingHandler.invoke(Router.kt:53)",
            "org.http4k.routing.RouterBasedHttpHandler.invoke(RouterBasedHttpHandler.kt:19)",
            "org.http4k.routing.RouterBasedHttpHandler.invoke(RouterBasedHttpHandler.kt:13)",
            "org.http4k.serverless.CommonKt$AddLambdaContextAndRequest$1$1.invoke(common.kt:14)",
            "org.http4k.serverless.CommonKt$AddLambdaContextAndRequest$1$1.invoke(common.kt:11)",
            "org.http4k.filter.ServerFilters$InitialiseRequestContext$invoke$1$1.invoke(ServerFilters.kt:345)",
            "org.http4k.filter.ServerFilters$InitialiseRequestContext$invoke$1$1.invoke(ServerFilters.kt:342)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:284)",
            "org.http4k.filter.ServerFilters$CatchAll$invoke$2$1.invoke(ServerFilters.kt:282)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.core.Http4kKt$then$2$1.invoke(Http4k.kt:15)",
            "org.http4k.serverless.ApiGatewayFnLoader.invoke$lambda$0(ApiGatewayFnLoader.kt:27)",
            "org.http4k.serverless.AwsLambdaEventFunction.handleRequest(AwsLambdaEventFunction.kt:15)",
            "com.amazonaws.services.lambda.runtime.api.client.EventHandlerLoader$2.call(EventHandlerLoader.java:905)",
            "com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:245)",
            "com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:197)",
            "com.amazonaws.services.lambda.runtime.api.client.AWSLambda.main(AWSLambda.java:187)"
        ]
    }

But why does that only happen in prod, but not in local tests? 🤔

Just to clarify: Setting up Kotshi, annotating the class that will be saved and rerunning fixes it. It’s just not transparent from the outside. I’ll think about how to add this to the documentation.

[dave]

Hmm. Do me a favour and just update to the latest version of http4k and connect? We had some clqsspath nonsense over the last couple of releases and want to rule that out.

[dave]

(Latest versions might not be present on maven website yet - check the changelog)

[Philipp Mayer]

I’ll try to push an example for this tomorrow. Thanks a ton for the help. ❤️
It’s working with Kotshi, so everything is fine. I was just curious why the error didn’t appear when running the tests locally.

The last bit I’m battling with is that

fun DynamoDb.createPedroTables() =
  createTable(
    TableName = TableName.of("pedro-artists"),
    KeySchema = KeySchema.compound(AttributeName.of("id")),
    AttributeDefinitions = listOf(Attribute.string().required("id").asAttributeDefinition()),
    ProvisionedThroughput = ProvisionedThroughput(
      ReadCapacityUnits = 10,
      WriteCapacityUnits = 10
    )
  )

The creation of tables works, but using it in the tableMapper doesn’t:

private val table: DynamoDbTableMapper<Artist, String, Unit> =
    dynamoDb.tableMapper<Artist, String, Unit>(
      tableName = TableName.of("pedro-artists"),
      hashKeyAttribute = Attribute.string().required("id"),
    )

Maybe also something for @Andrew O'Hara. I’ll look into it more closely tomorrow.

[Andrew O'Hara]

I can look closer at this later, but my kneejerk reaction is that kotlin-reflect is available in your local environment, but not in prod.

If you build a lens with kotshi, it will happily fall back to reflection if there are no kotshi mappers defined for the class. So it will work locally, but not on prod. So make sure you have serializers properly set up for the class you're serializing?

However, as you noticed, you get the same issue with Jackson, so probably not related.

[dave]

Yes - that makes sense. Connect doesn't have reflection available generally on purpose.

[Philipp Mayer]

Good callout Andrew, thanks. But if I use Jackson, I would have reflect on the classpath 🤔

[Andrew O'Hara]

So what's the issue with using the tableMapper? Do you get the same error as previously mentioned?

[Philipp Mayer]

I think my confusion lies in the following two problems:
1. I was getting a kotshi error in AWS that I was not getting when I was running the tests locally.
You pointed out that this might be because of kotlin-reflect, which makes sense, but for the MVP I used Jackson in the rest of the app, so Jackson (and as sub dependency kotlin-reflect) was on the classpath, also in production.

2. I was getting a kotshi error even though I was providing autoMarshalling = Jackson.

Just adding Kotshi + annotating the class fixed it for me, so I didn’t bother diving deeper. My suggestion is that I will create a PR that calls out that kotshi + ksp + the correct annotation at the entity to save need to be in place to make everything work. Wdyt?

[dave]

I'm still a bit confused as to the overall problem. Under what circumstances do things blow up - as in - what is the code trying to do? Kotshi will be. used for all standard dynamo serialisation. I wonder if this is a wrong import (like with Body.auto<>)?

[Philipp Mayer]

I will try to provide an example later 👍