Hi, I'm trying to contaminate my team with Kotlin....

# server

[Daniele Segato]

Hi, I'm trying to contaminate my team with Kotlin.

Our server architecture is currently using node.js (js/ts) on AWS lambda (serverless, through API gateway) and mongodb as main storage.

Since AWS released kotlin SDK I think now it would be a good time to propose some experiment and see if we can start writing some new stuff (at first) in kotlin and eventually adopt it everywhere.

I'm looking for people who use Kotlin on a serverless architecture to share their setup and tips to get started.

Thanks!

[Andrew O'Hara]

I wrote about something like this recently. It discusses the various options and includes a sample API application.

If you're building APIs, and can't/won't use SnapStart or native images, cold-starts (on the JVM) are going to be your biggest concern, and this guide will suggest the tools to make it far more manageable.

https://medium.com/better-programming/faster-kotlin-apis-on-aws-lambda-8694649bf9dd

[James Richardson]

Can't really recommend the aws sdk. It's too slow to initialise for lambda. Have you tried http4k connect?

[Daniele Segato]

Thanks, I'll have a look.

I'm familiar with the cold start issue, but I've also seen there have been a lot of improvements on that.

[Daniele Segato]

I'm afraid AWS isn't optional :-)

I do not know http4k but I'll have a look

[dave]

You're looking for a combination of small ZIP size, low memory footprint, and heap usage. Anything that involves jackson is a bad start - (and especially with the Kotlin reflect jar (2-3mb)) which is why connect uses Moshi (184kb) with generated adapters. The Kotlin AWS SDK gives you a dsl and suspend over the Java version - which will obvs bring in those libs as well (looking at another couple of meg). Usage of coroutines in general is also probably overkill for lambda when you're only doing 1 request at a time 🙃

[dave]

Proguard might be a good option here if you want to remain JVM, or you can look at jlink+Graalvm compilation to tree shake and reduce binary size)

[James Richardson]

Of course! Using AWS services from a lambda or another aws service is usually required and very useful. Its just that the AWS SDK is, how can I say, fully-featured, and has a particular style, that might mean that it's actually easier to just write against the AWS service API, rather than use the SDK. Remember the Bezos memo? It's the api that's the thing, not the sdk. Http4k has a lightweight implementation of some services and even if you don't use that, you might find the approach/mindset useful...

[Andrew O'Hara]

To be clear, http4k-connect is just a lightweight, reflectionless, and test-friendly SDK for common APIs, like AWS. It's built on top of Http4k, but that doesn't mean you need to adopt the entire Http4k ecosystem.

[sdeleuze]

Spring Boot + GraalVM native image shipped as a container works pretty well. The recording of my Kotlinconf related talk will be available in a few days. https://docs.spring.io/spring-boot/docs/current/reference/html/native-image.html

[sdeleuze]

We are also working on CRaC support to provide a JVM option but that’s still WIP.

[James Richardson]

Hey Sébastien - yes your talk was very interesting, I was there. However, we are definitely talking about different orders of magnitude. Spring boot apps may start in 30 seconds, while http4k apps may start in 0.3, without graal. With graal, spring boot may start in 300ms, http4k 30ms. Not knocking spring boot - for some it may be very appropriate, and it covers a whole load of use cases, however on lambda reflection on startup is something that can cause slow startup, and all of aws sdk, Jackson, and spring boot use it. You also get penalised for large binary footprint. Other light weight libraries use different techniques that may mean you can "scale to zero" by default, just out of the box.
Having wasm as a compile target, along with a suitable native i/o shim, will only make things more interesting, as using wasm workers in an isolate, as per cloudlfare functions, can load and start in only a very few ms.

[James Richardson]

Ultimately the choice of server tech, frameworks, libraries, deployment environment etc , is all a function of many forces and variables - so its great there are so many choices and more appearing every day.

[sdeleuze]

Hi James, glad you saw and liked the talk.

It is great to have lightweight solutions like http4k, and I am glad we have various options to run Kotlin workloads.

I think the regular JVM and its JIT are by design not designed for serverless workload regardless of the framework used. A JVM need to be warmed up before giving optimal performances, JIT requires CPU power after startup. Native and CRaC are IMO 2 better ways to go serverless with JVM workloads.

What I am showing here is figures for a full Petclinic with database creation at startup running on tiny cloud instances. So 2 important points to keep in mind:

• Local and Cloud startup times are different things.
• https://github.com/sdeleuze/spring-kotlin-functional is likely conceptually closer to what http4k offers, it leverages Kotlin serialization instead of Jackson, and you can even remove most of the runtime reflection using Spring AOT transformation on the JVM.
I will try to measure and share related data points later to allow people to see the bigger picture.

[dave]

I think we're all looking to do the most with least - and it's very good that JVM/Spring folks are starting to serious look at this, because the even the full petstore app is trivial - it's just a couple of endpoints and a database connection - it should easily run on 0.25 of a CPU with a few 100M's memory. It goes to show just how much work there is to do in this area for the JVM to remain competitive in the future landscape - I do worry that with all this native-compilation/build-time AOT weaving, we as a community are simply piling on more and more complexity instead of taking a step back and having a bit more of a think about if we need it at all.

(and TBH, I am actually am pretty shocked that even this simple example is taking 7s to startup on 2 modern CPUs and 4Gig RAM - that's an insane amount of horsepower to need for something so simple in 2023 - not to mention the $ and environmental cost of running this stuff in production. )

[Daniele Segato]

The recently announced AWS kotlin SDK doesn't build on the Java implementation. And it doesn't rely on the JDK at all. It is a set of kotlin multiplatform libraries.
I doubt they use Moshi or Jackson. They probably use kotlin serialization and ktor.

Coroutines are very helpful even if you have a single request imho.

[dave]

There is currently no Kotlin multiplatform IO library - currently the AWS SDK relies on Square's OKIO. There is no Ktor (or kotlinX serialization IIRC) dependency - it uses smithy for the client.

We chatted to Roman and the JB folks at KotlinConf and apparently there is a plan to port/migrate OKIO into the kotlinX space, which will make it a viable alternative to the JVM IO. Ktor itself also currently has it's own IO lib for the moment AFAIK so presumably will need to be migrated (v3?). When the datetime library goes stable as well, that should go a long way to completing the landscape. 🙂

[dave]

The point with coroutines isn't if they are useful or not (YMMV but I'm not sure why they should be needed by default in a Serverless environment 🙃) - it's more to do with the size of the binary which is output and thus impacts both the cold start time and runtime overhead when running in a JVM runtime. Snapstart might go some way to fixing cold start. Rust and Go have compact binary sizes by default, so that's what you're having to go up against - they have a natural advantage in that area. IMHO - Kotlin has a nicer/kinder language style - and that's the tradeoff.

The less magic hoops we have to jump through at compile-time, the better to ward off unintended surprises at runtime. 🙃

[Daniele Segato]

I'm not necessarily bound to JVM. With kotlin multiplatform I could also deploy native code or JavaScript (which doesn't sound like a good idea).

I wanted to know if there's anybody that have experience with that.

I've read in the article posted by @Andrew O'Hara above that SnapStart still has some limitations but I do not have the tools to evaluate them, I'm an Android developer.

Nor I know how much SnapStart actually fixes cold start.

I do know we have some cold start issues in our current backend with the Node environment and that historically non-js targets had worst cold starts. And I've read that SnapStart should fix that.

@dave I don't know where you got this information. From what I see the Kotlin SDK is multiplatform and doesn't use Okio: https://github.com/awslabs/aws-sdk-kotlin
https://github.com/awslabs/smithy-kotlin

[dave]

If you dig down far enough in the Dependency tree you'll see okio - although that is a runtime dep. I haven't dug through it in practice though.

https://mvnrepository.com/artifact/aws.smithy.kotlin/runtime-core-jvm/0.17.1

[dave]

Still, it does highlight the lack of a standard multiplatform Io lib. we really need one to help the ecosystem thrive. Depending on a third party vendor isn't going to cut it 🙃

[James Richardson]

I also just noticed the opening comment in this thread... which is rather good. "I'm trying to contaminate my team with kotlin" - autocorrect ftw!

[wakingrufus]

at a certain scale, your aws lambdas might start to incur a cost that exceeds the cost of running a jvm-based server, as a warmed up JVM can perform quite well for long-lived processes. in that case, building a microservice to host an API that has outgrown the scale of a lambda would be a great place to try kotlin. there are cool things happening with graalvm in regards to serverless as discussed above, but imo you would definitely be on the cutting edge there, and not really have a well-worn path to follow.

[dave]

^^ I can agree with this - the cost of a simple fargate instance is about $8 a month and TBH the complexity of getting it running isn't too much more (once you account for the extra infra cost - but am assuming that you already have that).

(Shameless plug: Of course, one of the nice things about http4k is that to convert it from a lambda to a running fargate server you can just remove the Lambda wrapper class and put your app straight into a main inside a simple docker image with no other changes to your actual app! 😉)

[Daniele Segato]

I see your point. But our entire architecture is on AWS lambda / API gateway.

It's kind of a bigger jump to take than just write 1 microservice with kotlin

[wakingrufus]

I know this is off topic, but isn't that the point of api gateway, you can hook your endpoints up to any number of back ends and change them?

[wakingrufus]

Imho, if you dont have a plan for how to do that, i would recommend starting to think about it before you need it, so you are ready when the need comes

[Daniele Segato]

@sdeleuze I've yet to see your talk, I didn't expected a spring talk to be about serverless architecture. Can the GrallVM solution be deployed as an AWS Lambda? Do you cover any of that in the talk? (Didn't find it in the article).

I lack some experience in everything around deployments and I only know bits of AWS.

@wakingrufus I suppose you are right about API Gateway, I don't know these things very well yet. I know our API Gateway is connected to some authentication system we developed and to the actual lambda. I've helped a bit with backend development and I know how to code, but I'm really lacking in the architecture part of it.

I'm trying to anticipate some questions my colleagues for the backend could have by proposing some solution that works while they get to know kotlin and, hopefully, love it.
So my idea was to keep the change minimal and just deploy a lambda written in kotlin without changing anything else.
I'm not against other solutions but I'd had to study a bit more in detail before proposing it to avoid making a step longer than my leg.

[sdeleuze]

@Daniele Segato Yes, a Spring Boot native application can be deployed in a Serverless platform like AWS Lambda, you can use Spring Cloud Function like in https://github.com/spring-cloud/spring-cloud-function/tree/main/spring-cloud-function-samples/function-sample-aws-native or build a custom handler/container following AWS Lambda documentation.

[sdeleuze]

Here are the refined data points with https://github.com/sdeleuze/spring-kotlin-functional + the JVM + AOT variant for Petclinic.

[wakingrufus]

Are any of these JVM, no autoconfigurarion (spring-fu functional style)?

[sdeleuze]

For https://github.com/sdeleuze/spring-kotlin-functional, regular autoconfiguration transformed AOT to functional configuration (so more stuff frozen at build, you can't change the beans for example, see https://github.com/sdeleuze/demo-profile-aot for more details).

[sdeleuze]

So no Spring Fu style here, but AOT turns @Bean to Spring Fu style from a runtime perspective.

[wakingrufus]

Ok so theoretically, spring fu would have jvm+aot levels of performance, and there would still be a lot more gains to be had by going native with that?

[sdeleuze]

Almost yeah, Spring Fu would be a bit faster but pretty close to jvm+aot and native data points.

[James Richardson]

I was interested in the story here - so I decided to try to make an equivalent comparison in http4k. The repo is at https://github.com/time4tea/http4k-functional and the results summary is at https://observablehq.com/d/53eb5df0da9b77fa

I can summarise, that the http4k servers started up in 11ms on 0.25CPU/0.5GB RAM, and 8ms on the larger configuration.

I tried to make the services do the same thing, but there may be a difference. Happy to make any corrections if there are grave errors on my part. Hopefully this is just some useful data points.

I don't want to start a which framework is better discussion - this is not the goal. There are loads of considerations in picking a library or framework. I did learn some things about Graal and Azure, and start up times of services on the Azure container service. In particular, there is some information in the repo about how to make a graal executable. This is the very first thing I did with Graal, so it may be a terrible example!

[Luca]

Hi guys I find this thread searching for CRaC word, do you know if ktor will be support this functionality?

[Andrew O'Hara]

How are you adapting ktor for Lambda? If you have access to the Lambda handler class, you can add the CRaC hooks to it without any particular support from the server.

[Luca]

Hi @Andrew O'Hara I'm running the ktor server on Kubernetes

[Andrew O'Hara]

I wasn't aware CRaC worked for standard containers. Good to know!

This thread was about Lambdas. You might have better luck asking your question directly in the #ktor channel. But as I mentioned previously, your server probably doesn't need to have any specific support for CRaC.

[Luca]

Great thanks I'm not aware of this channel 🙂 ... Yes I'm really new in CRaC searching a little I saw spring have "support" so I was wondering if there is something similar with ktor

[Andrew O'Hara]

I found this guide that can help you get started with CRaC for any application. There is a Spring example, but ahead of it, they integrate it into a plain main function.
https://bell-sw.com/blog/how-to-use-crac-with-java-applications/

[Luca]

Hey andrew many thanks!

[Luca]

Always there is spring example, I don't know if the server type can change things (jetty vs netty)