At DoorDash we are one of the largest adopters of Kotlin as a backend language. I wrote a blog post with a detailed comparison of all our top choices. It also explains how we overcame some challenges to realize Kotlin as a better version of Java. https://doordash.engineering/2021/05/04/migrating-from-python-to-kotlin-for-our-backend-services/

I am not sure, why do you need REPL for the backend, but Kotlin does have REPL (and Java has jShell as well, but Kotlin is more powerful in this regard). The REPL is built-in in IDEA and could be run in stand-alone mode, for example in https://github.com/Kotlin/kotlin-jupyter or in a new https://github.com/Kotlin/kotlin-interactive-shell

Yeah someone on Reddit posted that. The idea is debugging systems in prod by calling functions, etc

It is quite easy to load libraries both in Jupyter and in KI, just use

@file:DependsOn

annotations, you don't even need gradle for that. But Kotlin is a compiled language, so you can't just connect to the running machine and run some arbitrary code in it. It is not safe after all. I think there are a lot of tools on JVM to run online diagnostics though.

And we use those as well 😄 We have heavily invested in java agents and even lower level jvm, and ebpf metrics to help us understand our systems. It's just sometimes for those long-tail issues, being able to call our code in the live environment has been useful

You can embedd a kotlin scripting environment in your server and it will have access to the whole classpath. I would strongly discourage doing that for security and stability reasons, but it will work just fine.

We don't necessarily care about it running inside the live process, as long as it can pull as the same env vars, etc (or we can fake inject the env vars form the running process if we need to 😉).

Well it is actually simpler than in Python because you do not need to set the environment, dependencies are resolved via Ivy.

with

pipenv

thats true in python as well

If you can run a fake, the regular debug agent seems to do exactly whay you want. It allows to evaluate expressions in breakpoints. Of course, you can't use it for production environment.

At times. Before GRPC-Kotlin we had to do a lot of shimming ourself to move things to/from GRPCContext->CoroutineContext

Building

CoroutineContext

by hand is strongly discouraged and what you were writing about is relevant only to one of its keys - Job. I tink they added something for threadLocal. But in general in Kotlin it is possible in most cases to replace it by coroutine-local.

Correct, there are some ways to help with that, so we were providing the shims, and sometimes teams accidentally 'cached' the output (which resulted in them trying to used cancelled Contexts).

@Matt Anger what did you use to replace protokruft btw? Interested that anyone else used it! 😂

We are just using grpc-kotlin directly now.

Does that get rid of the hideous builder syntax?

Not all the way, but we've standardized on using the

apply {}

syntax to make it easier to read

Well glad it's working out for you. We binned grpc because quite frankly it was horrible.

Great article! Would love to read about what frameworks you settled on (and why) as well. Do you use any ORM? Http libraries? Dependency injection? 🙂

😄 It has its pain points for sure. Particularly around nullable vs notset, etc that matters for some of our endpoints. It has required a bit of re-education in how people think about their services

Yep - that's what we couldn't get around. Was thinking about wire as a better halfway house

Our kotlin-platform team has created a series of 'fake servers' + 'fake clients' to help with testing

Sounds similar to the SaaF mechanic we promote for Http4k projects. Testing through the boundary ftw.

At DoorDash we are one of the largest adopters of Kotlin as a backend language

Any info on size? LoC or number of services or anything like that?

Is not commonly used on the server side, meaning there are fewer samples and examples for our developers to use

Not sure I'd agree with this... yeah its popular on Android but plenty of people use it on the backend too: https://www.jetbrains.com/lp/devecosystem-2019/kotlin/.

We switched to strongly typed JSON marshalling with Jackson (although I'd use Moshi now). It was simpler, much more testable, less error prone, and had lots less hidieous boilerplate (even with Protokruft to take away the builder syntax). The effect on the project was marked and everyone was a lot happier, especially as we were using K8S without a service mesh at the time (which leads to persistent connection issues and you don't have to do client-side LB any more).

But that's just the serialization layer. You're comparing JSON with Protobuf, not gRPC.

It's just something you have to think of. Client-side load balancing isn't that bad as long as you use headless services in k8s

I'm comparing the entire experience that we had with grpc and protobuf to a much simpler pattern that worked out better and safer and more reliably. Things like testability probelms were nothing to do with protobuf, and overall we've found a strong correlation with people who like clever things and impending maintenance nightmares. Our developer experience was horrible. YMMV of course.

from a perf perspective gRPC will be better (as it is a binary protocol) but unless you are running at a very large scale that perf difference might not be enough to offset some of the "pains" (like already mentioned testing)

💯 (needs more 00s!). One of the problems with this industry is that everyone and their dog has been convinced that their scaling requirements are "very large". 😂

The other benefit of GRPC is that we can use annotations on the protobuf to declare behavior of the client 🙂

The other thing you get from protobuf and similar binary protocols like Avro is explicit rules around schema evolution. This is also a "big system problem" though.