If my application has a bunch of dependencies whic...

# ktor

[Levi]

If my application has a bunch of dependencies which various routes, etc. need to use it, is there a way to wire those dependencies throughout the application so that they're accessible throughout? In Haskell, I might use some Reader monad transformer to apply all the layers from my environment. I know I can just use an object singleton, but I'd like to keep things more testable by providing concrete implementations of the environment/dependencies and pass to my route functions all the way from the Application.module() level. Hopefully that questions make sense. tl;dr looking for whether there's some ktor/kotllin-idiomatic way to enable routes throughout my application to access a particular instance of some dependency.

[Shawn]

are you asking if you can do dependency injection in Kotlin?

[Levi]

Sorry, question may not have been very clear. Not whether it can be done, but what's a simple/idiomatic way to do it, perhaps specific to a ktor application, and without using something like koin

[Shawn]

You don't need a framework to do DI, you just... do it. You write your service classes/functions/dependents so that they accept their dependencies either via their constructors, factory methods, and/or function signatures, and then you wire up real dependencies in your application code, organized however you would like, such that they're visible to wherever you write your Application.module() function. In your test code, you pass in mocks/stubs where appropriate.

[Levi]

Right, that's the path I'd like to take and I think the direction I was heading in thus far if I understand you correctly. For example, in my Application.module() I wire up the dependencies and pass them through, i.e.:

fun Application.module() {
  val auth0Client = Auth0Client(environment.config)
  val userRepository = UserRepository() // args can be provided here to control the concrete behavior
  userRoutes(auth0Client, userRepository)
  }

I think what I'm missing is how to swap out those dependencies with the ones I'd like to test with. For example, if I have a test:

@Test
    fun testRoot() = testApplication {...}

how do I set up the test to use an Auth0Client, UserRepository, etc. that I'd like to use for testing?

[Shawn]

This is a very broad subject that'd be hard to cover succinctly in this format, but here are a few notes:
• in your tests, you should generally aim to separate unit tests from integration tests. Unit tests are where mocks and stubs come in handy because you're testing a small, exposed segment of code (e.g. a public function on an implementation class) and you want to inspect what the subject under test passes to its dependencies and control the return values that the subject receives from those mocked dependencies
• In unit tests, you want to mock interfaces that the dependencies implement as opposed to trying to mock implementing classes
• The standard for mocking on the JVM is Mockito, and is probably what I'd recommend as well
• In integration tests, you're performing an end-to-end test of code that's been wired up as close to how a real deployment would be wired up without actually calling whatever you can't afford for real (like, for example, a real payment processor if you're testing code that handles credit cards, or a live database to test if your DAO or Repository code is properly converting data objects into valid queries)

[Levi]

Right. I guess I am mixing unit with integration testing, which I shouldn't do. Focusing just on unit testing for now, this should be easy enough. Either use interfaces, as you said, or make the constructors my dependency classes take arguments (both are the same basic concept)

[Levi]

Out of curiosity, do you prefer Mockito vs MockK? If so, why?

[Levi]

(I don't have much experience with either)

[Shawn]

I don't really have any experience with MockK, so I'll leave that open for a subject matter expert to answer

[Shawn]

But I do want to stress that you want to use interfaces in either case

[Levi]

Just easier/more flexible?

[Levi]

re interfaces

[Shawn]

Well, it's considered bad form to mock implementing classes

[Shawn]

and up until recently, Mockito didn't support mocking final classes (which your implementing classes are by default in Kotlin)

[Shawn]

in practice, you don't want to be concerned with the specific behavior of a dependency implementation—what you really want is to see how your test subject interacts with its dependencies and how it handles specific values

[Shawn]

the _subject_'s behavior, not its dependencies

[Levi]

Right right

[Levi]

I know this is potentially a longer discussion with many different view, but is testing a specific route a unit test or integration test? I would think that once you're at the level of a route, that is closer to integration. Testing frameworks, like for Django, Ktor, etc. tend to push you towards integration tests. For true unit tests, I wouldn't even need testApplication since I'd just be testing small public functions and such, right?

[Shawn]

Well, it depends—testing a "route" by hitting it with an HTTP client and letting your service complete the whole request as it would in production is definitely an integration test. Testing one route can involve multiple services or controllers as well as caching and the underlying framework managing the various service elements. On the other hand, one route could just mean one call to one controller and end up being not that different from a unit test

[Levi]

right

[Levi]

The latter case is rarer in my experience

[Shawn]

Still, even if all the top level route logic does is call a controller method, it's useful to isolate the controller method from the controller's real dependencies in a unit test

[Shawn]

there's value in both is what I mean

[Levi]

For sure.

[Shawn]

Sorry, to answer your original question more directly, you absolutely do not need testApplication for a unit test—all you need is the class you're testing and the mocks it needs for instantiation

[Levi]

Thanks @Shawn Appreciate all the ideas and feedback. I seem to completely rethink testing every few years and this helped crystallize my approach a lot.

[Levi]

Unrelated to this discussion, but curious how you organize your tests. Do folks tend to (or is there a convention to) structure the directories and packages in src/test to match src/main?

[Shawn]

That's how I've been taught to do it, yeah, and in Java particularly it's helpful to match packages because then you can test package-private code when necessary by placing tests in the same package

[Levi]

Right. Never sure which Java things carry over to Kotlin since Kotlin is a lot less restrictive about package names, etc

[Levi]

But I think the organization makes it easier to navigate even if it isn't "needed"

[Shawn]

I guess the freedom is useful because you may have a need to group classes or files in a way that doesn't necessarily match package structure

[Levi]

right

[Shawn]

a naive example might be the ability to group all your integration tests together while still defining the tests themselves within the packages they're relevant to for access to symbols they wouldn't be able to see otherwise

[Levi]

I was also thinking of using the same package name for corresponding tests but appending .unit or .integration to split test types. Thoughts on that? Bad idea?

[Shawn]

I don't quite get what you're envisioning, but usually what I do is if I have a class Foo, my unit test is usually called FooTest and the integration test FooIntegrationTest

[Levi]

Sorry, let me clarify. I'm asking about (maybe conflating) both directories and package names (realizing that in Kotlin the dir/path does not need to match the package name for a particular class).

My src/main dir structure looks like this right now:

app/src/main/kotlin
├── Application.kt
├── auth0
│   ├── Auth0Client.kt
│   ├── Auth0UserType.kt
│   └── Token.kt
├── cache
│   └── ResidentCache.kt
├── kafka
│   ├── Consumer.kt
│   ├── Producer.kt
│   ├── Props.kt
│   ├── cache
│   │   └── KafkaResidentCache.kt
│   ├── model
│   │   ├── Resident.kt
│   │   ├── ResidentRegisteredEvent.kt
│   │   └── SignupUserData.kt
│   └── user
├── routing
│   ├── health
│   │   └── check
│   │       └── CheckHealthRoute.kt
│   └── user
│       ├── UserRoutes.kt
│       ├── authenticate
│       │   ├── A.kt
│       │   └── B.kt
│       ├── completeRegistration
│       │   ├── C.kt
│       │   └── D.kt
│       ├── sendOtpEmail
│       │   ├── Foo.kt
│       │   └── Bar.kt
│       ├── sendSmsOtp
│       │   ├── Foo.kt
│       │   └── Bar.kt
│       └── validateEmailOtp
│           ├── Foo.kt
│           └── Bar.kt
└── validation
    ├── Extensions.kt
    └── RequestValidations.kt

I have followed the convention here https://kotlinlang.org/docs/coding-conventions.html#directory-structure, i.e. omitting the common root where my root package is org.wol.useridentityservice. For example, the src files in the auth0 directory have package org.wol.useridentityservice.auth0 and so on.

Now I get to my test directory. Should the directory structure match main so that auth0 tests are in auth0 directy and the package would also be org.wol.useridentityservice.auth0?

[Levi]

Seems like it's as simple as matching the dir structure of main and using the same package name, like so

[Levi]

and then using TokenIntegrationTest to distinguish, as you suggest

[Levi]

That's a lot simpler than what I envisioned. I like it

[Shawn]

good stuff!

[Levi]

Thanks again, appreciate it

[Levi]

Having some trouble figuring out how to swap certain dependencies for test implementations for the purpose of integration testing. Specifically, my application uses EngineMain to start a server and in my Application.module() I set up some dependencies, such as an Auth0 client, kafka consumer, etc.

fun main(args: Array<String>) = io.ktor.server.netty.EngineMain.main(args)

fun Application.module() {
  val auth0Client = Auth0Client(environment.config)
  val residentCache = KafkaResidentCache()
  userRoutes(auth0Client, residentCache)
}

I'd like to write an integration test for my routes without relying on a live kafka broker or making requests to Auth0 (obviously). I was hoping I could have default dependencies that get passed all the way up from main over to the Application.module(), but I'm not sure if that's possible. I've seen some examples that seem a bit more flexible (e.g. this example where they have fun Application.main(httpClient: HttpClient = applicationHttpClient) which makes it easy to swap a client when testing), but still unclear what exactly this would look like in my case.

Is what I'm aiming for even the right goal? Should I instead be aiming to have an embedded Kafka (a la https://kotest.io/docs/extensions/embedded-kafka.html) or running Kafka in Docker so that the integration tests resemble the production scenario as close as is feasible?

[Levi]

I'll be damned, this actually works...I think 🤣

fun Application.module(auth0Client: Auth0Client = Auth0Client(environment.config),
                       residentCache: KafkaResidentCache = KafkaResidentCache()) {

[Levi]

Not sure why I assumed that wouldn't...