Does using `externalServices` to mock external services only kotlinlang #ktor

Does using `externalServices` to mock external ser...

Levi

12/21/2023, 4:04 PM

Does using

externalServices

to mock external services only work for client requests made directly with the client provided in

testApplication

? In other words, if in your test you call some of your application code which internally creates a client, will the requests made by that client not use the mock setup?

Aleksei Tirman [JB]

12/21/2023, 4:47 PM

Does using
externalServices
to mock external services only work for client requests made directly with the client provided in
testApplication
?

Yes and only with that client.

Levi

12/21/2023, 4:55 PM

Makes sense, thanks. Wasn't sure how much magic might be going on behind the scenes, but seems not much which is good 🙂 I'm having a bit of trouble testing my application then. Some of my routes take a dependency as an argument, in my case an

Auth0Client

. The

Auth0Client

, in turn, takes an

HttpClient

. I thought this would make it easy to test those routes by using the test client available in my test. I have routes:

Copy code

fun Application.userRoutes(auth0Client: Auth0Client, residentCache: KafkaResidentCache) {
  routing {
    route("/user") {
      sendOtpEmailRoute(auth0Client)
      validateEmailOtpRoute(auth0Client, residentCache)
      sendSmsOtpRoute(auth0Client)
      authenticateUserRoute(auth0Client)
      completeRegistrationRoute(auth0Client, residentCache)
    }
  }
}

and then my

main()

and `Application.module()`:

Copy code

fun main(args: Array<String>) = io.ktor.server.netty.EngineMain.main(args)

fun Application.module(auth0Client: Auth0Client = Auth0Client(environment.config),
                       residentCache: KafkaResidentCache = KafkaResidentCache()) {
  install(ContentNegotiation) { json() }
....
  userRoutes(auth0Client, residentCache)

}

where

Auth0Client

is:

Copy code

class Auth0Client(val auth0Domain: String, val clientId: String, val clientSecret: String, val client: HttpClient) {
  constructor(
    config: ApplicationConfig
  ) : this(
    config.property("auth0.domain").getString(),
    config.property("auth0.clientId").getString(),
    config.property("auth0.clientSecret").getString(),
    HttpClient(Apache5) {
      engine {
        followRedirects = true
        socketTimeout = SOCKET_TIMEOUT
        connectTimeout = CONNECTION_TIMEOUT
        connectionRequestTimeout = CONNECTION_REQUEST_TIMEOUT
      }
      install(Logging) {
        logger = Logger.DEFAULT
        level = LogLevel.ALL
      }
      install(ContentNegotiation) { json() }
    },
  )
...
...

I'm unable to figure out how to, in my test, pass in a test implementation of the

Auth0Client

which uses the test

client

so then I can test those routes without actually making calls to Auth0. I thought I could do something like

application { module(auth0Client, kafkaResidentCache) }

in my

testApplication

but that doesn't seem to work. Appreciate any guidance or advice you can offer.

Levi

12/21/2023, 4:57 PM

One of the issues I ran into when trying the approach of

application { module(auth0Client, kafkaResidentCache) }

was:

Copy code

io.ktor.server.application.DuplicatePluginException: Please make sure that you use unique name for the plugin and don't install it twice. Conflicting application plugin is already installed with the same key as `ContentNegotiation`

Levi

12/21/2023, 5:04 PM

Perhaps this all becomes easier to manage if I use

embeddedServer

Levi

12/21/2023, 5:28 PM

I see. Looks like the issue was that I retained the

Copy code

application {
        modules = [ org.wol.ApplicationKt.module ]
    }

which means when calling

application {...}

in my test that things get loaded/started twice

Levi

12/21/2023, 5:28 PM

So this does work with

embeddedServer

and removing that from

application.conf

, but I don't want to use

embeddedServer

😩

Levi

12/21/2023, 5:29 PM

Is there a way to use

embeddedServer

just for the purposes of tests? Or there is some other approach for easily being able to configure modules during tests but still using

EngineMain

AdamW

12/21/2023, 5:30 PM

My test setup is basically identical so that I pass

application { module(…) }

with test dependencies. I would try to debug that exception still 🤔 edit: whoops, you already figured it out

Levi

12/21/2023, 5:30 PM

Interesting, thanks. I was able to get rid of the exception, but so far only when switching to

embeddedServer

Levi

12/21/2023, 5:30 PM

hehe yeah

Levi

12/21/2023, 5:30 PM

Are you using

embeddedServer

AdamW

12/21/2023, 5:31 PM

Nope!

AdamW

12/21/2023, 5:31 PM

What does your testApplication { } block look like?

Levi

12/21/2023, 5:31 PM

blob thinking fast I'm doing something wrong then!

Levi

12/21/2023, 5:31 PM

One sec. Messy at the moment 🤣

Levi

12/21/2023, 5:37 PM

Copy code

testApplication {
        val host = "myhost"
        val client = createClient {
              install(ContentNegotiation) { json() }
              }
        val auth0Client = Auth0Client(host, "", "", client)
        application { module(auth0Client, kafkaResidentCache) }
        val expected = """{"key": "value"}"""
        externalServices {
            this@testApplication.install(io.ktor.server.plugins.contentnegotiation.ContentNegotiation) {
                json()
            }
            hosts("https://$host") {
                routing { post("passwordless/start") { call.respond(expected) } }
            }
        }
        val response =
          <http://client.post|client.post>("/user/otp-email") {
            // install(ContentNegotiation) { json() }
            contentType(ContentType.Application.Json)
            setBody("""{"email": "<mailto:someemail@example.com|someemail@example.com>"}""")
          }
        response.bodyAsText() shouldBe expected
      }

This works if I use

Copy code

fun main() {
    embeddedServer(Netty, port = 8081, module = Application::module).start(wait = true)
}

and remove the

modules = [...]

from my

application.conf

, but if I use:

Copy code

fun main(args: Array<String>) = io.ktor.server.netty.EngineMain.main(args)

it will fail due to

Copy code

io.ktor.server.application.DuplicatePluginException: Please make sure that you use unique name for the plugin and don't install it twice. Conflicting application plugin is already installed with the same key as `RequestValidation`

Levi

12/21/2023, 5:38 PM

Perhaps I can just provide a custom config for testing which removes the

modules = [...]

and that might resolve the issue

Levi

12/21/2023, 5:38 PM

¯\_(ツ)_/¯

AdamW

12/21/2023, 5:39 PM

If it’s at all feasible, try passing an

environment { }

by hand. In my case it’s actually desired, and seems to override the file-based config.

Levi

12/21/2023, 5:40 PM

Start with an empty maybe, like so?

Copy code

environment {
        config = MapApplicationConfig()
    }

👍 1

Levi

12/21/2023, 5:44 PM

Well I'll be damned...that does work! Thanks very much for chiming in, really appreciate it

Levi

12/21/2023, 5:44 PM

I think some of the stuff related to testing could be documented a lot better

Levi

12/21/2023, 5:45 PM

oh no..spoke too soon

Levi

12/21/2023, 5:45 PM

this only worked when I commented out

install(ContentNegotiation) { json() }

in my

Application.module()

. Doesn't work if I keep it in.

AdamW

12/21/2023, 5:45 PM

No problem! I just recently went through a similar hassle so at least it’s fresh in memory 😅

Levi

12/21/2023, 5:46 PM

I get:

Copy code

io.ktor.server.application.DuplicatePluginException: Please make sure that you use unique name for the plugin and don't install it twice. Conflicting application plugin is already installed with the same key as `ContentNegotiation`

AdamW

12/21/2023, 5:46 PM

Are you sure it’s the client install + plugin in your

testApplication

block?

Levi

12/21/2023, 5:46 PM

oh that might as simple as removing that plugin from the

externalServices

block

Levi

12/21/2023, 5:47 PM

That seems to resolve it

Levi

12/21/2023, 5:47 PM

I assumed I needed that in both my Application and the

externalServices

AdamW

12/21/2023, 5:47 PM

Oh yeah, I didn’t even see it at first. I’ve never seen

externalServices

before 😅

AdamW

12/21/2023, 5:48 PM

It could then be that you don’t need to pass the environment manually 🤔

Levi

12/21/2023, 5:49 PM

Yeah, that's here: https://ktor.io/docs/testing.html#external-services Not sure this is the best way to do things, but seems like an easy way to do test everything up until the external service

Levi

12/21/2023, 5:49 PM

True! lemme try that

Levi

12/21/2023, 5:50 PM

No dice, unfortunately. Will only work with the manual

environment

👍 1

Levi

12/21/2023, 5:51 PM

Well, this is still very good progress. I got a bunch of cleanup to do now. Thank you again!

AdamW

12/21/2023, 5:53 PM

No problem, I learned something new too 😅 Anyway, here’s the pattern I’m currently using to create a server in my tests: https://github.com/nomisRev/ktor-arrow-example/blob/main/src/test/kotlin/io/github/nomisrev/ktor.kt

Levi

12/21/2023, 5:54 PM

Awesome, going to take a close look. Thank you

Levi

12/21/2023, 5:54 PM

Looks like some good stuff in there to make things cleaner/more reusable.

👍 1

AdamW

12/21/2023, 5:57 PM

The only major difference is that I’m just creating a

HttpClient(MockEngine { … })

instead of using

createClient

. There’s a bit of annoying boilerplate mocking services this way, so I might look into

externalServices

Levi

12/21/2023, 5:57 PM

Right. I noticed that.

Levi

12/21/2023, 5:58 PM

Good news is that this exercise has forced me to rewrite my code in a way that makes it much, much more testable. Should've done TDD from the outset 🤣 I always end up thinking that, but still never do.

😂 1

Levi

12/21/2023, 9:04 PM

I like the

Dependencies

construct

Levi

12/21/2023, 9:05 PM

and the use of

resourceScope

from Arrow

267 Views

Open in Slack

Previous Next