Hi, does anyone have experience of testing an OpenAPI spec using Kotest? I'm asking because I'm ver...

# kotest

[Adam S]

Hi, does anyone have experience of testing an OpenAPI spec using Kotest?

I'm asking because I'm very tempted to start creating a Kotest version of Schemathesis, which I've used before to great effect. I'd like a Kotlin version though!

I think a Kotest 'OpenAPI' module would work really well. It would parse an OpenAPI spec (Swagger provides a Java parser), and generate random requests, using Arbs. So if a specific response isn't documented, or a parameter marked as 'optional' actually isn't, then the test would fail.

Stateful testing (e.g. make sure that after a successful POST /user/create that GET /user/{id} works) is possible too.

The benefits of this would be
• the OpenAPI spec drives the testing, and would be completely separate from the codebase (no need for spec or code generation, which is always limiting)
• there's no limitation or restriction on how to implement the API (for example Hikaku doesn't handle many custom implementations)
• because the spec can be tested, it would always be up-to-date.
I've attached a mock code example. I might work on a proof-of-concept...

Untitled.cpp

[sam]

That sounds awesome. So you'd just plummel endpoints, trying optional / non optional query params, making sure 200s come back ?

[Adam S]

basically, yes. But not just 200s, because the OpenAPI spec documents all valid response codes and response values.

The real improvement is stateful testing, which Schemathesis implements. In OpenAPI you can specifically document POST /user/create returns some ID, that can be used in GET /user/{id}. So that can be tested!

[sam]

Sounds like a great idea that fits in really well with the theme of property testing

[Adam S]

Cool! I'll make a ticket on GitHub

[Adam S]

https://github.com/kotest/kotest/issues/2952