https://kotlinlang.org logo
Title
v

Vidar Magnusson

04/11/2023, 11:41 AM
Hello everyone! I hope this is the right channel for this question. My team and I are currently looking to replace our framework for open API / swagger schema generation and possibly more broadly our REST framework entirely. We are currently using http4k for both purposes and have been using it for years. However, we have had quite some issues with it and specifically we have realized that it might be difficult or even impossible for us to meet new demands on our swagger schema generation with http4k. A part from http4k we are aware of spring/spring-boot which we'd preferably avoid. We have also investigated using ktor but it also seems insuficcient. This leads me to my questions: What frameworks do people here use for HTTP/Rest? If and how are you handling swagger/openapi schema generation? What benefits/problems do you see with your current setups?
j

jaguililla

04/11/2023, 12:02 PM
Hello, may I ask... which problems are you experiencing?
v

Vidar Magnusson

04/11/2023, 12:04 PM
Hello! With http4k specifically? It's mostly related to the swagger generation, especially with the new requirements I mentioned we need to have quite detailed control over the generated openAPI schema. For example generating multiple response examples using contracts.
j

jaguililla

04/11/2023, 12:06 PM
Ok... did you shared your issues with HTTP4K creators? may be they can offer you help to achieve your goals
d

dave

04/11/2023, 12:07 PM
afaik http4k is the only lib which generates the schema from the code, so you're into manually generating the endpoints from the spec. Not sure because I stopped looking ages ago... Does the Ktor one use code?
j

jaguililla

04/11/2023, 12:10 PM
I'm maintaining another toolkit, and OpenAPI is in my todo list, so understanding your issue could be of great help to me... Thats why I was asking
d

dave

04/11/2023, 12:10 PM
There are some ways around the polymorphic response types, but they're not very neat. The system does have limitations for sure
a

Adam S

04/11/2023, 12:11 PM
since you’re asking in the Kotlin Slack I suspect these are the answers you’re going to get: • Don’t use OpenAPI, use Kotlin/Multiplatform • http4kKtor IntelliJ plugin • there are a few Ktor plugins https://github.com/papsign/Ktor-OpenAPI-Generator https://github.com/nielsfalk/ktor-swagger https://github.com/bkbnio/kompendiumSpring Boot SpringDockxs-ts-gen (disclaimer, I’m the author) As far as I’m aware there’s not really a perfect solution. These all have pros/cons so it really depends on your situation. Personally I’d like to go a different direction: don’t generate the OpenAPI spec, but instead use it as a test-input for https://schemathesis.readthedocs.io/en/stable/. This means full control over the API spec, and it’s always tested.
d

dave

04/11/2023, 12:11 PM
@jaguililla good luck... Implementation of the whole spec is pretty hairy. It's my least favourite part of the entire http4k codebase
v

Vidar Magnusson

04/11/2023, 12:12 PM
I have not been in contact with the http4k creators, as mentioned we have been using it for a while and it has mostly worked very well for our purposes. However, we now have the possibility of doing a larger overhaul of our systems and are therefore looking for what options exists today before just continuing as we have. We have not yet determined that it is impossible to continue with http4k just that it will not be straight forward.
d

dave

04/11/2023, 12:12 PM
@Adam S you definitely need to maintain parity - one of the things that made us go the other way was the hassle of keeping things up to date
Of course, it doesn't help that the swagger ui doesn't even support V3 yet... And it's been out 2 years!
(and has been acquired by smartbear so no idea why it's not seen as a must have)
a

Adam S

04/11/2023, 12:13 PM
I had a vague idea of re-implementing Schemathesis in Kotest, since Kotest is great for PBT https://github.com/kotest/kotest/issues/2952
j

jaguililla

04/11/2023, 12:15 PM
Thanks @dave I've been thinking on generating the spec from a mix of the router and the tests, but it is tricky to get something simple for the user 😕
a

Adam S

04/11/2023, 12:15 PM
and there are other OpenAPI checkers too, e.g. https://github.com/zalando/zally
d

dave

04/11/2023, 12:15 PM
I think springfox was abandoned last time I checked.
It is definitely a hard problem 🤣
p

Philipp Mayer

04/11/2023, 12:16 PM
Springfox is a pita tbh - we’re using it and I can’t recommend it.
v

Vidar Magnusson

04/11/2023, 12:17 PM
Hi @dave btw! I want to emphasize that we overall really like the library! Our problem mostly comes from quite strict API guidelines that we need to adhere to where most other teams are using different languages. We're sort of in a situation of having to force whatever library we use to follow our requirements.
d

dave

04/11/2023, 12:19 PM
@Vidar Magnusson of course! You need to use something that fits your needs, and an overhaul of the contract stuff is definitely not on the cards right now. But thanks anyway 🙃
a

Adam S

04/11/2023, 12:19 PM
an alternative is to drop the REST API and move to GraphQL(!)
d

dave

04/11/2023, 12:20 PM
The state of the open API tech is another reason that we went for a custom solution with the http4k toolbox - the generators are just glorified templating engines
v

Vidar Magnusson

04/11/2023, 12:21 PM
@Adam S that was a discussion but for a great many reasons it's been postponed and for the moment all of our focus is on REST
d

dave

04/11/2023, 12:21 PM
Graphql is another fire that you might want to look twice at before leaping into. 😉
s

sciack

04/11/2023, 12:25 PM
Quarkus (and smallrye) could work (https://quarkus.io/guides/openapi-swaggerui#expose-openapi-specifications) but not sure if it's an improvement, you need to verify all the options.
v

Vidar Magnusson

04/11/2023, 12:26 PM
Yes any suggestions are appreciated!
j

jaguililla

04/11/2023, 12:27 PM
If you have very specific requirements, one thing you could do is to use your tools to generate the base spec, and later postprocess it with a custom program written by yourself
s

sciack

04/11/2023, 12:28 PM
or you go on the other direction and start from the schema, but create overhead to keep in sync with the code.
So you need to develop suite of test to verify that everything is in sync
j

jaguililla

04/11/2023, 12:29 PM
I would create an http client to store the messages sent to routes on the integration tests in files, and merge that on the schema generated from the code
v

Vidar Magnusson

04/11/2023, 12:29 PM
We have considered that, especially modifying the http4k schema looks like a strong possibility right now. We have also thought about doing a completely inhouse solution but would prefer to avoid it if at all possible.
c

christophsturm

04/11/2023, 12:30 PM
this will not help you find your server framework but for openapi code generation i recommend https://github.com/cjbooms/fabrikt
s

sciack

04/11/2023, 12:30 PM
and simplify the requirements? 🙂
a

Adam S

04/11/2023, 12:31 PM
with the http4k generator, what specifically works well, and what would you want to improve? The routes, response/request bodies, headers…?
c

christophsturm

04/11/2023, 12:32 PM
I think you should decouple the openapi stuff from the server framework, use fabrikt, for openapi, and any server you want. ktor is probably most common
v

Vidar Magnusson

04/11/2023, 12:34 PM
and simplify the requirements?
Likely not gonna happen and also especially with examples we see how we can assist the users of our APIs greately by following the requirements so we are very motivated to be on par with our colleagues not using Kotlin.
with the http4k generator, what specifically works well, and what would you want to improve? The routes, response/request bodies, headers…?
Most of our issues are with response schemas and examples for them.
I think you should decouple the openapi stuff from the server framework, use fabrikt, for openapi, and any server you want. ktor is probably most common
I have not looked at fabrikt previously but from reading the github page it appears to be about generating kotlin code from OpenAPI schemas? We're looking to do the opposit 😛
c

Cies

04/11/2023, 12:47 PM
you probably know about it, but if you want to expose some db tables by REST (or GraphQL) there exist many tools for that nowadays (Hasura, PostgREST, etc, etc.).
j

jaguililla

04/11/2023, 12:50 PM
Are you using data classes to for the requests and responses?
v

Vidar Magnusson

04/11/2023, 12:50 PM
@Cies Yeah unfortunately we need to have quite a bit of logic in between but good point! 🙂
@jaguililla yes we are
j

jaguililla

04/11/2023, 12:51 PM
And you would like to reuse their KDoc comments?
v

Vidar Magnusson

04/11/2023, 12:53 PM
We don't really use kdoc at all. 😛 Http4k allows us to provide instances of the data classes to be used as examples, which works for most cases.
c

christophsturm

04/11/2023, 12:54 PM
I have not looked at fabrikt previously but from reading the github page it appears to be about generating kotlin code from OpenAPI schemas? We’re looking to do the opposit
imo the only way to get decent openapi support is with a handwritten openapi file. that way you can support all features and can never come into the situation where you need to do changes that are not supported by your tool.
j

jaguililla

04/11/2023, 1:01 PM
Could you share one case in which data class examples is not enough for you?
v

Vidar Magnusson

04/11/2023, 1:02 PM
Fair point, I have not tried that approach myself and I see that it if it works well it could be very useful. I will bring it up to the team but I'm hesitant as I believe that our focus should be firstly having the code look/work the way we want and secondly to have the documentation look the way we want.
@jaguililla we have cases where we need to have multiple response types for the same status code (not my idea) which is possible in openAPI but not straight forward with http4k.
d

dave

04/11/2023, 1:05 PM
@Vidar Magnusson It is possible by using custom definition prefixes for each example. That generates the oneOf that you want for the same response code
The downside is that you need to model your kotlin classes to be an exact replica of the case - this is doable with sealed class hierarchies - so it will affect your models. The current http4k generator does not cope with fields BEING null (although it does support the optionality through nullable types
v

Vidar Magnusson

04/11/2023, 1:07 PM
yes we did run in to the null issues but have mostly just accepted that 😅
d

dave

04/11/2023, 1:07 PM
they're not really "issues" per say - it's by design - it's to do with the generator not being able to create the docs if the value is null.
anyway, the overall effect on the OpenApi spec is that you get one definition for each example response
which isn't amazing, but does work. We use the openapi definitions to generate typesafe typescript models with the exact response requirements for each case. it works pretty well TBH - or at least better than other things I've used
Anyway - if you do end up detaching your server from your openapi specs, you would be able to choose whatever tech you want at the back end... that's the upside of the effort that you'll need to put in 🙂
v

Vidar Magnusson

04/11/2023, 1:10 PM
@dave It's not quite oneOf that we are looking for, I must admit I do not recall exactly what the problem with that was as a colleague was the one who looked into that. We are trying to replicate what a related team has achieved in .NET which solved the problems with a inheritance based solution in the open API schema. I can see if I can find some example
d

dave

04/11/2023, 1:10 PM
BTW - if you want to record traffic to disk, there's already something inside the http4k-core to act as a proxy and record/replay the traffic to a store (disk/S3 ...)
(look for
TrafficFilters
)
b

Bence Erős

04/11/2023, 1:57 PM
My recommendation regarding schema generation is NOT generating it. Simply because json schema is so different conceptually from the type systems of programming languages that metadata will be either lost in transit (eg. inheritance is difficult) , or you end up with using only a small subset of json schema, which means you miss a lot of opportunities. With my team we take a contract-first approach, first hand-writing an openapi yaml in design phase, which is an agreement between the producer and consumer, before writing any implementation code. Then we incorporate the openapi yaml into our test infrastructure to verify the implementation conforms to the contract. More about it here : https://hazelcast.com/blog/contract-first-development-using-restassured-and-openapi/
v

Vidar Magnusson

04/11/2023, 2:16 PM
Maybe I'm just weird but every time I've written swagger schemas by hand I've always had a horrible experience and continuously just thought to myself that "this was supposed to be generated by a computer". Do you use any tools for writing the schema or do you just go ahead in vscode or w/e?
d

dave

04/11/2023, 2:25 PM
@Vidar Magnusson you're not weird. - it's definitely laborious - I've had the same experiences, and you definitely need to have something to back it up because otherwise you're going to be in a world of pain! But you are getting something for the effort and that is the flexibility that @Bence Erős has written about. It's all a tradeoff really between effort and convenience (isn't everything?!?! 😂 ) - every project's situation is different and you just have to apply judgement about what your team is comfortable with.
b

Bence Erős

04/11/2023, 2:26 PM
I don't mind writing schemas by hand. But I wrote 2 json schema validators so maybe I'm a little bit too much into it :D anyways, extracting schemas under #/components/schemas and using them with $refs helps a lot with maintainability
a

Adam S

04/11/2023, 2:30 PM
in a previous assignment I set up a Gradle task that would run the same OpenAPI-to-TypeScript converter as the frontend. That at least shifted a lot of the basic problems ‘to the left’, as they say.
it can help writing OpenAPI if everyone is on board with the basics of API design (whatever you decide they might be!). In previous teams we’ve followed the Zalando guidelines, because they’re comprehensive and clear https://opensource.zalando.com/restful-api-guidelines/
d

dave

04/11/2023, 2:34 PM
@Adam S that's a great list. thanks!
v

Vidar Magnusson

04/11/2023, 2:36 PM
Yeah I've seen the zalando guidelines before, I also quite like the swedish governments API guidelines (unfortunately only available in swedish AFAIK): https://dev.dataportal.se/rest-api-profil However, my company have their own guidelines which unfortunately is quite different to both of these.