For a kotlin code-gen tool, any recommendation for assertion library for writing expectations on gen...

# announcements

[Kunal Dabir]

For a kotlin code-gen tool, any recommendation for assertion library for writing expectations on generated code? I am inclining towards some kind of approval testing framework in Kotlin, any recommended testing-library?

I prefer to avoid going via verifying AST nodes route as it gets very verbose. FWIW, We are using kotlinpoet to generate code.

[Matteo Mirk]

I’m in a similar situation, only in Java. I don’t know what you mean by “approval testing framework”, but the easiest route I found is to check the generated source with unit tests. You can use Kotest or Spek for the job, which have everything you need included. If you need more assertions or prefer an alternate lib for those, you can add one of these for example:
• https://github.com/robstoll/atrium
• https://strikt.io/

[Kunal Dabir]

I am already using Kotest and currently reading files from test/resources. However updating the expectation files is cumbersome. Approval testing is a technique to verify the snapshot, you can get a glimpse here. https://approvaltests.com/

[Matteo Mirk]

Oh never heard of it, thanks!

[Matteo Mirk]

Okay so if I get it right you have several resources that act as the expectations and you assert that the generated source matches them. I can see why that is cumbersome, we have the same problem at my workplace with mock service responses: lots of JSON files that need to be maintained.
But in your case, I can suggest that what I did to test source generation was just to assert on relevant bits or characteristics of the source. This can be accomplished with substring or pattern matching, in this way I don’t care about all the boilerplate and formatting.

[Kunal Dabir]

The problem with substring matching is writing that in code is tricky, with lot of escapes characters and exact indentation and wrapping (imagine long constructor or function argument list).

[Kunal Dabir]

with JSON, we can parse them and assert on relevant section. With code this can be done with AST parsing, which again is not super easy to set matcher expectations on (assertions).

[Matteo Mirk]

Substring is just for simple cases, like I want to check that return count + 1 is in the output, for the rest you can use regexes. Escaping isn’t a problem anymore if you use raw strings: """...""". My rationale for this is to avoid useless testing, that is: when I have to craft an expected output with all the formatting, I’m basically testing that KotlinPoet does its job, but that’s redundant since the lib has already been tested. What I want to test are the relevant parts of the TypeSpec I’ve built that will produce the source I want.

[Kunal Dabir]

We are already using both substrings and file based expectations 🙂 but they are easy to write once but brittle to update. I was just hoping for a better way to assert on kotlin syntax.

[Kunal Dabir]

multiline literal string don’t match with generated code unless matched exactly as formated. We typically write assertions like
generatedCode shouldContain "data class Foo"
but the moment you add constructor params etc, it starts getting tricky

[Matteo Mirk]

If you use regular expressions you can ignore exact formatting, am I missing something from your setup?

[Kunal Dabir]

it’s not about whether it is possible, it just just setting exceptions on generated code via regex is quite brittle

[Kunal Dabir]

I feel approval testing is better suited so that we can view a diff (compare 2 versions) side by side

[glureau]

Hello @Kunal Dabir! I'm searching for an approval testing library for KMP, did you find something nice? (I'll need to make it work for Android and iOS platforms if possible)

[Kunal Dabir]

No succes @glureau 😞

[Matteo Mirk]

Kunal, what libraries or method did you end up using?