I wonder if there are any plans in the near future to allow

Question

I wonder if there are any plans in the near future to allow given and when and then and tests at the same nesting level I think this was discussed before as not being easy but it keeps on eating at me how much nicer the tests could look if it would be possible

sam · Answer

Example

dave08 · Answer

Like ```given and when and ```

dave08 · Answer

The chaining could give a hint to the framework which level we re holding in and if it s a leaf or a test

sam · Answer

And would these nest in output like currently or be consided sibling tests

dave08 · Answer

Well it would be better the latter but a bit hacky but the former would still avoid the nested testing code which is already a big improvement it might not always make sense in the output though

sam · Answer

The latter is easy to do without the dot You could just list tests vertically

dave08 · Answer

I wonder how Spock does it I never used it but it has it all on one level

sam · Answer

Spock is able to use groovys dynamic nature for these tricks

dave08 · Answer

But that would mess up the class setup to list them vertically

sam · Answer

But then it means using groovy wink

sam · Answer

Would it

sam · Answer

Is it much different to what you pasted

dave08 · Answer

Well what I pasted could potentially save some kind of fixture object that is passed through the chain Which would allow the setup of the test

dave08 · Answer

Class level objects could always be used but they re a bit more dangerous we d need to assure that they re always reset

sam · Answer

ah ok

dave08 · Answer

Minutest did it that way but they don t have all the features that kotest has <https github com dmcg minutest structure tests>

sam · Answer

I was thinking something like ```val fixture = given qweqwe when qeqw then werewr ```

dave08 · Answer

That puts the fixture outside the given especially when it s immutable then the given block might end up being only half the setup

dave08 · Answer

There might even be a provided class in the when context for setting a `result` and a provided one in the given for setting some fixture objects ```given qweqwe this is a GivenContext with a var for fixtures fixture 1 = RepoStub fixture 0 = SutClass fixture 1 when qeqw this is a WhenContext with the fixtures in GivenContext and a result var result = fixture 0 methodUnderTest then werewr this has all the above result shouldBe fixture 1 stubValue ``` maybe something like this

dave08 · Answer

`fixture 0 ` and `fixture 1 ` might not be so nice though but there might be better ways

sam · Answer

We could probably pass out through like you want

sam · Answer

Can you create a ticket and I ll experiment

dave08 · Answer

Sure thanks

dave08 · Answer

Here it is <https github com kotest kotest issues 2209>

sam · Answer

Great thanks

christophsturm · Answer

can you show some examples of tests that use this syntax and show why its better

dave08 · Answer

I have tests with complicated situations and interactions more like legacy code that I need to test and refactor and having all the setup description on one `given` line or a bunch of nested `and`s gets pretty messy This proposition wouldn t be a replacement for the current BehaviourSpec but rather an optional addition to it < christophsturm>

dave08 · Answer

I guess it s less for code built with SRP in mind

christophsturm · Answer

ah thats interesting I m thinking a lot about what the perfect testing syntax looks like but i guess i have only been thinking of tests for well structured code

sam · Answer

It gets tricky when a project grows because everyone has their own preferred syntax and it s a balancing act of keeping everyone happy vs kitchen sink of features

dave08 · Answer

Yup I was hesitant to propose this blush but when I found myself thinking about it in a bunch of occasions I wondered if others might also need such a thing Legacy spaghetti code can lead to a complex test suite as it is until slowly refactored

dave08 · Answer

But I wouldn t mind if there was a better idea for this I personally have a conflict whether everything should be spelled out in the test descriptions or some things could count on the test code to show

christophsturm · Answer

one problem is that if you have all those features to support your old legacy code people will not know what features are ok to use for new code

christophsturm · Answer

so they will keep writing too complicated tests even for new features

dave08 · Answer

It s not many features it s just a certain process that too many components were handling differently and that currently has too many possible states that have to be handled

sam · Answer

I m not sure Dave s proposal is even possible to support Because a test in kotest is a function TestContext gt Unit So you need to pass the test context into the next builder but you can only do that when you re inside the lambda So chaining these calls might not be possible

sam · Answer

But I will explore it

christophsturm · Answer

ah sorry i was not really talking about your proposal but in general for me the perfect test runner has not many features and just one testing style

sam · Answer

Just wait until someone asks for a feature in failfast that you don t need Do you add it to expand the functionality or do you say no slightly smiling face

christophsturm · Answer

nobody uses it anyway so thats not going to happen slightly smiling face

sam · Answer

well you never know

dave08 · Answer

Yeah I guess kotest already has quite a few things that you don t personally ever use < sam> sweat smile

sam · Answer

absolutely if I had my way I d delete behavior spec I think its awful joy

dave08 · Answer

So what do you use

sam · Answer

I only ever use FunSpec

dave08 · Answer

Wow not even FreeSpec

sam · Answer

nope

christophsturm · Answer

I didnt know that you know about failfast sam did you look at it

sam · Answer

I did

christophsturm · Answer

I did not want to advertise it here and also I want to make sure it works for all my use cases before advertising it more

sam · Answer

Feel free to advertise it here I m not precious about it If it offers some functionality for people that kotest doesn t then great that helps kotlin in general

christophsturm · Answer

it sure must look like i have severe nih syndrome I started writing an orm then i thought the tests run not fast enough then i started writing a test runner and this weekend i started writing a rest web server because i was frustrated with ktor and http4k does not support coroutines

sam · Answer

http4k not supporting coroutines is incredibly frustrating I would use it tomorrow if it had coroutine support but they seem to think it doesn t matter if you follow the threads no pun intended I think that s mistake personally

christophsturm · Answer

hehe

sam · Answer

It seems easy to make a test framework and when it s focused on your own use cases it probably is easy There s some bugs that people bring up time after time and they re out of kotest s control The classic one about parent tests being included in test counts that s just how gradle works And gradle has bugs in its testing implementation that have remained open for years

christophsturm · Answer

in my web server handlers look like this ```class UserService RestService suspend fun create user User User delay 1 return user copy id = userId suspend fun show userId Int User delay 1 return User id = userId toString name = User $userId ``` that fits 99 of the code that i write and makes it really easy to test

christophsturm · Answer

in failfast i now support junit engine for idea support but in gradle i still only run the tests as a main method

sam · Answer

The problem with NIH is that it s not so bad doing the simple stuff but when you need some rock solid handling of url encoding or websockets or whatever you re really going to spend a lot of time on what has been done before

christophsturm · Answer

hmm yeah I did not really write it from scratch i use undertow like i did in my first kotlin web projects

sam · Answer

undertow is good I built some scala stuff on it ages ago

christophsturm · Answer

it seems that it does not have much momentum all the redhat server side stuff seems to focus on quarkus now

christophsturm · Answer

but it still gets security updates and has no bugs that bother me

christophsturm · Answer

so i thought i start with something i know

sam · Answer

what didn t you like about ktor

christophsturm · Answer

error handling sometimes things just dont work and the error message is not useful and trying to debug it is very complicated

sam · Answer

yeah for sure the phase install parts of it are odd

christophsturm · Answer

it does not feel robust at all only the happy path is really tested

sam · Answer

I know we had an issue upgrading from 1 4 2 to 1 5 0 which broke the CORS support

sam · Answer

had to roll back

christophsturm · Answer

when you sent an authorization header to it that is misspelled you get an internal server error with a huge stacktrace that says nothing about the problem or if you send json that does not have the expected format thats both use cases that happen really often

christophsturm · Answer

I want small libs with good testing coverage

sam · Answer

tested with failfast of course slightly smiling face

christophsturm · Answer

or kotest

christophsturm · Answer

so what did you think of failfast

sam · Answer

very nicely implemented

sam · Answer

I liked the configuring which tests to run programatically That sits nicely with my functional programming mindset

sam · Answer

I didn t delve deeply enough to look at the auto test stuff but I like that idea SBT in scalaland does a good job of that and I quite liked it there

christophsturm · Answer

oh nice thank you

christophsturm · Answer

the autotest stuff does not yet work as well as it should but i did find out that you can find dependent classes by reading just the beginning of the class file constant pool

christophsturm · Answer

i thought that it would be enought to just run updated class files because kotlin would recompile tests that use a class when the class changes but kotlin is too clever for that and recompiles only when the interface changes

sam · Answer

look at classgraph

christophsturm · Answer

yeah classgraph can do it and seems to be really fast

sam · Answer

yeah it can scan quickly

christophsturm · Answer

i just try to keep the code that runs single threaded really small and with no dependencies

christophsturm · Answer

as soon as i have found a single test to run its much more relaxed but i want all cores to work from the beginning

sam · Answer

when you get it working flawlessly do a PR to kotest slightly smiling face

christophsturm · Answer

where would autotest fit best into the kotest architecture

sam · Answer

would have to make a gradle task that spins up and keeps running I guess

sam · Answer

that s how SBT does it

christophsturm · Answer

ah right so the idea plugin and the gradle plugin could support it

sam · Answer

yeah

sam · Answer

could be added here <https github com kotest kotest gradle plugin>

sam · Answer

kotest gradle plugin that doesn t need junit at all

christophsturm · Answer

i think if it just works in idea and works there perfectly that would be enough

sam · Answer

yeah probably

sam · Answer

then maybe it could be added to the intellij plugin for kotest

christophsturm · Answer

I started working on an idea plugin for failfast but there seems to be zero documentation

christophsturm · Answer

the best documentation for creating a test runner plugin seems to be the kotest plugin

sam · Answer

lol

sam · Answer

probably is yes

sam · Answer

I spent many a night pouring over that

sam · Answer

You are putting a lot of effort into recreating the entire kotest ecosystem slightly smiling face

christophsturm · Answer

it sure looks that way I just wanted to iterate fast and build exactly the syntax that i wanted so starting from scratch seemed to be the best solution

sam · Answer

You will have learned a lot about how junit platform gradle etc works while doing that so feel free to hop in and contribute to kotest in any way you want

christophsturm · Answer

i did find a kotest bug that i did not report yet it seems the pitest plugin has a dependency on pitest and that leads to using an older pitest version and you cannot override it in gradle

sam · Answer

seems more like a gradle bug gradle should pick whatever version is latest

sam · Answer

I have seen this behavior in gradle quite a bit though the `implementation` thing doesn t seem super robust

sam · Answer

It might be related to being a multi platform project

christophsturm · Answer

i think the dependency should just be compileOnly

sam · Answer

sam · Answer

I need to move pitest to its own repo soon so I will do it then before 4 5 release

christophsturm · Answer

lol sorry i should have just created a ticket

sam · Answer

that s fine took me 2 seconds

sam · Answer

I wonder if you took the KotestEngineLauncher programatically registered tests like in failfast if it would run the tests fast enough for you

sam · Answer

```KotestEngineLauncher withSpec DummySpec1 class withSpec DummySpec2 class launch ```

christophsturm · Answer

i was thinking of a queue based design where the discovery starts putting tests into the queue and execution starts as soon as there is one test in the queue

sam · Answer

might save you a second depends how slow discovery is

christophsturm · Answer

the motto of failfast is with enough cores any test suite can run in one second

sam · Answer

lol well Kotest has 4000+ tests

sam · Answer

good luck running that in lt 1 second

christophsturm · Answer

yeah its more theoretical and it only works if you have a lot of short tests

christophsturm · Answer

my orm test suite breaks that expectation and takes 10 seconds

sam · Answer

right

sam · Answer

there s some tests in kotest that are testing Thread sleeps so they ll never run fast

christophsturm · Answer

why do they do that

christophsturm · Answer

coroutines are really great for multi threaded test running if a test needs a dependency that takes some time to spin up i do it in a separate io scheduler to make the cpu core available to a different test

sam · Answer

to test concurrency works for both suspension and blocking

sam · Answer

saying you should be using coroutines is not adequate for people who can t don t use suspension

christophsturm · Answer

sure

sam · Answer

so for things like the timeout functionality you want to test that a timeout works for both blocking and suspension

sam · Answer

``` Test config timeout = 300 milliseconds Thread sleep 100 ```

christophsturm · Answer

you will always have a smoke test suite that is slower

sam · Answer

``` Test 2 config timeout = 300 milliseconds Thread sleep 500 should fail ```

christophsturm · Answer

hmm tests like that only work single threaded or the first test will also fail if you have too much load

sam · Answer

yes so you need to put in a bit of wiggle room

sam · Answer

that s why I don t do ``` Test config timeout = 5 milliseconds Thread sleep 4 ```