https://kotlinlang.org logo
Title
d

dave08

05/20/2021, 12:07 PM
This is a very nice feature in jqwik: https://jqwik.net/docs/current/user-guide.html#collecting-and-reporting-statistics also the way assumptions return an error if too many possibilities were skipped: https://jqwik.net/docs/current/user-guide.html#assumptions and in general, having some stats printed like there would add tons of value to property testing in kotest... it would make it much easier to make tests.
j

Jonathan Olsson

05/20/2021, 12:17 PM
Yeah, they have similar stuff in eqc and proper in the erlang world. I assume jqwik as well has the possibility to attribute weights to the different actions as well (i.e. if one action due to preconditions gets executed too infrequently)?
Only having looked superficially, jqwik appears pretty full featured and competent. But I really don't like the annotations. 😄
d

dave08

05/20/2021, 12:20 PM
I'm not really familiar with jqwik, I was just looking through the docs after you mentioned it, and I found some features that would be GREAT in kotest... but I don't like more than just the annotations there... it's just not Kotlinish... and the issue for Kotlin support's been around for a few months with no love...
The fact that they require using their runner for the tests is a big 👎🏼 for me. I like Kotest for a bunch of reasons and wouldn't abandon it too quickly
There seems to be
classify()
and
checkCoverage()
for assumptions already 🤔
But I wonder if it can print that out without the check coverage...
s

sam

05/20/2021, 1:30 PM
What are you looking for in reporting that classify doesn't already do
d

dave08

05/20/2021, 1:34 PM
Say this example from jqwik:
@Property
void simpleStats(@ForAll RoundingMode mode) {
    Statistics.collect(mode);
}
will create an output similar to that:

[MyTest:simpleStats] (1000) statistics = 
    FLOOR       (158) : 16 %
    HALF_EVEN   (135) : 14 %
    DOWN        (126) : 13 %
    UP          (120) : 12 %
    HALF_UP     (118) : 12 %
    CEILING     (117) : 12 %
    UNNECESSARY (117) : 12 %
    HALF_DOWN   (109) : 11 %
s

sam

05/20/2021, 1:34 PM
So you're looking for automatic labelling
I assume that only works on a couple of types
d

dave08

05/20/2021, 1:34 PM
It gives a nice report, not only when checking coverage... and automatic labelling is a plus...
s

sam

05/20/2021, 1:35 PM
Easy enough to do for numbers and enums
d

dave08

05/20/2021, 1:36 PM
That way, when making the tests, one can ensure that the distribution is enough to cover all cases
Automatic labelling would make it even easier...
s

sam

05/20/2021, 1:37 PM
I'm not sure i see the benefit. Is it to check the platforms random is actually random
Also kotest has this great feature of edgecases that other prop tests don't have
We can add it though
d

dave08

05/20/2021, 1:40 PM
Is it to check the platforms random is actually random
No, rather to make sure that filtering and all other steps produce a reasonable amount of variants. The programmer's errors, not the framework. And edgecases need to be contrived, and when you don't know when they're already covered by the random ones sufficiently, it's not so easy to do in more complicated setups.
s

sam

05/20/2021, 1:42 PM
Ok
I'll put it in the roadmap
d

dave08

05/20/2021, 1:46 PM
Thanks 😉, I looked up alternatives, you're really doing great work with Kotest! I hope all these requests are OK, I guess I could do it on my end using the
PropertyContext
I just supposed others could benefit from this... I would make a PR, but I'm not familiar with MPP and how you would like this to be done.
1
I guess the easiest way to do it would be something like:
fun PropertyContext.printResult() {
        val attempts =  attempts().toDouble()

        classifications().forEach { labelled, occurences ->
            val percent = (occurences.toDouble() / attempts) * 100.0
            println("$labelled ($occurences): ${percent.roundToInt()} %")
        }
    }
s

sam

05/20/2021, 2:03 PM
The more requests the better. Keep em coming
👍🏼 2
d

dave08

05/20/2021, 2:04 PM
And then prepend it to checkAll {}.printResult()
And classifying enums:
inline fun <reified T : Enum<T>> PropertyContext.classify(label: T) {
    classify(label.name)
}
... the wonders of kotlin extension functions 😁. I made my own classifiers too for domain specific uses, this might be nice to document!
👍🏻 1
s

sam

08/21/2021, 10:26 PM
d

dave08

08/22/2021, 11:11 AM
Very nice 👌🏼!
:plus1: 1