Emil Kantis
09/25/2021, 9:24 AMcheckAll<Int> { it shouldBeGreaterThan 1000 }
Given the above, wouldn’t it make sense that shrinking should inform me that it is succeeding for 1000, failing for 999?bind
, but this left me a little confused 🙂 I actually think we construct the RTrees properly, but was a bit confused by the above. Writing a test similar to my example, the input was shrunk to 0 in all components immediately then failed. Iexpected it to try all values in the RTree 🙂mitch
09/26/2021, 1:10 PMIn property based testing, whenever a failure occurs the framework will try to reduce it to something smaller so that the end-user only has to cope with a very simple and small input. It is designed to help developers focusing on the real cause without having to manually investigate several potential sources of bugs.Let’s imagine there’s a test which has some arbs as an input and it fails on some sample of
<http://Arb.int|Arb.int>
. The framework realize that, hey maybe we’ll try int = 0
would that fail? It goes and rerun the test (i.e. the lambda inside our checkAll { ... }
. Does that fail? if it does, maybe looking for further more complex parameter isn’t too useful because 0 already gives a simple enough input to reproduce the test failure.
If we consider to look for the boundary case, we’re inverting the problem to an unbounded search. That is a very difficult task. I don’t know if there’s a guaranteed deterministic output out of it. I need to search for literatures that covers that.
There are several questions on that in general.. how might that search space look like? can that be defined for any type? What would be the risk for the test framework when dealing larger search space? For instance.. shrinks has a bounded search space, and even that we still have problems about cartesian product rerunning the test lambdas millions of times..
Now for the problem at hand: wishing to harvest the values in the rtree children, you might be interested in the following code
test("arbInt srhinking") {
val arb = <http://Arb.int|Arb.int>(0..100)
val shrinks = arb.sample(RandomSource.seeded(1234L)).shrinks
shrinks.value() shouldBe 54
shrinks.children.value.map { it.value() } shouldContainExactly listOf(
0,
1,
18,
27,
36,
49,
50,
51,
52,
53
)
}
Emil Kantis
09/26/2021, 3:00 PMa
, and succeeding input b
, then we could shrink from a
towards b
. Or vice versa, if a
is smaller than b
sam
09/26/2021, 3:14 PMEmil Kantis
09/26/2021, 4:13 PMS_f
, it would be impossible to search upwards. However, if we have at least one succeeding sample S_s
, we can use that to constrain the problem, right? If S_s > S_f
, We can shrink the S_s
sample using S_f
as lower bound. If S_f > S_s
, can shrink S_f
with S_s
as lower bound. In both cases, we have a finite search space that should lead us to a boundary between success and failure.
I think I need more practical experience with property testing to reason about it properly.. 🙂sam
09/26/2021, 4:33 PMmitch
09/26/2021, 9:40 PMsam
09/26/2021, 9:41 PM