https://kotlinlang.org logo
#kotest
Title
# kotest
j

James Eschner

02/21/2023, 3:21 AM
Hello all, question about Arbs. I have a situation where my code is trying to generate an impossible-to-generate arb. To show a simple example:
Copy code
data class Foo(val x: Boolean)

val fooArb: Arb<Foo> = arbitrary {
    Foo(x = false)
}

val stuck = fooArb.filter { it.x }.next()
Now, obviously this won't work. This is a simple example but in my case, the arb is much more complex (and transient) so the situation wasn't immediately obviously. What I would have hoped would happen is that the program would eventually time out or throw trying to generate the "next" arb, but what I am witnessing is the kotest test hangs seemingly indefinitely, making it very hard to debug. What I am wonder is if there is a way to better detect this situation? Can I set a timeout? Force an error? Etc... ** Not a contribution
a

Adam S

02/21/2023, 9:05 AM
Instead of filtering, try defining an 'assumption' https://kotest.io/docs/proptest/property-test-assumptions.html if there are too many values that fail the assumption, the test should fail
j

James Eschner

02/21/2023, 2:11 PM
If I just want to pull a single value from an arb, is there a clean way to do that with these assumptions? ** Not a contribution
a

Adam S

02/21/2023, 2:11 PM
.sample()
I think?
j

James Eschner

02/21/2023, 2:30 PM
Well yes there is
sample
and I was also using
next
above. You were suggesting that I use an assumption, I was wondering how that would work to grab a single value ** Not a contribution
With
filter
it's pretty clear, I can use
Copy code
fooArb.filter { it.x }.next() // substitute sample here if you'd like
with assumptions it doesn't look like, based on the docs, that it's a drop-in replacement for
filter
a

Adam S

02/21/2023, 2:56 PM
filter
and `assume are similar, but different
filter
acts before the test has started, while
assume
acts during an active test. I rarely use
filter
, because there's a risk of filter out all elements (as you found out), but also it can be slow. For example, f 90% of the generated values are discarded, then that's a waste of time.
With filter it's pretty clear, I can use
Copy code
fooArb.filter { it.x }.next() // substitute sample here if you'd like
this doesn't look like standard usage... are you using Arb in a
checkAll()
or
forAll()
?
j

James Eschner

02/21/2023, 3:54 PM
I do use
checkAll
and
forAll
, where appropriate. But if I don't want to incur the cost of the loop I will occasionally just call
next
so that I can generate a random arb. Additionally, using `checkAll`/`forAll` doesn't solve the problem at hand, if you pass a filtered arb into these functions it can still hang. We could probably go back and forth for a bit on best practices for arb testing but the simple comment I have is that ideally calling
.next()
doesn't hang indefinitely and would either time out or throw an exception if there is no possible arb to be generated ** Not a contribution
a

Adam S

02/21/2023, 4:20 PM
oh I wasn't asking because of best practices, of course it's fine to use Arbs outside of a
checkAll()
, it's just not typical so I didn't understand your use case.
assume()
requires
checkAll()
or
forAll()
, so that won't work for you Basically your question isn't related to Arbs really, because you could have the same problem with a Sequence. This example will loop infinitely, because all generated values are smaller than 100.
Copy code
generateSequence { Random.nextInt(1, 100) }
  .filter { it > 100 }
  .first()
probably the simplest workaround is to just limit the number of samples you take
Copy code
generateSequence { Random.nextInt(1, 100) }
  .take(1000)
  .filter { it > 100 }
  .first()
which will fail, but at least it won't loop infinitely You can use
Arb.samples()
to get a sequence of values, and then just make sure to limit them https://github.com/kotest/kotest/blob/0f07e00794ee61bc9cfa4cdb9c0ac9113cf570ca/kotest-property/src/commonMain/kotlin/io/kotest/property/Gen.kt#L110-L116
j

James Eschner

02/21/2023, 4:55 PM
Ah the
take
thing is an interesting bit, I will check it out thank you!
Stumbled across this situation again (trying to call
next
on an impossible-to-generate arb). The most painful part about it is the test hangs indefinitely... I thought that maybe setting a timeout would help (at least the test would complete), but it turns out the timeout does not work... I tried setting from code (
ShouldSpec
) and system properties...
Copy code
timeout = 5000L
invocationTimeout = 5000L
Any ideas? * Not a contribution *
s

sam

12/08/2023, 7:20 PM
The timeouts only work if it's cooperative concurrency. If you're blocking (or at least not yielding), then you would need to set the blocked=true flag as well.
32 Views