Tests not ending with custom shrinker about fscheck HOT 5 CLOSED

Arrays/lists can have very long shrink sequences. It tries to remove one element at a time.

from fscheck.

FsCheck has no magic way to prevent shrinking if you literally evaluate it explicitly. Under normal circumstances, it won't shrink unless the random part of the test fails.

Shrinking can take a long time. It can also get stuck in a loop, if shrinking a value yields the same value again somewhere along the way.

from fscheck.

Sorry maybe I didn't explain well. I'm defining an arbitrary instance like

let arbInstance =
    { new Arbitrary<...>() with
        override x.Generator = ...

        override x.Shrinker(dates) = ...
        }

and the issue I'm having is that putting

let _ = (ArbMap.defaults |> ArbMap.arbitrary |> Arb.toShrink) dates |> Seq.length

inside my Shrinker method override freezes my tests, even when they're passing, in which case I would expect Shrinker to never be called.

I suppose it doesn't really matter, since the implementation shouldn't evaluate the infinite sequence anyways, it was just confusing behaviour and took me a while to figure out what was happening.

I'll make a PR adding to the docs that the sequence of shrinks may be infinite, and evaluating that sequence may freeze the tests even if no tests are failing.

from fscheck.

I see. Yes, FsCheck relies on the laziness of the returned sequence. It will call Shrinker during normal evaluation of the test, and capture the resulting sequence. This allows it to pick up shrinking with the values the test failed with. It effectively captures Shrinker(dates) in a closure, but only evaluates the resulting sequence one at a time, and only evaluates the sequence if the random test fails.

from fscheck.

Thanks for the explanation, I'll update my PR tomorrow.

I managed to minimize the issue I was running into, it turned out to be just that the shrink sequence was a lot bigger than I thought. A minimal repro:

    [<Property>]
    let ``test`` () =
        Prop.forAll (
            { new Arbitrary<int[]>() with
                override x.Generator = Seq.init 500 (fun i -> i) |> Seq.toArray |> Gen.constant

                override x.Shrinker(vs) =
                    let shrunk = (ArbMap.defaults |> ArbMap.arbitrary |> Arb.toShrink) vs
                    // let _ = Seq.length shrunk
                    Seq.empty }
        )
        <| fun (vs) -> true

Here, uncommenting the Seq.length line makes the test take 10 seconds. Commented it takes less than a second.

My original case was generating a datetime array with a size much larger than 500, so it appeared the test was stuck in an infinite loop.

from fscheck.