Do you have examples where Python’s Hypothesis (and its shrinking) works better than Haskell’s QuickCheck? This would let us improve Haskell’s QuickCheck.
My understanding is that Haskell’s QuickCheck can shrink functions, and with parametric polymorphism, can work with vector types and more [1], which would be difficult if not impossible in Python.
And in general, the base types covered by the shrinking in Python’s Hypothesis should have analogues by the shrinking in Haskell’s QuickCheck. And with deriving, Haskell’s QuickCheck should get shrinking ‘for free’ as in Python’s Hypothesis.
Also, Haskell’s QuickCheck extends to stateful and parallel checking, which is not supported in Python [2].
I have to admit the last time I seriously worked with QuickCheck was quite a while ago.
> My understanding is that Haskell’s QuickCheck can shrink functions [...]
Are you talking about shrinking a value of type (a -> b) (for some concrete a and b)? In Hypothesis, if you write a generator that spits out functions, you get a shrinker for free. So that shrinker will shrink functions.
As far as I can tell, QuickCheck's shrinking is based on values. I just checked, the type is still `shrink :: a -> [a]`, so that can't have changed.
In Hypothesis shrinking is based on the generation process. In Hypothesis generators act a bit like 'Functors' in Haskell, ie you can map over them. (You can also filter and `<>` and do monadic binds etc.)
So in Hypothesis you can take a generator that produces integers, and create one that produces only even numbers by (in Haskell terms) mapping (2) over it. Now, the shrinker you get for free will also only produce even numbers.
That's not possible (or at least not easily possible) in Haskell. You would have to define at least a newtype wrapper.
Apart from shrinking, Haskell's QuickCheck can also learn a lot from Hypothesis in terms of floating point number generation. The last time I used QuickCheck seriously, they basically generated floats by converting from a sort-of uniformly sampled fractional number. I see that this commit https://github.com/nick8325/quickcheck/commit/07942642d7987b... seems to have made float generation a lot smarter!
Hypothesis is especially 'nasty' when it generates floats. It has a good chunk of probability allocated to things like various NaNs, infinities, sub-normal numbers, zero, one float past zero, etc, plus of course some probability mass on uniformly chosen floats. See https://hypothesis.readthedocs.io/en/latest/data.html#hypoth... for the docs, but you should check out the code, too.
> In Hypothesis shrinking is based on the generation process. In Hypothesis generators act a bit like 'Functors' in Haskell, ie you can map over them. (You can also filter and `<>` and do monadic binds etc.)
You are right that the generators in Hypothesis correspond to `Functor`s, so you can map over it. Indeed, in QuickCheck, its `Gen` [1] (type of `arbitrary` [2], the generator in QuickCheck) is a `Functor`, and moreover a `Monad`, so you can do monadic binds on `Gen`.
> So in Hypothesis you can take a generator that produces integers, and create one that produces only even numbers by (in Haskell terms) mapping (2) over it. Now, the shrinker you get for free will also only produce even numbers.
> That's not possible (or at least not easily possible) in Haskell. You would have to define at least a newtype wrapper.
Since `Gen` in QuickCheck is a `Functor`, to multiply its output by 2 to generate only even numbers, you can do this in Haskell as:
fmap (\x -> x*2) (arbitrary @Int)
Or, for those preferring OOP-style chaining:
arbitrary @Int <&> (*2)
where `<&>` is `fmap` with arguments flipped, and `(*2) = \x -> x*2`, and `@Int` is type application to specialize `arbitrary` to generate `Int`.
I think what you are hinting at is that, QuickCheck separates generation and shrinking, while Hypothesis combines generation and shrinking. If you prefer combining generation and shrinking, you may want to check out hedgehog in Haskell, see [3] for a discussion of this trade-off.
I think by using the `MonadGen` in hedgehog [4], you should be able to map over and bind over generators, with automatic shrinking, much like Hypothesis.
Hypothesis was first released in 2013, while hedgehog in 2017, so it is possible that hedgehog was inspired by Hypothesis or similar property-based testing libraries.
But in general, I would be surprised if such ‘functional’ APIs (map, filter, reduce, bind, etc.) could not be ported to Haskell.
Yes, I know that `Gen` in QuickCheck is a Monad. What I am saying is `Arbitrary` isn't a Monad.
I know, Arbitrary technically can't be a Monad, because the kinds are wrong. But I mean 'morally': you can fmap the generation process, but you can't fmap the shrinking. Exactly because generation and shrinking are separated; and shrinking is driven purely by the value and type of the generated item, but has no access to any information about the generation process.
> But in general, I would be surprised if such ‘functional’ APIs (map, filter, reduce, bind, etc.) could not be ported to Haskell.
Yes, have a look at the Jack library I already linked to.
My understanding is that Haskell’s QuickCheck can shrink functions, and with parametric polymorphism, can work with vector types and more [1], which would be difficult if not impossible in Python.
And in general, the base types covered by the shrinking in Python’s Hypothesis should have analogues by the shrinking in Haskell’s QuickCheck. And with deriving, Haskell’s QuickCheck should get shrinking ‘for free’ as in Python’s Hypothesis.
Also, Haskell’s QuickCheck extends to stateful and parallel checking, which is not supported in Python [2].
[1]: https://library.mlabs.city/mastering-quickcheck "Mastering QuickCheck: Advanced yet Practical Techniques for Property-Based Testing"
[2]: https://stevana.github.io/the_sad_state_of_property-based_te... "The sad state of property-based testing libraries: A survey of property-based testing libraries"