sample: draw n values from a generator without running a property #1

New issue

Closed

opened 2026-09-13 20:03:59 +09:00 by natsukium · 2 comments

natsukium commented

2026-09-13 20:03:59 +09:00

Owner

Missing

A way to draw n values from a generator deterministically without running a property: sample { seed; tests; maxSize ? 100; } gen -> [ value ], using the same seed splitting and size cycle check uses.

Where it was needed

jsonschema2nix (tests/generators.nix, samples) checks one direction of its first invariant in a derivation: generated (schema, value) pairs are handed to a Python validator at build time, which a forAll predicate cannot do. To get the cases it reaches into nixcheck.rng.fromSeed / rng.split and reimplements the size cycle by hand:

samples = { seed, tests, maxSize ? 40 }: g:
  let
    go = i: state:
      if i >= tests then [ ] else
      let s = nixcheck.rng.split state;
          size = i - (i / (maxSize + 1)) * (maxSize + 1);
      in [ (g size s.left).value ] ++ go (i + 1) s.right;
  in go 0 (nixcheck.rng.fromSeed seed);

That couples the test to rng internals and to how check happens to pace sizes today; if the runner changes, the sampled distribution silently diverges from what the properties see.

Suggested shape

sample opts gen in property.nix sharing the loop with check.
Optionally sampleTree returning the trees, for callers that want to shrink a failing external check by hand.

## Missing A way to draw `n` values from a generator deterministically without running a property: `sample { seed; tests; maxSize ? 100; } gen -> [ value ]`, using the same seed splitting and size cycle `check` uses. ## Where it was needed jsonschema2nix (`tests/generators.nix`, `samples`) checks one direction of its first invariant in a derivation: generated (schema, value) pairs are handed to a Python validator at build time, which a `forAll` predicate cannot do. To get the cases it reaches into `nixcheck.rng.fromSeed` / `rng.split` and reimplements the size cycle by hand: ```nix samples = { seed, tests, maxSize ? 40 }: g: let go = i: state: if i >= tests then [ ] else let s = nixcheck.rng.split state; size = i - (i / (maxSize + 1)) * (maxSize + 1); in [ (g size s.left).value ] ++ go (i + 1) s.right; in go 0 (nixcheck.rng.fromSeed seed); ``` That couples the test to `rng` internals and to how `check` happens to pace sizes today; if the runner changes, the sampled distribution silently diverges from what the properties see. ## Suggested shape - `sample opts gen` in `property.nix` sharing the loop with `check`. - Optionally `sampleTree` returning the trees, for callers that want to shrink a failing external check by hand.

natsukium referenced this issue

2026-09-13 21:24:22 +09:00

Report: reproduction info (seed/size/index) and replay of a single case #7

natsukium referenced this issue

2026-09-13 21:24:22 +09:00

classify / cover / collect: distribution labels in the Report #8

natsukium commented

2026-09-13 21:24:52 +09:00

Author

Owner

Plan

Extract the case-producing part of the check loop (lib/property.nix) into cases { seed; tests; maxSize; } gen -> [ { tree; size; index; } ]. check folds over it, sample maps .value, sampleTree returns the trees. Same seed => the exact cases check sees, by construction.

Shape follows fast-check fc.sample(arb, { seed, numRuns }) (explicit seed) rather than Hedgehog Gen.sample (IO, fixed size 30), since Nix is pure.

The same cases list is what #7 (replay info) and #8 (labels) need, so land those on top of this refactor.

Order in the overall roadmap: after #6, #5, #2.

## Plan Extract the case-producing part of the `check` loop (`lib/property.nix`) into `cases { seed; tests; maxSize; } gen -> [ { tree; size; index; } ]`. `check` folds over it, `sample` maps `.value`, `sampleTree` returns the trees. Same seed => the exact cases `check` sees, by construction. Shape follows fast-check `fc.sample(arb, { seed, numRuns })` (explicit seed) rather than Hedgehog `Gen.sample` (IO, fixed size 30), since Nix is pure. The same `cases` list is what #7 (replay info) and #8 (labels) need, so land those on top of this refactor. Order in the overall roadmap: after #6, #5, #2.

natsukium referenced this issue

2026-09-13 21:24:52 +09:00

gen.sized / gen.recursive: size-driven depth for recursive generators #5

natsukium commented

2026-09-14 00:44:13 +09:00

Author

Owner

Implemented in ff3990b, pushed to main.

Implemented in ff3990b, pushed to main.

natsukium closed this issue

2026-09-14 00:44:13 +09:00