bias/limit for input values that are less relevant

[dt-eric-lefevreardant]

In our tests, some of the input values generated by arbitraries are not as important as others. For example, in this function, the values are clearly important, while the date does not matter much:

function sortNumbersAscending(values: number[], currentDate: Date) {
    log(currentDate, "Sorting");
    // actually sort, etc.
    return sortedValues;
}

When writing the tests on our project, we are interested in using arbitraries for the values, but we also interested in making clear to the reader that any date works. So we end up with tests such as one:

fc.assert(
    fc.property(fc.array(fc.integer()), fc.date(), (data, date) => {
        const sortedData = sortNumbersAscending(data, date);
        for (let i = 1; i < data.length; ++i) {
            expect(sortedData[i - 1]).toBeLessThanOrEqual(sortedData[i]);
        }
    })
);

The issue however is that fast-check will try to vary all inputs, including the date, even though the date does not prove anything interesting about the code. This reduces the number of relevant combinaisons of values passed to the code, while we'd prefer fast-check to vary the data more aggressively, and the date less, or not at all.
We could, of course, pass a hard-coded date (eg. sortNumbersAscending(data, new Date("2000-01-01"))), but then it begins to look suspicious to the reader who might wonder if there is anything special about this date. We think that passing a fast-check-generated value instead makes clearer that any value here is acceptable.

We've started addressing this by introducing helpers for those cases that ensure that only a small number of possible values are generated. Here is an example:

export function fcUnvaryingDate(): Arbitrary<Date> {
    return fc.constantFrom(...fc.sample(fc.date(), 1));
}

In some cases, this significantly reduces the number of necessary runs, especially when dealing with enums.

It seems that this is a problem that others might also have. Are there well known patterns to address this issue?