How to Audit a Random Tool Yourself

Test one: the frequency count

Run the tool many times and tally the outcomes. For a uniform draw over k outcomes, each should appear about 1/k of the time, and the typical deviation in a count of n draws is about the square root of n.

Concretely: a d6 rolled 6,000 times should give each face about 1,000 times, with a typical deviation of about 29. Anything from roughly 940 to 1,060 per face is unremarkable; a face appearing 850 times is worth investigating.

This is the single most useful test because it catches the commonest real defect — modulo bias — which shows up as a consistent excess in the low outcomes. It needs a few thousand samples to be meaningful, which is why it is worth automating rather than doing by hand.

Test two: the pairs test

Frequency alone cannot detect a generator that produces the right proportions in the wrong order. A sequence that cycles 1,2,3,4,5,6 repeatedly passes a frequency test perfectly and is obviously not random.

The pairs test tallies consecutive outcomes rather than individual ones. For a fair d6 there are 36 possible pairs and each should appear about 1/36 of the time. A generator with any short-term structure shows up as some pairs being much more or less common than others.

A useful special case is counting immediate repeats. For a fair d6, about 1 roll in 6 should equal the previous one. A tool that never repeats — or repeats far more than that — is doing something other than an independent draw, which may or may not be disclosed.

Test three: the boundary check

Off-by-one errors are common, silent and easy to detect. Set the tool to a small range and check whether both endpoints actually appear.

For a range of 1 to 10, run a few hundred draws and confirm you see both 1 and 10. A range that never produces its maximum is off by one and will be producing 1 to 9, which nobody notices until they need the top value.

The same check applies to a range of size one. Set the minimum equal to the maximum and confirm the tool returns that value rather than erroring or hanging, which is a surprisingly common failure in hand-rolled range code.

• Set a small range and confirm both endpoints appear.
• Set minimum equal to maximum and confirm the tool returns that value.
• Set the range backwards and see whether it handles it or fails loudly.
• For a decimal tool, check whether endpoint frequency is halved by rounding — that one is expected.

Test four: the source read

For a client-side tool, the code is on your machine. Open the developer tools, find the script, and search for two things: `Math.random` and the shape of the range conversion.

`Math.random` is not a defect by itself and is a defect if the tool claims cryptographic randomness. That is a straightforward contradiction between claim and implementation, and it is worth checking on anything that markets its fairness.

The range conversion tells you whether rejection sampling is present. A bare `% n` on a random byte is modulo bias; a loop that discards values above a threshold is the correct pattern. This is the one test that identifies the defect rather than merely detecting it.

What none of these can tell you

They cannot establish that a generator is cryptographically secure. Passing every statistical test is consistent with being completely predictable — a seeded generator passes all of them and gives away its entire future to anyone with the seed.

They cannot detect a server-side tool doing anything at all, since you see only the outputs. For a tool that sends your list somewhere and receives a winner, no client-side test can distinguish a fair draw from a chosen one.

And they cannot detect a draw that was re-run until the organiser liked it. That is not a property of the tool, and no amount of testing the tool will reveal it — which is why the evidence side of fairness is a separate problem from the mechanism side.

Frequently asked questions

Tools that use this

Related guides