Interactive courtroom walkthrough with a start screen: part 1 compares a criminal trial to a hypothesis test, part 2 covers Type I and Type II errors

Innocent until proven guilty

How a criminal jury trial works like a hypothesis test

Part 1 · Steps 1 to 3

Setting the scene

The starting assumption, the bar for evidence, and what "guilty" and "not guilty" mean in stats terms.

Part 2 · Steps 4 to 6

When verdicts go wrong

Convicting an innocent person and letting a guilty one go free: Type I and Type II errors.

1. The starting assumption

A criminal trial starts by assuming the defendant is innocent.

In stats, this starting point is the null hypothesis

H₀: the defendant is innocent

Like a jury, a statistician starts out skeptical. In a study, H₀ is usually "no effect." For example, a new tutoring program makes no difference to test scores, or a new drug has no effect on blood pressure. Innocent in court is like no effect in a study. We keep assuming H₀ until the evidence is strong enough to reject it.

2. The bar for evidence

Evidence barStrength of evidence bar with a high threshold labeled beyond reasonable doubt. Suggestive evidence falls short; overwhelming evidence clears it. How strong is the evidence of guilt? Beyond reasonable doubt Suggestive, but not enough Overwhelming: clears the bar

To convict, the prosecution has to prove guilt beyond reasonable doubt. "Probably did it" is not enough.

In stats, this is the bar for rejecting H₀

Reject H₀ only if the evidence clears the bar

We set the bar in advance, before looking at the data. The significance level, α, sets how high that bar is: a smaller α means a higher bar.

3. What do the verdicts mean?

A jury's verdict is either "guilty" or "not guilty." Remember, H₀ is innocence. Match each stats term to a verdict.

So: reject H₀ = guilty. Fail to reject H₀ = not guilty. Failing to reject H₀ never proves H₀ is true.

4. The jury reaches a verdict

5. A different trial

6. Putting it in the table

H₀: the defendant is innocent

What really happened
Did itH₀ is falseDidn't do itH₀ is true
VerdictGuiltyReject H₀CorrectType I errorinnocent person convicted
Not guiltyFail to reject H₀Type II errorguilty person goes freeCorrect

Courts set a high bar because convicting an innocent person is seen as the worse mistake. The cost is that some guilty people go free. Raise the bar and a Type I error becomes less likely, but a Type II error becomes more likely.

Keep in mind: when the verdict is read, the jury can't know for sure what really happened. Sometimes new evidence, like DNA, comes out years later. A study works the same way: you see your result (the row), but not the truth (the column). So we can't tell whether one result is an error. Instead, we think about the chance of each error: α is the chance of a Type I error when H₀ is true, and β is the chance of a Type II error when H₀ is false. We will discuss β in class.