Errors In Hypothesis Tests

When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.

1. The decision is not to reject null hypothesis when it is true (correct decision).
2. The decision is to reject the null hypothesis when it is true (incorrect decision known as a Type I error).
3. The decision is not to reject the null hypothesis when, in fact, it is false (incorrect decision known as a Type II error).
4. The decision is to reject the null hypothesis when it is false (correct decision whose probability is called the Power of the Test).

Each of the errors occurs with a particular probability. The Greek letters α and β represent the probabilities.

α = probability of a Type I error = P(Type I error) = probability of rejecting the null hypothesis when the null hypothesis is true.

β = probability of a Type II error = P(Type II error) = probability of not rejecting the null hypothesis when the null hypothesis is false.

α and β should be as small as possible because they are probabilities of errors. They are rarely zero.

The Power of the Test is 1 – β. Ideally, we want a high power that is as close to one as possible. Increasing the sample size can increase the Power of the Test.

This test is adapted from Openstax, Introductory Statistics, Section 9.2 Outcomes of Type I and Type II Errors.