9.9:
Errors In Hypothesis Tests
A hypothesis test generally begins by assuming that the null hypothesis is true.
If, in reality, such a null hypothesis is true, rejecting it may lead to an incorrect and misleading conclusion.
This mistake of rejecting the true null hypothesis is known as the Type-I error.
On the other hand, when the null hypothesis is false, but the test result indicates failure of its rejection, the decision again remains erroneous.
This mistake of failing to reject the false null hypothesis is known as the Type-II error.
A test result that indicates rejecting the null hypothesis when it is actually false, or failing to reject it when it is actually true, leads to a correct decision.
The acceptable probability value of Type-I error is the significance level ɑ, which is commonly 0.05 or 0.01.
The probability of Type-II error is denoted by β. It is calculated from the pre-determined probability of rejecting a false null hypothesis, commonly known as the power of the hypothesis test.
9.9:
Errors In Hypothesis Tests
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
- The decision is not to reject null hypothesis when it is true (correct decision).
- The decision is to reject the null hypothesis when it is true (incorrect decision known as a Type I error).
- The decision is not to reject the null hypothesis when, in fact, it is false (incorrect decision known as a Type II error).
- The decision is to reject the null hypothesis when it is false (correct decision whose probability is called the Power of the Test).
Each of the errors occurs with a particular probability. The Greek letters α and β represent the probabilities.
α = probability of a Type I error = P(Type I error) = probability of rejecting the null hypothesis when the null hypothesis is true.
β = probability of a Type II error = P(Type II error) = probability of not rejecting the null hypothesis when the null hypothesis is false.
α and β should be as small as possible because they are probabilities of errors. They are rarely zero.
The Power of the Test is 1 – β. Ideally, we want a high power that is as close to one as possible. Increasing the sample size can increase the Power of the Test.
This test is adapted from Openstax, Introductory Statistics, Section 9.2 Outcomes of Type I and Type II Errors.