10.5:
Multiple Comparison Tests
A multiple comparison test, or MCT is a type of post hoc analysis generally conducted after comparing multiple samples using hypothesis tests such as ANOVA.
When many groups are compared, or multiple factors are tested in some groups, the MCT mainly helps identify a specific group that is significantly different from the others, or a factor that causes a significant effect.
For example, when comparing two groups of zebrafish it is easy to identify a group with a significantly different mean length at a 0.05 significance level.
If we increase the number of test groups, it becomes increasingly difficult to find the group with significantly different mean.
In such cases, a pairwise comparison also gives higher rates of Type-I error.
MCT helps determine a significantly different group in such cases by correcting the alpha values to reduce the Type-I error.
There are different types of MCTs that can be used for equal or unequal sample sizes. The most commonly used MCT is the Bonferroni test.
10.5:
Multiple Comparison Tests
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number of samples increases. This is because the number of sample pairs to be compared or pairwise comparisons increases with the number of samples. Further, the percentage of Type-I error increases with the number of pairwise comparisons.
An MCT will help identify the significantly different mean among multiple samples by correcting the significance alpha values and reducing the Type-I error. Additionally, one can use different MCTs for datasets with equal or unequal sample sizes. An example of a commonly used MCT is the Bonferroni test.