Name: using the q test to flag outlier data
Uploaded: 2024-04-04T10:51:12.000Z
Duration: 1 min
Channel: JoVE

K12 Standards

Texas Essential Knowledge and Skills

High School

Chemistry

CHEM.2

CHEM.2.C

Detection of Gross Error: The Q Test

Ausreißer sind Datenpunkte, die sich stark vom Rest des Datensatzes unterscheiden.
Der Q-Test von Dixon ist ein Signifikanztest, mit dessen Hilfe bestimmt werden kann, ob diese inkonsistenten Datenpunkte im Datensatz beibehalten oder abgelehnt werden sollen.
Mathematisch gesehen ist die Q-Test-Statistik das Verhältnis der absoluten Differenz zwischen dem Ausreißer und dem nächstgelegenen Datenpunkt und dem Bereich der Grundgesamtheit.
Die Q-Test-Statistik oder der Ablehnungsquotient wird dann mit dem tabellarisch dargestellten kritischen Q-Wert für ein bestimmtes Signifikanzniveau und einen bestimmten Freiheitsgrad verglichen.
Wenn der Ablehnungsquotient gleich oder größer als der tabellarische Wert ist, wird die Nullhypothese verworfen, und der vermutete Ausreißer kann verworfen werden.
Im Gegenteil, die Nullhypothese wird akzeptiert, wenn der experimentelle Wert kleiner ist als der tabellarische Q-Wert. Dann muss der vermutete Ausreißer im Datensatz beibehalten werden.

erkennung grober fehler: der q-test

Wenn ein oder mehrere Datenpunkte weit vom Rest der Daten entfernt erscheinen, muss festgestellt werden, ob es sich um Ausreißer handelt und ob sie aus dem Datensatz entfernt werden sollten, um eine genaue Darstellung des Messwerts sicherzustellen. In vielen Fällen sind Ausreißer auf grobe Fehler (oder menschliche Fehler) zurückzuführen und spiegeln das zugrunde liegende Phänomen nicht genau wider. In einigen Fällen spiegeln diese scheinbaren Ausreißer jedoch echte phänomenologische Unterschiede wider. In diesen Fällen können wir statistische Methoden verwenden, um zu bestimmen, ob der Ausreißer beibehalten werden soll. Eine statistische Methode, die uns dabei helfen kann, Ausreißer beizubehalten oder auszuschließen, heißt Q-Test. Um den Q-Test durchzuführen, ordnen wir zunächst die Werte in einem Datensatz in der Reihenfolge steigender Werte an. Anschließend berechnen wir den Q-Wert, indem wir das Verhältnis der absoluten Differenz zwischen einem Datenpunkt und seinem benachbarten Datenpunkt ermitteln. Dieser Q-Wert wird dann mit dem tabellierten kritischen Q-Wert auf einem gewählten Signifikanzniveau und geeigneten Freiheitsgraden verglichen. Wenn der Q-Wert dem Referenz-Q-Wert in der Tabelle entspricht oder diesen überschreitet, wird der Datenpunkt als Ausreißer betrachtet und daher aus dem Datensatz abgelehnt. Hier ist es sinnvoll, den Datenpunkt zu ignorieren, da die Größe der Abweichung nicht logisch durch zufällige (unbestimmte) Fehler erklärt werden kann. Wenn andererseits der Q-Wert kleiner als der Referenzwert in der Tabelle ist, sollte der Datenpunkt beibehalten werden, und die Interpretation ist, dass der Unterschied zwischen diesem Datenpunkt und den übrigen Daten innerhalb vernünftiger (statistischer) Erwartungen liegt .

Erkennung grober Fehler: Der Q-Test

Ausreißer sind Datenpunkte, die sich stark vom Rest des Datensatzes unterscheiden.
Der Q-Test von Dixon ist ein Signifikanztest, mit dessen Hilfe festgestellt werden kann, ob ...

Education

Lehre

JoVE Core

Analytical Chemistry

Analytische Chemie

Unit 1

Einheit 1

Chemical Applications of Statistical Analyses

Chemische Anwendungen statistischer Analysen

chemical-applications-of-statistical-analyses

KEY TERMS AND CONCEPTS

WICHTIGE BEGRIFFE UND KONZEPTE

using the q test to flag outlier data

The Q test helps identify outlier data points that sit far from the rest of a data set. In high school lab work, this can help students decide whether an unusual value should stay in the data or be removed. Outliers often come from gross errors, or human mistakes, and may not reflect the real result being measured.

Some points that look like outliers may still reflect a true difference in the phenomenon being studied. In those cases, a statistical test can help judge whether the value should be kept or rejected. The Q test provides that check by comparing a suspect point with the rest of the ordered data.

To perform the Q test, the values are first arranged in increasing order. Next, the Q value is calculated as the ratio of the absolute difference between a data point and its adjacent data point. That Q value is then compared with the tabulated critical Q value for a chosen significance level and the correct degrees of freedom.

If the calculated Q value is equal to or greater than the table value, the point is treated as an outlier and rejected from the data set. In that case, the deviation is considered too large to be explained by random, indeterminate errors. If the Q value is smaller than the reference value, the point is retained because the difference is within normal statistical expectation.

Using the Q Test to Flag Outlier Data

si units: 2019 redefinition

SI Units in Chemistry Measurements

SI units give chemistry measurements a common standard. A measurement has two parts: a number that shows magnitude and a unit that gives the comparison scale. Using the same unit system helps keep results consistent and reduces mistakes in scientific communication. The Système International d'Unités, or SI units, includes both fundamental units and derived SI units. Fundamental units describe base physical quantities such as mass, distance, temperature, time, electric current, luminous...

degrees of freedom

Degrees of Freedom in Statistical Calculations

Degrees of freedom describe how many values in a statistical calculation are free to vary. They represent the minimum number of independent numbers needed to determine a statistic. In a data set, some values may be independent while others depend on them. A simple example uses three unknown numbers with a mean of 10. The first two numbers can be chosen freely. The last number cannot be chosen on its own because it must keep the mean at 10. In that case, the data set has two degrees of freedom.<br...

statistical analysis: overview

Mean, Median, Range, Precision & Accuracy

Statistical analysis helps compare repeated measurements and judge how good the results are. When the same sample is measured again and again, small differences can appear. These differences are called errors. Researchers use statistics to describe the data and to decide whether a method is suitable. The mean and median are common ways to find the central value of a data set. The mean is the sum of all results divided by the total number of results. The median is the middle value after the...

types of errors: detection and minimization

types-of-errors-detection-and-minimization

Measurement Errors: Sources and Examples

Measurement errors are the differences between an obtained result and the true value or estimated central value. They can be written as absolute error or relative error. Absolute error is the numerical difference from the true or central value. Relative error compares that difference to the true or central value and is often shown as a percentage. Errors can also be grouped by their source, magnitude, and sign. In many experiments, the main categories are systematic, random, and gross errors.

systematic error: methodological and sampling errors

systematic-error-methodological-and-sampling-errors

Reducing Bias in Measurement Errors

Systematic errors are bias in measurement, and their sources can often be identified and reduced. In this topic, the main categories are sampling, instrumental, methodological, and personal errors. Each one comes from a different part of the experiment, so each one needs a different fix. Sampling errors happen when the sampling method is poor or when the wrong population is chosen. These errors can be reduced by improving the sampling strategy. Better sample selection helps the results...

random error

Random Error in Measurements

Random error in measurements comes from uncontrollable variables that affect data from one trial to the next. These variables can include changes in the environment, imperfections in an instrument, or the natural variation of the thing being measured. Because the direction and size of the error can change from one measurement to the next, random error is hard to predict, estimate, or describe directly. Although random error is difficult to remove, its overall effect can be studied in a large...

standard deviation of calculated results

Reading Data Spread with Standard Deviation

Standard deviation describes how spread out data are around the mean. It helps show how tightly values cluster near the center and how far they sit from that central value. Many large data sets follow a Gaussian distribution, also called a normal distribution. A Gaussian distribution has a bell-shaped curve. The mean, or central value, appears in the middle, where the most frequent values are found. As data points move farther from the center, deviation increases and frequency decreases. The...

introduction to z scores

Reading z Scores for Outliers

z scores show how far a value is from the mean in standard deviation units. A z score, also called a standardized value, tells whether a data point sits above or below the mean, μ. Values larger than the mean have positive z scores, while values smaller than the mean have negative z scores. If x equals the mean, then the z score is zero. The z score scale is centered on zero. That means the mean of all z scores is zero, and their standard deviation is one. This makes z scores useful for...

uncertainty: overview

Measuring and Reporting Uncertainty

Uncertainty is part of analytical chemistry measurement. When repeated measurements are taken, random errors can cause the results to differ in small ways. These repeated values are compared with the estimated or expected value to judge how much uncertainty is present. Uncertainty is usually written after the estimated or expected value. It should also be reported with the correct number of significant figures, which are the digits needed to show a precise result. The possible variation from...

propagation of uncertainty from random error

propagation-of-uncertainty-from-random-error

How Uncertainty Changes in Calculations

Uncertainty changes as measurements move through calculations. In a multi-step experiment, each measurement adds some uncertainty. Because the steps happen one after another, the uncertainty from one step carries into the next step and affects the final result. The way uncertainty is combined depends on the math operation. For addition and subtraction, the final result is reported with absolute uncertainty. Absolute uncertainty is found as the square root of the sum of the absolute...

propagation of uncertainty from systematic error

propagation-of-uncertainty-from-systematic-error

Atomic Mass Uncertainty in Isotope Samples

Atomic mass can vary because different samples contain different isotope ratios. For oxygen, the measured atomic mass is a weighted average of the isotopic masses in that sample. A single sample may not match the true atomic mass of oxygen found across all oxygen molecules on Earth. That creates sampling error, which is a type of systematic error. This kind of uncertainty is not random. It is tied to the system being measured. Within the uncertainty range, the chance of finding a particular...

uncertainty: confidence intervals

Confidence Intervals and Sample Size

Confidence intervals show a range of values around a sample mean that may contain the true mean. They are written as a probability percentage. A 95% confidence interval means the statistician is 95% confident that the true mean lies within that range. The two ends of the range are called confidence limits. These limits are estimated from the sample mean, the standard deviation, and the statistical factor t, or t-score. The t-score depends on the number of degrees of freedom and the confidence...

significance testing: overview

Significance Testing in Analytical Chemistry

Significance testing helps analytical chemists decide whether a difference is meaningful or just caused by random error. It is used to test whether a claim about a parameter is valid. In analytical chemistry, it mainly helps determine whether a change between two values comes from determinate error or indeterminate error. A determinate error has a clear source. It may come from a change in the measurement protocol, the analyst, or the sample itself. These changes can create a deviation from...

identifying statistically significant differences: the f-test

identifying-statistically-significant-differences-the-f-test

Comparing Variance with the F-Test

The F-test compares variance in two data sets or compares a sample variance with a population variance. It helps decide whether an indeterminate error can explain a difference in the values. In statistics, variance shows how spread out the data are. The test assumes that the data set or sets are normally distributed. It also assumes that the data sets are independent of each other. These conditions must be met before the F-test is used. The F test statistic is found by dividing one variance...

comparing experimental results: student's t-test

comparing-experimental-results-student-s-t-test

Student's t-Test for Significance Checks

Student's t-test is a statistical tool for checking whether two means are different in a meaningful way. It can compare a sample mean with a population mean, or it can compare the means from two data sets. The test uses the mean, standard deviation, and number of measurements, along with a chosen confidence interval. The test statistic is then compared with a table of critical values at that confidence level. If the test statistic is smaller than the critical value, the null hypothesis is...

detection of gross error: the q test

The Q test helps identify outlier data points that sit far from the rest of a data set. In high school lab work, this can help students decide whether an unusual value should stay in the data or be removed. Outliers often come from gross errors, or human mistakes, and may not reflect the real result being measured. Some points that look like outliers may still reflect a true difference in the phenomenon being studied. In those cases, a statistical test can help judge whether the value should...

calibration curves: linear least squares

Using a Calibration Line to Find Unknowns

A calibration curve links an instrument's signal to known concentrations of a substance. Scientists use it to set response levels with standards. They can also fit an equation to the curve and use it to find the concentration of unknown samples. When the data make a straight line, the linear least-squares method is the standard way to fit the curve. This method finds the line that best matches the points by minimizing the sum of the squared differences between the predicted and actual values.

calibration curves: correlation coefficient

calibration-curves-correlation-coefficient

Interpreting Correlation in Calibration Curves

Calibration curves use the correlation coefficient, written as r, to show how strongly two variables are related and which direction the relationship takes. In a linear calibration curve, this value helps describe the association between the variables being compared. The correlation coefficient ranges from -1 to +1. A value of +1 means a perfect positive linear correlation, while -1 means a perfect negative correlation. A value of 0 means there is no correlation between the two variables. A...

correlation and regression

Using Regression to Measure Variable Links

Correlation and regression are used to study the relationship between two variables. Correlation shows how strongly the variables are associated. Regression helps describe that relationship with a line that can be used for prediction. In linear regression, the relationship is written as a correlation coefficient. This coefficient is shown as r and ranges from -1 to +1. A positive value means the variables move in the same direction. A negative value means they move in opposite directions. <br...

difference from background: limit of detection

difference-from-background-limit-of-detection

Limit of Detection and Quantification

The limit of detection, or LOD, is the smallest amount of analyte that can be distinguished from background noise. In measurement science, background noise is the natural signal in a sample that can interfere with the analyte signal. The LOD helps show whether a substance is present, but it does not usually give a reliable amount. The LOD is set at the concentration where the analyte signal is three times larger than the standard deviation of the blank signal. A blank is a sample without the...

quantifying and rejecting outliers: the grubbs test

quantifying-and-rejecting-outliers-the-grubbs-test

Grubbs Test for Spotting Data Outliers

The Grubbs test is a statistical way to spot a possible outlier in a data set. It is used when the data is assumed to follow a normal distribution. A Grubbs test helps decide whether one unusual numerical value should be treated as a true outlier. To run a two-tailed Grubbs test, first find the absolute difference between the questionable value and the mean. Then divide that difference by the sample standard deviation. The result is the Grubbs statistic, or G. Next, compare the calculated G...

what is anova?

ANOVA for Comparing Sample Means

ANOVA, or Analysis of Variance, is a statistical test used to compare the means of three or more samples. It was developed by Ronald Fisher in 1918. In high school statistics, ANOVA helps determine whether group averages are equal or whether at least one group is different. Before ANOVA is used, the samples should meet three key assumptions. The data should come from normally distributed samples. The samples should also be randomly and independently selected. In addition, the populations...

key terms and concepts

Learn how the Q test helps students identify outlier data, compare a suspected value to a critical table value, and decide whether to keep or reject it.

Using the Q Test to Flag Outlier Data | JoVE K12

analytical chemistry

JoVE Core Analytical Chemistry simplifies complex chemical principles by breaking them down into clear, engaging animations, making topics from SI Units to NMR spectroscopy more accessible and straightforward. Additionally, the scientist-in-action videos highlight the application of these concepts in real-world research conducted in laboratories globally.

Classifying Matter by State

Chemistry is the study of matter and the changes it undergoes. Matter is anything that has mass and occupies space. Matter is all around us; the air, ...

Classifying Matter by Composition

Matter: Pure Substances and Mixtures
According to its composition, the matter can be classified into two broad categories &mdash; pure substances and ...

physical-and-chemical-properties-of-matter

Physical and Chemical Properties of Matter

The characteristics that enable us to distinguish one substance from another are called properties.

Physical Properties of Matter

A physical property is ...

CHEM.2.A

Types of Errors: Detection and Minimization

Error is the deflection of an obtained result from the expected or true results of an experiment. This happens due to the uncertainty associated with the ...

Systematic Error: Methodological and Sampling Errors

Systematic errors, depending on their source, are of four types &#8211; sampling, instrumental, method, and personal errors.
Sampling errors occur due to ...

Random Error

Indeterminate or random errors arise from several uncontrollable variables in successive measurements.
Since these errors can neither be predicted nor ...

Uncertainty: Overview

The uncertainty reflects the possible range of values in which the result of a measurement can exist.
However, uncertainty varies from error, which is the ...

Propagation of Uncertainty from Random Error

In an experiment, multiple arithmetic operations are often required. Here, the uncertainty associated with the first measurement propagates to the next in ...

Propagation of Uncertainty from Systematic Error

The atomic mass of an element obtained from different sources changes slightly due to the variation in relative isotope concentration from one source to ...

CHEM.2.B

SI Units: 2019 Redefinition

Measurement is integral to analytical chemistry. Each record comprises a number &#8211; denoting the magnitude&#160; &#8211; and the unit &#8211; a ...

Degrees of Freedom

The degree of freedom is the number of independent pieces of information or sample values required to perform any calculation.
The degrees of freedom vary ...

Statistical Analysis: Overview

An analysis is usually conducted by replicated sampling or repeated measurements on the same sample. This leads to scattered results rather than a single ...

Standard Deviation of Calculated Results

A plot of relative deviation from the mean and its frequency of occurrence appears as a Gaussian curve. This probability distribution curve of a ...

Introduction to z Scores

The z score, or standardized score, is the number of standard deviations that a given value is away from the mean. It is one of the commonly used measures ...

Uncertainty: Confidence Intervals

Standard deviation provides a measure of nearness between the sample mean and the true mean reliably for a large number of measurements.
So, when there ...

Significance Testing: Overview

Is the difference between the two values due to an unexplainable random error or a systematic error that can be rationalized by a hypothetical model?
The ...

Identifying Statistically Significant Differences: The F-Test

The F-test checks if the difference between two variances is too large to be explained by an indeterminate error. It compares the variance of a sample and ...

Comparing Experimental Results: Student's t-Test

The influence of changing the method, the sample, or the analyst on the analysis results is studied by altering only one in a pair of experiments.
The ...

Outliers are those data points extremely different from the rest of the data set.
Dixon&#39;s Q-test is a significance test that helps determine whether ...

Correlation and Regression

Regression and correlation are statistical techniques that examine the relationship between two variables.&#160;
While regression is used to understand ...

Difference from Background: Limit of Detection

Background noise is intrinsic to any measurement interfering with the detection of the analyte signal. To analyze if the measured signal is from the ...

Quantifying and Rejecting Outliers: The Grubbs Test

Grubbs&#39; test, like Dixon&#39;s Q-test, is a statistical test to identify the outliers in data with a normal distribution. Here, the number of ...

What is ANOVA?

The analysis of variance&#8212;abbreviated as ANOVA &#8212;is used when the means of three or more samples need to be tested for equality.
For example, ...

Scientific and engineering practices. The student analyzes and interprets data to derive meaning, identify features and patterns, and discover relationships or correlations to develop evidence-based arguments or evaluate designs. The student is expected to:

Using the Q Test to Flag Outlier Data

Description

Transcript

Tags

More videos by Standards