Diebold Mariano Test

The loss differential is formed by comparing the errors from two forecasts under the same chosen loss function at each time point. Its average represents the observed difference in predictive performance. The test then evaluates whether that mean differs significantly from zero, distinguishing a consistent advantage from an apparent difference caused by variation across the forecast period.

Forecast errors and their loss differentials can be related across successive time points rather than behaving as independent observations. Serial correlation can therefore affect the estimated variability of the mean differential. A robust variance estimate accounts for this dependence, making the statistical comparison more appropriate for time-series forecasts used in engineering systems.

The loss function determines how forecast errors are translated into performance penalties before the two models are compared. Consequently, the result reflects the engineering meaning assigned to error, rather than an abstract accuracy measure alone. Selecting a suitable loss function helps align the test with whether demand, failure, energy, or signal prediction errors matter most.

A mean differential near zero indicates little average separation between the competing forecasts under the selected loss function. The test examines whether that observed difference is statistically distinguishable from zero, rather than treating any numerical advantage as decisive. This helps engineers avoid selecting a model solely because it performs slightly better in a particular sample.

An engineer first identifies two competing forecasts and the corresponding observed time-series outcomes. Errors are evaluated with a common loss function, producing a sequence of loss differentials. The mean differential is then tested against zero using a variance estimate that accommodates possible serial correlation. The resulting comparison supports a reasoned choice between predictive models.

The method can compare forecasts for engineering quantities such as demand, failures, energy use, and signals. In each case, it indicates whether one model’s apparent predictive advantage is statistically meaningful under the selected error measure. This evidence can support model selection and contribute to more dependable predictive systems where forecast quality influences engineering decisions.