Retrospective Forecasting Using Provincial Grid Data and a Quota-Based Hybrid Physical-AI Model

Summary

This study proposes a hybrid cost-forecasting model for power grids that combines physical cost quotas with dynamic macroeconomic/technological adjustments and XGBoost-based residual compensation. Achieving 2.34% MAPE, it balances accuracy and interpretability, addressing regulatory transparency needs in tariff setting.

Abstract

The global energy transition and ongoing electricity market reforms require power grid enterprises to balance reliable power supply with increasingly stringent transmission and distribution tariff regulations. Traditional budgeting methods based on historical extrapolation often fail to reflect the physical basis of asset operations, whereas data-driven machine learning models achieve high predictive accuracy but lack the transparency required for regulatory cost verification. To address the trade-off between forecasting accuracy and interpretability, this study proposes a hybrid cost-forecasting model based on cost quotas. The framework uses standardized operating quotas as the physical budgeting baseline and incorporates a dynamic mechanism for quota evolution driven by macroeconomic conditions and technological progress. Extreme Gradient Boosting (XGBoost) is employed to capture nonlinear residuals beyond the quota-based estimates, while SHAP (Shapley Additive exPlanations) is used to interpret the contribution of key cost drivers. The model was evaluated using 16 years of anonymized operational data from a provincial power grid in China. It achieved a mean absolute percentage error (MAPE) of 2.34%, reducing forecasting errors by 61.8%, 46.6%, and 34.1% compared with SARIMAX, standalone XGBoost, and Attention-LSTM models, respectively. The proposed framework integrates engineering cost-quota principles with explainable artificial intelligence, providing both accurate long-term cost forecasts and a transparent decision-support tool for regulatory permitted-cost verification.

Introduction

The global energy transition is reshaping power grid asset management and operational practices through the integration of distributed renewable generation, increasing extreme weather events, and widespread deployment of IoT technologies, all of which demand greater operational flexibility and maintenance resources to ensure system reliability^1,2. At the same time, electricity regulators have strengthened transmission and distribution tariff oversight through frameworks such as the UK's RIIO and the US Federal Energy Regulatory Commission's rate-of-return reviews, emphasizing transparency in cost justi....

Protocol

This study used anonymized operational and financial data collected from a provincial power grid in eastern China. All data were aggregated and de-identified before analysis, and no personally identifiable or sensitive individual-level information was included. Therefore, ethical approval was not required. Data access and analysis complied with applicable data protection regulations and institutional agreements governing power sector information.

Forecasting framework overview

To integrate physics-based and data-driven forecasting approaches, a hybrid forecasting framework was developed that comb....

Results

Panoramic dimensionality-reduced comparison of forecasting accuracy

When all Tree-structured Parzen Estimator (TPE)-optimized models were evaluated on the 2023–2025 out-of-sample test set, which included post-pandemic recovery fluctuations and extreme high-temperature events, the performance of mainstream forecasting algorithms could be objectively compared. To provide a rigorous and comprehensive evaluation, four benchmark models representing different methodological approach.......

Discussion

This study demonstrates that integrating industrial cost quotas with machine learning provides an interpretable and accurate framework for long-term power grid cost forecasting. The results show that climate-related variables, particularly extreme high-temperature days and line trip impact indices, are the dominant drivers of cost residuals, highlighting the increasing influence of external disturbances on utility expenditures. These findings suggest that weather-related costs should no longer be treated as incidental op.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
E-Grid anonymized monthly accounting dataset	Provincial power grid in eastern China	N/A	Anonymized monthly production and operation cost dataset (2010–2025) comprising 192 observations, including physical asset, operation and maintenance, macroeconomic, and environmental data. Used for model development (156 observations) and out-of-sample testing (36 observations).
Matplotlib plotting library	Matplotlib development team	3.5.2	Used to generate manuscript figures and graphical outputs, including model-performance plots, topology visualizations, residual diagnostics, radar charts, and SHAP-related visual summaries.
NumPy numerical computing library	NumPy developers	1.22.3	Used for numerical array operations, matrix calculations, and reproducibility control. A fixed random seed of 42 was applied where relevant.
Pandas data manipulation library	pandas development team	1.4.2	Used for data import, integration, restructuring, filtering, and preprocessing, including missing-value imputation, time-series alignment, and preparation of model input variables.
Python programming environment	Python Software Foundation	3.9.13	Programming environment used to implement the complete forecasting workflow, including data preprocessing, feature engineering, model training, hyperparameter optimization, prediction, statistical testing, and performance evaluation.
Scikit-learn machine learning library	scikit-learn contributors	1.0.2	Used for data preprocessing, training-validation splitting, auxiliary model evaluation, and statistical procedures, including Pearson correlation analysis and calculation of selected performance metrics.
SciPy scientific computing library	SciPy contributors	1.9.0	Used for statistical calculations, including Pearson correlation analysis and supporting procedures for outlier treatment and winsorization.
Tree-structured Parzen Estimator implementation	Optuna contributors	3.1.0	Bayesian optimization implementation used to tune XGBoost hyperparameters, including maximum tree depth, learning rate, and regularization parameters. The Tree-structured Parzen Estimator was implemented through the open-source Optuna package.
XGBoost software library	DMLC / XGBoost contributors	1.7.1	Decision-tree-based ensemble learning library used for nonlinear residual compensation. The model implemented a regularized objective function with Huber loss and tree-complexity penalties, as described in Equations 7–9.