Long Zhao is an assistant professor in the Department of Analytics & Operations (DAO) at NUS Business School, National University of Singapore. He received his Ph.D. in Decision Sciences from the McCombs School of Business at the University of Texas at Austin. Dr. Zhao’s research interests lie in data-driven decision-making. His research has been the finalist of INFORMS Data mining best paper competition and the finance section best student paper competition.
| Venue | Hon Sui Sen Memorial Library – HSS 4-5 |
| Link to Register
(Hybrid session) |
https://nus-sg.zoom.us/meeting/register/tZcsd–vqT4pGtaBggmJWaMKIuN6uTstODTJ |
| Title | Constructing Quantiles via Forecast Errors: Theory and Empirical Evidence |
| Abstract | Probabilistic forecasts (such as quantiles) are essential inputs to decision-making in the face of uncertainty. However, the most common type often comes in the form of point forecasts. As such, it is necessary for the decision maker to construct uncertainty measures around the obtained point forecasts. One simple approach proposed in the literature suggests leveraging historical forecast errors to create quantile estimators around the given point forecast (referred to as the E2Q method). The sample quantile and normal approximation are two popular E2Q estimators. The former relies on the empirical distribution of the forecast errors while the latter treats the underlying distribution as if it were normal. Despite their prevalence, the relative performances of the two estimators remain unknown. In this paper, we find that the performance of a quantile estimator depends on its bias and variance. In particular, higher variance always leads to worse performance. Furthermore, unbiasedness is never optimal for a fixed variance and becomes less and less appealing as variance increases. Thus, as an asymptotically unbiased estimator, the sample quantile is appealing only when its variance is small. We confirm our theoretical findings using the M5 forecast competition data. Since this competition consists of both the “accuracy” (point) and “uncertainty” (quantile) tracks, we also compare the E2Q method with other methods that directly forecast quantiles. We found that the E2Q method using the top point forecasts can outperform the top direct quantile forecasts. This empirical finding suggests that the E2Q method can be a promising alternative to forecasting quantiles directly. |
