Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Abstract
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The expost threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. Given probabilistic model output, it is intuitive that a decision rule is independent of the data or model specification, as thresholds on probabilities represent a willingness to issue a false alarm vis-à-vis missing a crisis. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Read article
Should Forecasters Use Real-time Data to Evaluate Leading Indicator Models for GDP Prediction? German Evidence
Katja Heinisch, Rolf Scheufele
Abstract
In this paper we investigate whether differences exist among forecasts using real-time or latest-available data to predict gross domestic product (GDP). We employ mixed-frequency models and real-time data to reassess the role of survey data relative to industrial production and orders in Germany. Although we find evidence that forecast characteristics based on real-time and final data releases differ, we also observe minimal impacts on the relative forecasting performance of indicator models. However, when obtaining the optimal combination of soft and hard data, the use of final release data may understate the role of survey information.
Read article
Qual VAR Revisited: Good Forecast, Bad Story
Makram El-Shagi, Gregor von Schweinitz
Journal of Applied Economics,
No. 2,
2016
Abstract
Due to the recent financial crisis, the interest in econometric models that allow to incorporate binary variables (such as the occurrence of a crisis) experienced a huge surge. This paper evaluates the performance of the Qual VAR, originally proposed by Dueker (2005). The Qual VAR is a VAR model including a latent variable that governs the behavior of an observable binary variable. While we find that the Qual VAR performs reasonable well in forecasting (outperforming a probit benchmark), there are substantial identification problems even in a simple VAR specification. Typically, identification in economic applications is far more difficult than in our simple benchmark. Therefore, when the economic interpretation of the dynamic behavior of the latent variable and the chain of causality matter, use of the Qual VAR is inadvisable.
Read article
Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Abstract
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The ex-post threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Read article
Outperforming IMF Forecasts by the Use of Leading Indicators
Katja Drechsel, Sebastian Giesen, Axel Lindner
IWH Discussion Papers,
No. 4,
2014
Abstract
This study analyzes the performance of the IMF World Economic Outlook forecasts for world output and the aggregates of both the advanced economies and the emerging and developing economies. With a focus on the forecast for the current and the next year, we examine whether IMF forecasts can be improved by using leading indicators with monthly updates. Using a real-time dataset for GDP and for the indicators we find that some simple single-indicator forecasts on the basis of data that are available at higher frequency can significantly outperform the IMF forecasts if the publication of the Outlook is only a few months old.
Read article
Qual VAR Revisited: Good Forecast, Bad Story
Makram El-Shagi, Gregor von Schweinitz
Abstract
Due to the recent financial crisis, the interest in econometric models that allow to incorporate binary variables (such as the occurrence of a crisis) experienced a huge surge. This paper evaluates the performance of the Qual VAR, i.e. a VAR model including a latent variable that governs the behavior of an observable binary variable. While we find that the Qual VAR performs reasonably well in forecasting (outperforming a probit benchmark), there are substantial identification problems. Therefore, when the economic interpretation of the dynamic behavior of the latent variable and the chain of causality matter, the Qual VAR is inadvisable.
Read article
Does Central Bank Staff Beat Private Forecasters?
Makram El-Shagi, Sebastian Giesen, A. Jung
IWH Discussion Papers,
No. 5,
2012
Abstract
In the tradition of Romer and Romer (2000), this paper compares staff forecasts of the Federal Reserve (Fed) and the European Central Bank (ECB) for inflation and output with corresponding private forecasts. Standard tests show that the Fed and less so the ECB have a considerable information advantage about inflation and output. Using novel tests for conditional predictive ability and forecast stability for the US, we identify the driving forces of the narrowing of the information advantage of Greenbook forecasts coinciding with the Great Moderation.
Read article
The Performance of Short-term Forecasts of the German Economy before and during the 2008/2009 Recession
Katja Drechsel, Rolf Scheufele
International Journal of Forecasting,
No. 2,
2012
Abstract
The paper analyzes the forecasting performance of leading indicators for industrial production in Germany. We focus on single and pooled leading indicator models both before and during the financial crisis. Pairwise and joint significant tests are used to evaluate single indicator models as well as forecast combination methods. In addition, we investigate the stability of forecasting models during the most recent financial crisis.
Read article