Optimizing Policymakers’ Loss Functions in Crisis Prediction: Before, Within or After?
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The expost threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. Given probabilistic model output, it is intuitive that a decision rule is independent of the data or model specification, as thresholds on probabilities represent a willingness to issue a false alarm vis-à-vis missing a crisis. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
An Evaluation of Early Warning Models for Systemic Banking Crises: Does Machine Learning Improve Predictions?
IWH Discussion Papers,
This paper compares the out-of-sample predictive performance of different early warning models for systemic banking crises using a sample of advanced economies covering the past 45 years. We compare a benchmark logit approach to several machine learning approaches recently proposed in the literature. We find that while machine learning methods often attain a very high in-sample fit, they are outperformed by the logit approach in recursive out-of-sample evaluations. This result is robust to the choice of performance measure, crisis definition, preference parameter, and sample length, as well as to using different sets of variables and data transformations. Thus, our paper suggests that further enhancements to machine learning early warning models are needed before they are able to offer a substantial value-added for predicting systemic banking crises. Conventional logit models appear to use the available information already fairly effciently, and would for instance have been able to predict the 2007/2008 financial crisis out-of-sample for many countries. In line with economic intuition, these models identify credit expansions, asset price booms and external imbalances as key predictors of systemic banking crises.
Fiscal Policy and Fiscal Fragility: Empirical Evidence from the OECD ...
Does Machine Learning Help us Predict Banking Crises? ...
Predicting Earnings and Cash Flows: The Information Content of Losses and Tax Loss Carryforwards
IWH Discussion Papers,
We analyse the relevance of losses, accounting information on tax loss carryforwards, and deferred taxes for the prediction of earnings and cash flows up to four years ahead. We use a unique hand-collected panel of German listed firms encompassing detailed information on tax loss carryforwards and deferred taxes from the tax footnote. Our out-of-sample predictions show that considering accounting information on tax loss carryforwards and deferred taxes does not enhance the accuracy of performance forecasts and can even worsen performance predictions. We find that common forecasting approaches that treat positive and negative performances equally or that use a dummy variable for negative performance can lead to biased performance forecasts, and we provide a simple empirical specification to account for that issue.
Are Qualitative Inflation Expectations Useful to Predict Inflation?
Journal of Business Cycle Measurement and Analysis,
This paper examines the properties of qualitative inflation expectations collected from economic experts for Germany. It describes their characteristics relating to rationality and Granger causality. An out-of-sample simulation study investigates whether this indicator is suitable for inflation forecasting. Results from other standard forecasting models are considered and compared with models employing survey measures. We find that a model using survey expectations outperforms most of the competing models. Moreover, we find some evidence that the survey indicator already contains information from other model types (e. g. Phillips curve models). However, the forecast quality may be further improved by completely taking into account information from some financial indicators.
Forecasting Currency Crises: Which Methods signaled the South African Crisis of June 2006?
South African Journal of Economics,
In this paper we test the ability of three of the most popular methods to forecast South African currency crises with a special emphasis on their out-of-sample performance. We choose the latest crisis of June 2006 to conduct an out-of-sample experiment. The results show that the signals approach was not able to forecast the out-of-sample crisis correctly; the probit approach was able
to predict the crisis but only with models, that were based on raw data. The Markov-regime- switching approach predicts the out-of-sample crisis well. However, the results are not straightforward. In-sample, the probit models performed remarkably well and were also able to detect, at least to some extent, out-of-sample currency crises before their occurrence. The recommendation is to not restrict the forecasting to only one approach.
Three methods of forecasting currency crises: Which made the run in signaling the South African currency crisis of June 2006?
IWH Discussion Papers,
In this paper we test the ability of three of the most popular methods to forecast the South African currency crisis of June 2006. In particular we are interested in the out-ofsample performance of these methods. Thus, we choose the latest crisis to conduct an out-of-sample experiment. In sum, the signals approach was not able to forecast the outof- sample crisis of correctly; the probit approach was able to predict the crisis but just with models, that were based on raw data. Employing a Markov-regime-switching approach also allows to predict the out-of-sample crisis. The answer to the question of which method made the run in forecasting the June 2006 currency crisis is: the Markovswitching approach, since it called most of the pre-crisis periods correctly. However, the “victory” is not straightforward. In-sample, the probit models perform remarkably well and it is also able to detect, at least to some extent, out-of-sample currency crises before their occurrence. It can, therefore, not be recommended to focus on one approach only when evaluating the risk for currency crises.