An Evaluation of Early Warning Models for Systemic Banking Crises: Does Machine Learning Improve Predictions?
Johannes Beutel, Sophia List, Gregor von Schweinitz
Abstract
This paper compares the out-of-sample predictive performance of different early warning models for systemic banking crises using a sample of advanced economies covering the past 45 years. We compare a benchmark logit approach to several machine learning approaches recently proposed in the literature. We find that while machine learning methods often attain a very high in-sample fit, they are outperformed by the logit approach in recursive out-of-sample evaluations. This result is robust to the choice of performance measure, crisis definition, preference parameter, and sample length, as well as to using different sets of variables and data transformations. Thus, our paper suggests that further enhancements to machine learning early warning models are needed before they are able to offer a substantial value-added for predicting systemic banking crises. Conventional logit models appear to use the available information already fairly effciently, and would for instance have been able to predict the 2007/2008 financial crisis out-of-sample for many countries. In line with economic intuition, these models identify credit expansions, asset price booms and external imbalances as key predictors of systemic banking crises.
Artikel Lesen
Arbeitspapiere
Fiscal Policy and Fiscal Fragility: Empirical Evidence from the OECD ...
Zur Seite
Publikationen
Does Machine Learning Help us Predict Banking Crises? ...
Zur Seite
Predicting Earnings and Cash Flows: The Information Content of Losses and Tax Loss Carryforwards
Sandra Dreher, Sebastian Eichfelder, Felix Noth
IWH-Diskussionspapiere,
Nr. 30,
2017
Abstract
We analyse the relevance of losses, accounting information on tax loss carryforwards, and deferred taxes for the prediction of earnings and cash flows up to four years ahead. We use a unique hand-collected panel of German listed firms encompassing detailed information on tax loss carryforwards and deferred taxes from the tax footnote. Our out-of-sample predictions show that considering accounting information on tax loss carryforwards and deferred taxes does not enhance the accuracy of performance forecasts and can even worsen performance predictions. We find that common forecasting approaches that treat positive and negative performances equally or that use a dummy variable for negative performance can lead to biased performance forecasts, and we provide a simple empirical specification to account for that issue.
Artikel Lesen
Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Abstract
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The expost threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. Given probabilistic model output, it is intuitive that a decision rule is independent of the data or model specification, as thresholds on probabilities represent a willingness to issue a false alarm vis-à-vis missing a crisis. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Artikel Lesen
Should Forecasters Use Real-time Data to Evaluate Leading Indicator Models for GDP Prediction? German Evidence
Katja Heinisch, Rolf Scheufele
Abstract
In this paper we investigate whether differences exist among forecasts using real-time or latest-available data to predict gross domestic product (GDP). We employ mixed-frequency models and real-time data to reassess the role of survey data relative to industrial production and orders in Germany. Although we find evidence that forecast characteristics based on real-time and final data releases differ, we also observe minimal impacts on the relative forecasting performance of indicator models. However, when obtaining the optimal combination of soft and hard data, the use of final release data may understate the role of survey information.
Artikel Lesen
Does the Technological Content of Government Demand Matter for Private R&D? Evidence from US States
Viktor Slavtchev, S. Wiederhold
American Economic Journal: Macroeconomics,
Nr. 2,
2016
Abstract
Governments purchase everything from airplanes to zucchini. This paper investigates the role of the technological content of government procurement in innovation. In a theoretical model, we first show that a shift in the composition of public purchases toward high-tech products translates into higher economy-wide returns to innovation, leading to an increase in the aggregate level of private R&D. Using unique data on federal procurement in US states and performing panel fixed-effects estimations, we find support for the model's prediction of a positive R&D effect of the technological content of government procurement. Instrumental-variable estimations suggest a causal interpretation of our findings.
Artikel Lesen
Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Abstract
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The ex-post threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Artikel Lesen
Predicting Financial Crises: The (Statistical) Significance of the Signals Approach
Makram El-Shagi, Tobias Knedlik, Gregor von Schweinitz
Journal of International Money and Finance,
Nr. 35,
2013
Abstract
The signals approach as an early-warning system has been fairly successful in detecting crises, but it has so far failed to gain popularity in the scientific community because it cannot distinguish between randomly achieved in-sample fit and true predictive power. To overcome this obstacle, we test the null hypothesis of no correlation between indicators and crisis probability in three applications of the signals approach to different crisis types. To that end, we propose bootstraps specifically tailored to the characteristics of the respective datasets. We find (1) that previous applications of the signals approach yield economically meaningful results; (2) that composite indicators aggregating information contained in individual indicators add value to the signals approach; and (3) that indicators which are found to be significant in-sample usually perform similarly well out-of-sample.
Artikel Lesen
