Optimizing Policymakers’ Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Macroeconomic Dynamics,
Nr. 1,
2021
Abstract
Recurring financial instabilities have led policymakers to rely on early-warning models to signal financial vulnerabilities. These models rely on ex-post optimization of signaling thresholds on crisis probabilities accounting for preferences between forecast errors, but come with the crucial drawback of unstable thresholds in recursive estimations. We propose two alternatives for threshold setting with similar or better out-of-sample performance: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. Given probabilistic model output, it is intuitive that a decision rule is independent of the data or model specification, as thresholds on probabilities represent a willingness to issue a false alarm vis-à-vis missing a crisis. We provide real-world and simulation evidence that this simplification results in stable thresholds, while keeping or improving on out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Artikel Lesen
Does Machine Learning Help us Predict Banking Crises?
Johannes Beutel, Sophia List, Gregor von Schweinitz
Journal of Financial Stability,
December
2019
Abstract
This paper compares the out-of-sample predictive performance of different early warning models for systemic banking crises using a sample of advanced economies covering the past 45 years. We compare a benchmark logit approach to several machine learning approaches recently proposed in the literature. We find that while machine learning methods often attain a very high in-sample fit, they are outperformed by the logit approach in recursive out-of-sample evaluations. This result is robust to the choice of performance metric, crisis definition, preference parameter, and sample length, as well as to using different sets of variables and data transformations. Thus, our paper suggests that further enhancements to machine learning early warning models are needed before they are able to offer a substantial value-added for predicting systemic banking crises. Conventional logit models appear to use the available information already fairly efficiently, and would for instance have been able to predict the 2007/2008 financial crisis out-of-sample for many countries. In line with economic intuition, these models identify credit expansions, asset price booms and external imbalances as key predictors of systemic banking crises.
Artikel Lesen
An Evaluation of Early Warning Models for Systemic Banking Crises: Does Machine Learning Improve Predictions?
Johannes Beutel, Sophia List, Gregor von Schweinitz
Abstract
This paper compares the out-of-sample predictive performance of different early warning models for systemic banking crises using a sample of advanced economies covering the past 45 years. We compare a benchmark logit approach to several machine learning approaches recently proposed in the literature. We find that while machine learning methods often attain a very high in-sample fit, they are outperformed by the logit approach in recursive out-of-sample evaluations. This result is robust to the choice of performance measure, crisis definition, preference parameter, and sample length, as well as to using different sets of variables and data transformations. Thus, our paper suggests that further enhancements to machine learning early warning models are needed before they are able to offer a substantial value-added for predicting systemic banking crises. Conventional logit models appear to use the available information already fairly effciently, and would for instance have been able to predict the 2007/2008 financial crisis out-of-sample for many countries. In line with economic intuition, these models identify credit expansions, asset price booms and external imbalances as key predictors of systemic banking crises.
Artikel Lesen
Information Feedback in Temporal Networks as a Predictor of Market Crashes
Stjepan Begušić, Zvonko Kostanjčar, Dejan Kovač, Boris Podobnik, H. Eugene Stanley
Complexity,
September
2018
Abstract
In complex systems, statistical dependencies between individual components are often considered one of the key mechanisms which drive the system dynamics observed on a macroscopic level. In this paper, we study cross-sectional time-lagged dependencies in financial markets, quantified by nonparametric measures from information theory, and estimate directed temporal dependency networks in financial markets. We examine the emergence of strongly connected feedback components in the estimated networks, and hypothesize that the existence of information feedback in financial networks induces strong spatiotemporal spillover effects and thus indicates systemic risk. We obtain empirical results by applying our methodology on stock market and real estate data, and demonstrate that the estimated networks exhibit strongly connected components around periods of high volatility in the markets. To further study this phenomenon, we construct a systemic risk indicator based on the proposed approach, and show that it can be used to predict future market distress. Results from both the stock market and real estate data suggest that our approach can be useful in obtaining early-warning signals for crashes in financial markets.
Artikel Lesen
Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Abstract
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The expost threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. Given probabilistic model output, it is intuitive that a decision rule is independent of the data or model specification, as thresholds on probabilities represent a willingness to issue a false alarm vis-à-vis missing a crisis. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Artikel Lesen
Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Abstract
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The ex-post threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Artikel Lesen
Returns to Skills around the World: Evidence from PIAAC
Eric A. Hanushek, Guido Schwerdt, Simon Wiederhold, Ludger Woessmann
European Economic Review,
January
2015
Abstract
Existing estimates of the labor-market returns to human capital give a distorted picture of the role of skills across different economies. International comparisons of earnings analyses rely almost exclusively on school attainment measures of human capital, and evidence incorporating direct measures of cognitive skills is mostly restricted to early-career workers in the United States. Analysis of the new PIAAC survey of adult skills over the full lifecycle in 23 countries shows that the focus on early-career earnings leads to underestimating the lifetime returns to skills by about one quarter. On average, a one-standard-deviation increase in numeracy skills is associated with an 18 percent wage increase among prime-age workers. But this masks considerable heterogeneity across countries. Eight countries, including all Nordic countries, have returns between 12 and 15 percent, while six are above 21 percent with the largest return being 28 percent in the United States. Estimates are remarkably robust to different earnings and skill measures, additional controls, and various subgroups. Instrumental-variable models that use skill variation stemming from school attainment, parental education, or compulsory-schooling laws provide even higher estimates. Intriguingly, returns to skills are systematically lower in countries with higher union density, stricter employment protection, and larger public-sector shares.
Artikel Lesen
Do We Need New Modelling Approaches in Macroeconomics?
Claudia M. Buch, Oliver Holtemöller
Financial Cycles and the Real Economy: Lessons for CESEE Countries,
2014
Abstract
The economic and financial crisis that emerged in 2008 also initiated an intense discussion on macroeconomic research and the role of economists in society. The debate focuses on three main issues. Firstly, it is argued that economists failed to predict the crisis and to design early warning systems. Secondly, it is claimed that economists use models of the macroeconomy which fail to integrate financial markets and which are inadequate to model large economic crises. Thirdly, the issue has been raised that economists invoke unrealistic assumptions concerning human behaviour by assuming that all agents are self-centred, rationally optimizing individuals. In this paper, we focus on the first two issues. Overall, our thrust is that the above statements are a caricature of modern economic theory and empirics. A rich field of research developed already before the crisis and picked up shortcomings of previous models.
Artikel Lesen
Do We Need New Modelling Approaches in Macroeconomics?
Claudia M. Buch, Oliver Holtemöller
IWH Discussion Papers,
Nr. 8,
2014
Abstract
The economic and financial crisis that emerged in 2008 also initiated an intense discussion on macroeconomic research and the role of economists in society. The debate focuses on three main issues. Firstly, it is argued that economists failed to predict the crisis and to design early warning systems. Secondly, it is claimed that economists use models of the macroeconomy which fail to integrate financial markets and which are inadequate to model large economic crises. Thirdly, the issue has been raised that economists invoke unrealistic assumptions concerning human behaviour by assuming that all agents are self-centred, rationally optimizing individuals. In this paper, we focus on the first two issues. Overall, our thrust is that the above statements are a caricature of modern economic theory and empirics. A rich field of research developed already before the crisis and picked up shortcomings of previous models.
Artikel Lesen