Systemic Real and Financial Risks: Measurement, Forecasting, and Stress Testing

Systemic Real and Financial Risks: Measurement, Forecasting, and Stress Testing. By Gianni de Nicolo & Marcella Lucchetta
IMF Working Paper No. 12/58
Feb 2012
http://www.imf.org/external/pubs/cat/longres.aspx?sk=25745.0

Summary: This paper formulates a novel modeling framework that delivers: (a) forecasts of indicators of systemic real risk and systemic financial risk based on density forecasts of indicators of real activity and financial health; (b) stress-tests as measures of the dynamics of responses of systemic risk indicators to structural shocks identified by standard macroeconomic and banking theory. Using a large number of quarterly time series of the G-7 economies in 1980Q1-2010Q2, we show that the model exhibits significant out-of sample forecasting power for tail real and financial risk realizations, and that stress testing provides useful early warnings on the build-up of real and financial vulnerabilities.

Excerpts

Introduction

The 2007-2009 financial crisis has spurred renewed efforts in systemic risk modeling. Bisias et al. (2012) provide an extensive survey of the models currently available to measure and track indicators of systemic financial risk. However, three limitations of current modeling emerge from this survey. First, almost all proposed measures focus on (segments of) the financial sector, with developments in the real economy either absent, or just part of the conditioning variables embedded in financial risk measures. Second, there is yet no systematic assessment of the out-of-sample forecasting power of the measures proposed, which makes it difficult to gauge their usefulness as early warning tools. Third, stress testing procedures are in most cases sensitivity analyses, with no structural identification of the assumed shocks.

Building on our previous effort (De Nicolò and Lucchetta, 2011), this paper contributes to overcome these limitations by developing a novel tractable model that can be used as a real-time systemic risks’ monitoring system. Our model combines dynamic factor VARs and quantile regressions techniques to construct forecasts of systemic risk indicators based on density forecasts, and employs stress testing as the measurement of the sensitivity of responses of systemic risk indicators to configurations of structural shocks.

This model can be viewed as a complementary tool to applications of DSGE models for risk monitoring analysis. As detailed in Schorfheide (2010), work on DSGE modeling is advancing significantly, but several challenges to the use of these models for risk monitoring purposes remain. In this regard, the development of DSGE models is still in its infancy in at least two dimensions: the incorporation of financial intermediation and forecasting. In their insightful review of recent progress in developments of DSGE models with financial intermediation, Gertler and Kyotaki (2010) outline important research directions still unexplored, such as the linkages between disruptions of financial intermediation and real activity. Moreover, as noted in Herbst and Schorfheide (2010), there is still lack of conclusive evidence of the superiority of the forecasting performance of DSGE models relative to sophisticated data-driven models. In addition, these models do not typically focus on tail risks. Thus, available modeling technologies providing systemic risk monitoring tools based on explicit linkages between financial and real sectors are still underdeveloped. Contributing to fill in this void is a key objective of this paper.

Three features characterize our model. First, we make a distinction between systemic real risk and systemic financial risk, based on the notion that real effects with potential adverse welfare consequences are what ultimately concerns policymakers, consistently with the definition of systemic risk introduced in Group of Ten (2001). Distinguishing systemic financial risk from systemic real risk also allow us to assess the extent to which a realization of a financial (real) shock is just amplifying a shock in the real (financial) sector, or originates in the financial (real) sector. Second, the model produces real-time density forecasts of indicators of real activity and financial health, and uses them to construct forecasts of indicators of systemic real and financial risks. To obtain these forecasts, we use a dynamic factor model (DFM) with many predictors combined with quantile regression techniques. The choice of the DFM with many predictors is motivated by its superior forecasting performance over both univariate time series specifications and standard VAR-type models (see Watson, 2006). Third, our design of stress tests can be flexibly linked to selected implications of DSGE models and other theoretical constructs. Structural identification provides economic content of these tests, and imposes discipline in designing stress test scenarios. In essence, our model is designed to exploit, and make operational, the forecasting power of DFM models and structural identification based on explicit theoretical constructs, such as DSGE models.

Our model delivers density forecasts of any set of time series. Thus, it is extremely flexible, as it can incorporate multiple measures of real or financial risk, both at aggregate and disaggregate levels, including many indicators reviewed in Bisias et al. (2012). In this paper we focus on two simple indicators of real and financial activity: real GDP growth, and an indicator of health of the financial system, called FS. Following Campbell, Lo and MacKinlay (1997), the FS indicator is given by the return of a portfolio of a set of systemically important financial firms less the return on the market. This indicator is germane to other indicators of systemic financial risk used in recent studies (see e.g. Acharya et al., 2010 or Brownlee and Engle, 2010).

The joint dynamics of GDP growth and the FS indicator is modeled through a dynamic factor model, following the methodology detailed in Stock and Watson (2005). Density forecasts of GDP growth and the FS indicator are obtained by estimating sets of quantile autoregressions, using forecasts of factors derived from the companion factor VAR as predictors.  The use of quantile auto-regressions is advantageous, since it allows us to avoid making specific assumptions about the shape of the underlying distribution of GDP growth and the FS indicator. The blending of a dynamic factor model with quantile auto-regressions is a novel feature of our modeling framework.

Our measurement of systemic risks follows a risk management approach. We measure systemic real risk with GDP-Expected Shortfall (GDPES ), given by the expected loss in GDP growth conditional on a given level of GDP-at-Risk (GDPaR), with GDPaR being defined as the worst predicted realization of quarterly growth in real GDP at a given (low) probability. Systemic financial risk is measured by FS-Expected Shortfall (FSES), given by the expected loss in FS conditional on a given level of FS-at-Risk (FSaR), with FSaR being defined as the worst predicted realization of the FS indicator at a given (low) probability level.

Stress-tests of systemic risk indicators are implemented by gauging how impulse responses of systemic risk indicators vary through time in response to structural shocks. The identification of structural shocks is accomplished with an augmented version of the sign restriction methodology introduced by Canova and De Nicolò (2002), where aggregate shocks are extracted based on standard macroeconomic and banking theory. Our approach to stress testing differs markedly from, and we believe significantly improves on, most implementations of stress testing currently used in central banks and international organizations. In these implementations, shock scenarios are imposed on sets of observable variables, and their effects are traced through "behavioral" equations of certain variables of interest. Yet, the ?shocked? observable variables are typically endogenous: thus, it is unclear whether we are shocking the symptoms and not the causes. As a result, it is difficult to assess both the qualitative and quantitative implications of the stress test results.

We implement our model using a large set of quarterly time series of the G-7 economies during the 1980Q1-2010Q1 period, and obtain two main results. First, our model provides significant evidence of out-of sample forecasting power for tail real and financial risk realizations for all countries. Second, stress tests based on this structural identification provide early warnings of vulnerabilities in the real and financial sectors.