Evaluating early warning indicators of banking crises: Satisfying policy requirements

Evaluating early warning indicators of banking crises: Satisfying policy requirements
By Mathias Drehmann and Mikael Juselius
BIS Working Papers No 421

Aug 2013

http://www.bis.org/publ/work421.htm
 

Early warning indicators (EWIs) of banking crises should ideally be evaluated on the basis of their performance relative to the macroprudential policy maker's decision problem. We translate several practical aspects of this problem - such as difficulties in assessing the costs and benefits of various policy measures as well as requirements for the timing and stability of EWIs - into statistical evaluation criteria. Applying the criteria to a set of potential EWIs, we find that the credit-to-GDP gap and a new indicator, the debt service ratio (DSR), consistently outperform other measures. The credit-to-GDP gap is the best indicator at longer horizons, whereas the DSR dominates at shorter horizons.

JEL classification: C40, G01
Keywords: EWIs, ROC, area under the curve, macroprudential policy


Excerpts:

In the empirical part of the paper, we apply our approach to assess the performance of 10 different EWIs. We mainly look at the EWIs individually, but at the end of the paper we also consider how to combine them. Our sample consists of 26 economies, covering quarterly time series starting in 1980. The set of potential EWIs includes more established indicators such as real credit growth, the credit-to- GDP gap, growth rates and gaps of property prices and equity prices (eg Drehmann et al (2011)) as well as the non-core liability ratio proposed by Hahm et al (2012).  We also test two new measures: a country’s history of financial crises and the debt service ratio (DSR). The DSR was first suggested in this context by Drehmann and Juselius (2012) and is defined as the proportion of interest payments and mandatory repayments of principal to income. An important data-related innovation of our analysis is that we use total credit to the private non-financial sector obtained from a new BIS database (Dembiermont et al (2013)).

We find that the credit-to-GDP gap and the DSR are the best performing EWIs in terms of our evaluation criteria. Their forecasting abilities dominate those of the other EWIs at all policy-relevant horizons. In addition, these two variables satisfy our criteria pertaining to the stability and interpretability of the signals. As the credit-to- GDP gap reflects the build-up of leverage of private sector borrowers and the DSR captures incipient liquidity constraints, their timing is somewhat different. While the credit-to-GDP gap performs consistently well, even over horizons of up to five years ahead of crises, the DSR becomes very precise two years ahead of crises. Using and combining the information of both indicators is therefore ideal from a policy perspective. Of the remaining indicators, only the non-core liability ratio fulfils our statistical criteria. But its AUC is always statistically smaller than the AUC of either the credit-to-GDP gap or the DSR. These results are robust with respect to different aspects of the estimation, such as the particular sample or the specific crisis classification used.