Techniques for Verifying the Accuracy of Risk Measurement Models

TLDR: In this paper, the authors consider the formal statistical procedures that could be used to assess the accuracy of value at risk (VaR) estimates and show that verification of the accuracy becomes substantially more difficult as the cumulative probability estimate being verified becomes smaller.

Abstract: Risk exposures are typically quantified in terms of a "value at risk" (VaR) estimate. A VaR estimate corresponds to a specific critical value of a portfolio's potential one-day profit and loss distribution. Given their functions both as internal risk management tools and as potential regulatory measures of risk exposure, it is important to assess and quantify the accuracy of an institution's VaR estimates. This study considers the formal statistical procedures that could be used to assess the accuracy of VaR estimates. The analysis demonstrates that verification of the accuracy of tail probability value estimates becomes substantially more difficult as the cumulative probability estimate being verified becomes smaller. In the extreme, it becomes virtually impossible to verify with any accuracy the potential losses associated with extremely rare events. Moreover, the economic importance of not being able to reliably detect an inaccurate model or an under-reporting institution potentially becomes much more pronounced as the cumulative probability estimate being verified becomes smaller. It does not appear possible for a bank or its supervisor to reliably verify the accuracy of an institution's internal model loss exposure estimates using standard statistical techniques. The results have implications both for banks that wish to assess the accuracy of their internal risk measurement models as well as for supervisors who must verify the accuracy of an institution's risk exposure estimate reported under an internal models approach to model risk.