Share these talks and lectures with your colleaguesInvite colleagues
Assessment of model risk in the aggregate: Contribution of quantification
The ability to assess model risk in the aggregate is desirable, as it provides senior management with information on the overall risk associated with models used by the organisation. More importantly, it enables effective challenges and hence a robust measurement by connecting isolated information in a transparent way. This paper introduces a quantitative element to the assessment framework that utilises various model statistics from model development and performance monitoring periods. The paper links the model life cycle concept to risk quantification within families of similar models by introducing two risk measures: the Model Robustness Index (MRI) and the Model Stability Index (MSI). MRI is used to evaluate individual models’ robustness and goodness of fit. MSI employs key ongoing monitoring metrics and can be used along with judgmental factors as a dynamic measure of risk as model performance changes over time. Next, the paper introduces an approach to establish thresholds for ongoing performance monitoring. The paper demonstrates the value of these concepts by applying them to an example using binary logistic regression, that is, a simple application of a machine learning algorithm.
The full article is available to institutions that have subscribed to the journal
Liming Brotcke is a quantitative manager at the Federal Reserve Bank of Chicago. She leads a team that supports large and Large Institution Supervision Coordinating Committee (LISCC) bank supervision across the Federal Reserve System. She has extensive modelling experience in the consumer lending and sufficient working knowledge of other modelling areas including wholesale, securities, market and liquidity, derivatives, and operational. Prior to joining the Fed, she worked at Citi Group and Discover Financial Services developing models and managing portfolios.
Raymond Brastow is a senior financial economist at the Federal Reserve Bank of Richmond. He participates in model reviews at large banks including the qualitative assessment of stress test models. Ray has actively published in academic journals, most recently in the field of residential real estate transactions. He has taught economics at several universities and is currently Emeritus Professor of Economics at Longwood University in Virginia.
CitationBrotcke, Liming and Brastow, Raymond (2018, December 1). Assessment of model risk in the aggregate: Contribution of quantification. In the Journal of Risk Management in Financial Institutions, Volume 12, Issue 1.