An RMA Publication THE JOURNAL OF ENTERPRISE RISK MANAGEMENT. November 2017 rmahq.org. STRATEGIES FOR C&I LENDING p. 38

Transcription

1 An RMA Publication n o i ss i M omplished THE JOURNAL OF ENTERPRISE RISK MANAGEMENT November 2017 rmahq.org Acc STRATEGIES FOR C&I LENDING p. 38

2 ENTERPRISERISK RMA/AFS PROGRAM HELPS BANKS IMPROVE AND MAINTAIN CCAR DATA QUALITY In 2013, RMA and AFS decided to address banks growing frustration with ever-evolving CCAR reporting requirements. The goal was to develop a program to harness the collective resources of the industry, bring greater transparency and understanding to the data collection process, and create an anonymous way to benchmark regulatory data quality and track progress. This article describes that journey. BY WALTER YOUNG, JEREMY CHALSON, AND DOUG SKINNER Many RMA Journal readers remember well, perhaps painfully so, the first few years of the Federal Reserve s Comprehensive Capital Analysis and Review (CCAR) program, whose growing pains created havoc and frustration among the banks obliged to participate. There were numerous complaints about vague requirements and inadequate communication from the Fed, which in November 2011 had issued the final rule requiring all U.S. bank holding companies with more than $50 billion in assets to submit annual capital plans for review. That rule also introduced the FR Y-14Q data collection and reporting forms, 24 The RMA Journal November 2017 November 2017 The RMA Journal 25

3 ENTERPRISERISK RMA/AFS PROGRAM HELPS BANKS IMPROVE AND MAINTAIN CCAR DATA QUALITY In 2013, RMA and AFS decided to address banks growing frustration with ever-evolving CCAR reporting requirements. The goal was to develop a program to harness the collective resources of the industry, bring greater transparency and understanding to the data collection process, and create an anonymous way to benchmark regulatory data quality and track progress. This article describes that journey. BY WALTER YOUNG, JEREMY CHALSON, AND DOUG SKINNER Many RMA Journal readers remember well, perhaps painfully so, the first few years of the Federal Reserve s Comprehensive Capital Analysis and Review (CCAR) program, whose growing pains created havoc and frustration among the banks obliged to participate. There were numerous complaints about vague requirements and inadequate communication from the Fed, which in November 2011 had issued the final rule requiring all U.S. bank holding companies with more than $50 billion in assets to submit annual capital plans for review. That rule also introduced the FR Y-14Q data collection and reporting forms, 24 The RMA Journal November 2017 November 2017 The RMA Journal 25

4 FIGURE 1: RMA/AFS DATA QUALITY INDEX Aggregate RMA/AFS Data Quality Index Corporate and CRE Loans 100% 99% 98% 97% 96% 95% 94% 93% 92% 91% 90% 97.1% 97.0% 96.4% 96.7% 96.9% 1Q16 2Q16 3Q16 4Q16 1Q17 Aggregate RMA/AFS Data Quality Index Corporate and CRE Loans - 1Q17 Market Bank A Bank B Bank C Bank D Bank E Bank F Bank G Bank H 91.3% 92.5% 95.5% 95.9% 95.1% 96.9% 97.5% 98.0% 99.0% 90% 92% 94% 96% 98% 100% which greatly increased the scope and scale of loan data required for CCAR in an effort to promote consistency of data across participating banks. From there, CCAR reporting requirements evolved rapidly, leading to a seemingly constant stream of new data elements, revisions to existing ones, and modified definitions. The velocity and depth of the changes (many of which required significant alterations to technology and business processes), along with the Fed s perceived lack of specificity and communication, caused tremendous frustration for banks. This disconnect, arguably, also contributed to a much higher level of qualitative failures in the early years of CCAR compared to today. The Birth of CCAR Data-Quality Benchmarking In April 2013, about a year and a half after the final rule was issued, RMA and AFS gathered together a group of banks to discuss these challenges and define a program that would accomplish three primary objectives: 1. Bring greater transparency and understanding to the CCAR data collection process and share emerging best practices. 2. Pool the data being gathered and leverage it for the collective benefit of banks risk management needs. 3. Create a means to benchmark individual banks data quality against peer reporters. The program would be a bank-led and -defined program. There would be regular workshops to review results, discuss new data requirements and changes, and examine ways banks could leverage data (viewed by some as having little utility beyond regulatory reporting) to enhance credit risk management practices and even provide market intelligence to drive growth and profits. FIGURE 2: PERCENT OF RECORDS WITH KEY DATA-POINT CHANGES Corporate Loans: Percent of Records - 2-Digit NAICS Changes % of records displaying code change 4Q16 to 1Q17 CRE Loans: Percent of Records - Property Location Changes % of records displaying code change 4Q16 to 1Q17 3.5% 3.0% 3.05% 3.5% 3.0% 2.5% 2.0% 1.60% 1.5% 1.0% 0.5% 0.0% Market Bank A 2.33% 2.02% 1.54% 1.62% 1.18% 0.69% 0.35% Bank B Bank C Bank D Bank E Bank F Bank G Bank H 2.5% 2.0% 1.5% 1.0% 0.98% 1.10% 0.64% 0.57% 0.61% 0.5% 0.38% 0.43% 0.23% 0.15% 0.0% Market Bank A Bank B Bank C Bank D Bank E Bank F Bank G Bank H On Previous Page: Shutterstock.com 26 The RMA Journal November 2017

5 But First, What Is Data Quality? The term may appear at first blush to be self-evident, but defining data quality for purposes of comparison and tracking is challenging. We wanted to be able to compare data quality consistently across the participating banks. Since all banks depend on the same types of data for both risk management and regulatory reporting (risk ratings, industry codes, collateral information, etc.), we focused on a few key characteristics that would be common to all banks and could be tracked and measured accordingly. Data Completeness Data completeness was arguably the easiest to define (if not the easiest to achieve). We developed a Data Quality Index, which reflects the amount of missing or invalid values a bank has in a given file. For example, an index reading of 90% means 10% of the information was either missing or invalid. Given the thousands of wholesale loan facilities per bank requiring over 150 different data attributes each, you are talking about a lot of opportunities for missing or invalid data! Consider the Fed s propensity to add new fields and modify definitions to existing ones, and you begin to appreciate the challenge banks have in obtaining and maintaining high-quality, reliable filings. Figure 1 shows an example of our Data Quality Index comparisons. The index achieves two important goals. First, it allows banks to see a baseline level of quality and determine whether theirs is improving, deteriorating, or staying the same. Second, it gives them a horizontal comparison to peers to determine if they are ahead of the curve, meeting the status quo, or falling behind. Data Consistency Another important measure of data quality is data consistency. While some data points (like the loan balance) change constantly, others tend to remain relatively constant. Material changes in their values period-to-period can indicate an operational error or even a more systemic problem. Take, for example, a borrower s industry NAICS code or the zip code of a real estate property. These data points should remain constant throughout the life of a loan (Figure 2). Even the appraised value of a commercial property would not be expected to change quarterly. We can measure period-to-period changes in these types of fields and identify change volumes that exceed a normal experience. Those outliers can alert us to look for transactional or operational weaknesses that may be causing those changes. Why is this important? For starters, industry codes, geographic locations, and property values are all critical inputs to default models, stress testing, and capital adequacy and allocation planning. Information systems and reporting not only have to record data accurately at the origination of the transaction, they have to keep it accurate throughout the life of the loan. Regular monitoring of data consistency can help identify irregularities that emerge. Data Reasonableness We ve all seen data sets that are complete and consistent from the prior period, but somehow just don t pass the smell test. Some would call this the sanity check. We can address this by looking at the distributions of our data sets by certain segmentation criteria. Some data sets might lend themselves to relatively normal distributions, such as a loan portfolio s risk rating distribution. FIGURE 3: DSCR DISPERSION BY BANK Debt Service Coverage Rate (DSCR) Debt Service Dispersion Coverage by Rate CRE (DSCR) Records Dispersion by CRE Records 75th Percentile Median 25th Percentile DSCR Market Bank A Bank B Bank C Bank D Bank E Bank F Bank G Bank H November 2017 The RMA Journal 27

6 FIGURE 4: DATA-QUALITY HEAT MAP: MISSING VALUES BY DEAL SIZE Sample Bank Sample Peer Group <$1 M <$1 M <$5 M <$5 M <$10 M <$10 M <$25 M <$25 M <$50 M <$50 M <$100 M <$100 M >$100 M >$100 M Critical (High) Critical (Medium) Critical (Low) Moderate (High) Moderate (Medium) Moderate (Low) Minimal (High) Minimal (Medium) Minimal (Low) Critical (High) Critical (Medium) Critical (Low) Moderate (High) Moderate (Medium) Moderate (Low) Minimal (High) Minimal (Medium) Minimal (Low) 0% %-1% - 5%-6% - > >=10 = 10% % Missing High % M Low % Missing High % Missing For something like this, data anomalies (such as a disproportionate amount of data with an exceptionally high probability of default, for example) can alert us to a possible data error. Take, for example, the illustration in Figure 3, a comparative dispersion of commercial real estate debt service coverage ratios (DSCR) across banks. Banks B and F have a median DSCR of only 0.8, which implies that a significant amount of the borrowers for those banks are not generating enough income to service their loans (< 1.0). Outside of a period of prolonged economic crisis, this would be highly unusual and worth a closer look. Given the number of variables that go into a DSCR (net operating income, the interest rate, the term / amortization schedule, etc.), many data points can cause problems. Looking at dispersions of key fields and ratios by bank can alert us to possible data-entry errors. Other reasonableness tests rely on the bank s knowledge of its own portfolio, such as industry mix and concentrations, percentage of unsecured credit, mix of funded versus unfunded exposure, or One challenge in tackling a comprehensive dataquality issue is the tendency to want to solve for everything immediately, or boil the ocean. perhaps exposure to a higher-risk category of lending such as construction lending. By tracking these types of data distributions regularly, we can identify data anomalies and also submission control / XML problems early for research and remediation before they result in any negative repercussions for the bank s data quality or reputation. Developing Effective Tools to Benchmark and Track Progress One challenge in tackling a comprehensive data-quality issue is the tendency to want to solve for everything immediately, or boil the ocean. An ability to agree on and prioritize pain points is key to making progress. Unfortunately, the Fed does not tell banks which fields are most important in terms of driving their full stress-testing results. So RMA and AFS worked with the banks to develop a system for ranking (loan) field importance based on the FR Y-14Q schedule requirements to differentiate the relative importance and impact on stress testing and related modeling of each field. For example, most would agree that fields like borrower risk rating and industry code must weigh more heavily in stress test modeling than do, say, the CUSIP number or the company s stock symbol (all of which are required CCAR fields). A ranking system allows banks to prioritize pain points, identifying which fields to work on first. Pain points can then be further segmented by the size of the exposure, or by the organization s 28 The RMA Journal November 2017

7 TABLE: CCAR EXECUTIVE SUMMARY SCORECARD RMA/AFS BUSINESS INTELLIGENCE CATEGORY MIN BANK MAX BANK CRE Data Quality Corporate Data Quality Wtd Avg Data Quality Index 93.68% 95.72% 96.04% Percent of Exposure - Other Security Type 0.07% 7.61% 7.67% Percent of Records - 2-Digit NAICS Changes 0.58% 0.19% % of records displaying code change quarter-over-quarter 0.36% Percent of Records - Zip Code Changes 0.61% 0.15% % of records displaying code change quarter-over-quarter 0.61% Wtd Avg Data Quality Index 99.40% 99.85% 99.85% Percent of Exposure - Other Loan Purpose 0.08% 8.12% 19.20% Percent of Records - Property Type Changes 0.36% 0.11% % of records displaying code change quarter-over-quarter 0.39% Percent of Records - Property Location Changes 0.96% 0.05% % of records displaying code change quarter-over-quarter 0.88% Note: Actual scorecard has been condensed for purposes of this article. Sample Bank Medium Source: RMA/AFA CCAR Benchmarking Program 99.70% 30.61% 3.17% 1.24% 99.95% 52.88% 1.22% 2.66% managerial or geographic hierarchy, so banks can pinpoint the areas of greatest need and get the most immediate return from data remediation. The heat maps in Figure 4 show the sample bank on the left with a concentration of missing values for critical, highimpact fields for some of the largest borrowers in the bank (>$50M in deal size). With a standard field-ranking system in place, we can rank-order data quality at the loan level, track progress from period to period, and compare results across banks. Further, our Data Quality Index gives us our singular, comprehensive measure of portfolio data quality. For chief data officers, the index shows how the bank measures up against its peers and the market, while helping banks focus on the largest customer relationships that might have data problems. Executive-level Visibility Another key component of a sound data-quality program is regular reporting to and visibility with executive management. For this purpose, RMA and AFS developed a comprehensive Data Quality Another key component of a sound dataquality program is regular reporting to and visibility with executive management. and Credit Risk Scorecard, as shown in the Table. An executive-level scorecard provides the answer to the questions Are we getting better or at least making progress? and How are we doing compared to our peers? Bringing It All Together: What Is the Business Value? At the top of this article, we noted how simply defining data quality can be challenging. Illustrating the business value can be equally daunting. Fortunately, the Federal Reserve has helped us out. In its annual CCAR instructions, the Fed explicitly discusses the consequences of deficient data quality, stating in part that if the quality of a bank s submitted data is deemed too deficient, the Fed could assign a high loss rate (90th percentile value) to the portfolio balances to complete the stress testing. You could then think of that as the true cost of poor data quality. Take the example in Figure 5. As of the first quarter of 2017, the median probability of default (PD) for all corporate exposures was 88 basis points, or 0.88%. However, the 90th percentile value was 441 basis points, or 4.4%. Say a bank had material issues with missing, invalid, or erroneous PDs. Per the Fed s instructions, it reserves the right to substitute an average PD value of 4.4% to the portfolio balances for purposes of stress testing capital requirements! A 4.4% probability of default is solidly in that speculative, special-mention category. The implications of this to the November 2017 The RMA Journal 29

8 FIGURE 5: DISTRIBUTION OF CORPORATE LOAN RECORDS BY PROBABILITY OF DEFAULT (PD) Corporate Loans - 1Q17 FREQUENCY 16.0% 14.0% 12.0% 10.0% 8.0% 6.0% 4.0% 2.0% 0.0% 0.15% 0.25% 0.35% 0.50% 0.85% 1.35% bank s asset-quality profile, adequacy of loan loss reserves, and projected capital needs are enormous, costing the bank millions of dollars in capital that could otherwise be used to support growth plans or returns to shareholders. Weaknesses in indicative data are equally punitive. As discussed earlier, net operating income (NOI) is a key indicator of a commercial property s health. It is also a key input into the debt service FIGURE 6: DISTRIBUTION OF CRE LOAN RECORDS BY DSCR CRE Loans - 1Q17 FREQUENCY 14% 12% 10% 8% 6% 4% 10TH PERCENTILE: % PD 2.70% MEDIAN: 1.42 MEDIAN: 0.88% 4.00% 5.50% 90TH PERCENTILE: 4.41% 7.00% 10.00% 20.00% % coverage ratio, which is an indicator of the adequacy of a property s operating cash flow to service debt obligations. Yet maintaining a current property NOI is often a pain point for a bank and not always readily available from bank information systems. Take the example in Figure 6. As of the first quarter of 2017, the median DSCR for commercial real estate loans was However, the 10th percentile value was just If a bank has sparse, dated, or otherwise unreliable property NOIs or any other input into the DSCR the Fed may substitute a DSCR of just 0.45, which is equivalent to saying the property does not throw off enough cash to pay back the lender. These properties may very well be modeled for CCAR purposes as defaulted obligations, or at a minimum modeled using a much higher PD than may be accurate. Punitive indeed! Impact Internal bank studies have shown, depending on bank size and the amount of missing data, that each 1% improvement in data quality for critical fields can reduce Fed loss rates from 25 to 200 basis points. The analyses can also be used to measure credit-risk-adjusted returns across peers and the banking industry. Conclusion The CCAR Data Quality Benchmarking Program from RMA and AFS provides far more than a one-time compliance benefit to banks that deploy it. Its greatest benefit lies in its ongoing business value as an extremely effective tool to first improve and then sustain data quality. Through its collaborative workshops and continuing introduction of new metrics, charts, and insights, the program provides a constantly evolving, dynamic resource that aids in the discovery, tracking, and resolution of loan-data problems. Even if Congress or the regulators scale back the intensity of CCAR reviews for some institutions, a program such as this can remain a key part of a bank s strategic arsenal for data-quality management. For more information on the RMA/AFS Data Quality Benchmarking Program, please contact Doug Skinner at AFS at or Mark Zmiewski at RMA at % 0% DSCR Over 4.0 Note 1. Board of Governors of the Federal Reserve System, Dodd-Frank Act Stress Test 2017: Supervisory Stress Test Methodology and Results, June 2017, p The RMA Journal November 2017