Collateral Underwriter, Regression Models, Statistics, Gambling with your License

Size: px

Start display at page:

Download "Collateral Underwriter, Regression Models, Statistics, Gambling with your License"

Myron Park
5 years ago
Views:

1 Collateral Underwriter, Regression Models, Statistics, Gambling with your License Keith Wolf, SRA, AI-RRS Kwolf Consulting Inc. 05/20/2015

2 There are Lies, Damned Lies and Statistics (Mark Twain) Also known as False Positives in Collateral Underwriter or in Hard Science Hypothesis Testing, Type I errors

3 Collateral Underwriter UAD & CU Fail Without Data Integrity Vertical & Horizontal Inequity COD & PRD

4 Data Science, Statistics & Appraisers Statistics are Descriptive and Inferential Statistical Models Provide Suggested Solutions Mathematics, Statistics, Chemistry are Hard Sciences with rigid rules that can t be broken. Economics, Appraisal, Social Sciences are Soft Sciences: with rules that can be broken based on Professional Judgement. Data Science is a blend of Hard and Soft Science. Appraisers are Data Scientists that use Statistics tempered by Professional Judgement and Experience. Statistics are Probabilities with embedded Variance Analysis error rates with False Positive conclusions. Statistics DO NOT Estimate value, appraisers do.

5 APB Valuation Advisory #5 Identifying Comparable Properties In Automated Valuation Models for Mass Appraisal values will suffer when an important property attribute is either missing, incorrectly specified or calibrated during the modeling process. it is important for appraisers to learn about data exploration techniques, to avoid problems with missing data, and models that have been mis-specified, resulting in irrational and unexplainable calibration of a variable. estimate a value by using an iterative process of data and analysis Some off the shelf models are not iterative

6 Statistical Descriptive Tools Excel Pivot Table & Chart Year Built Average Price Subdivision

7 Statistical Descriptive Tools Tableau Table Year Built / MLS Type Type Subdivision & Price/ Design

8 MLS Type / Predominant Price Statistical Descriptive Tools StatWing MLS Type Area Breakdown 50% Percentages 45% 40% 35% 30% 25% 20% 15% 10% Percentage of Attached Single Percentage of Detached Single 5% 0%

9 Redstone: Bradford Technology Vertical Comparison

10 CU Provides Statistically-Derived Adjustments False Positive Appraisal Quality Progression/Regression, Define Material?

11 Standard 6 Mass Appraisal, Development and Reporting An Automated Valuation Model (AVM) is a mathematically based computer software program that produces an estimate of market value. Computer Assisted Mass Appraisal (CAMA) A system of appraising property, usually only certain types of real property, that incorporates computersupported statistical analyses such as multiple regression analysis and adaptive estimation procedure CU is a CAMA system with an embedded AVM

12 APB Valuation Advisory #5 Sample Model Structures - Direct Market Method Additive models have the form: MV = B0 + (B1 X1) + (B2 X2) + (Bxx Xxx) (Regression Coefficients) In a multiplicative model, the contribution of the variables are multiplied rather than added: MV = R0 * (X1^B1) * (X2^B2) *(Xxx^Bxx) (Rates, Multipliers, Ratios) Hybrid models are a combination of additive and multiplicative models. MV =πgq x [(πbq x Σ BA) + (πlq x Σ LA) + Σ OA] (Land & Building Components, VRA) A full discussion on the topic of model structures is available in the IAAO Standard on AVMs.

13 APB Valuation Advisory #5 valuation models require data representative of the full range of transaction prices and associated property characteristics in a defined area Client Supplementary Guidelines Bias Statistics through Confounding comps must bracket subjects features, be within 1 mile, 6 months, x% of GLA, Value must be within actual sale prices of comps and not above or below relies on market data from standardized property coding methods (UAD) different property coding standards produces different results The appraisers, skill, knowledge, and experience play a role in knowing which and much weight to assign an attribute/variable leading to CREDIBILITY.

14 Appraiser Independence? Coefficient of Dispersion can derive net and gross adjustment % from the market

15 CREDIBILITY The sales selected for the adjustments have attributes closest in the defined categories collectively and are placed on the adjustment grid. (REMOVE THE CONFOUNDING) Defining the metric selection is a skill in defining attribute importance and magnitude of metric selection that will improve the appraiser s impartiality from personal judgment with an objective method. Since the coefficients (adjustment rates) are derived from the transactions and have been tested using various statistical methods within the modeling process, the appraiser has presented a level of independence, impartiality, and objectivity Comparable selection must be emphasized over the adjustment process (NO MORE 10%, 15% 25% ILLUSIONARY QC RULES THAT VIOLATE APPRAISER INDEPENDENCE) the importance of finding comparable that are as similar as possible to the subject greatly reduces the need for making many (sometimes controversial) adjustments to the sales prices.

Confidence Interval A 95% confidence level is 2 Standard deviations from the mean therefore 95% of the combinations fall under the Curve 2 standard Deviations 1 Standard Deviation Statistics has a

16 Confidence Interval A 95% confidence level is 2 Standard deviations from the mean therefore 95% of the combinations fall under the Curve 2 standard Deviations 1 Standard Deviation Statistics has a concept called the third variable, Confounding. Confounding is the third variable in a probability matrix with no easy way of segmenting out its corresponding contribution to value. ANOVA is a method for testing differences among means by analyzing variance. The difference in the mean could be the adjustment supported by paired means for the difference in Characteristics.5 Standard Deviation

17 Statistical Inferential Analytical Tools Statwing Redstone

Statistical Inferential Analytical Tools Excel AVT SUMMARY OUTPUT Regression Statistics Multiple R 0.973264715 R Square 0.947244205 Adjusted R Square 0.876903145 Standard Error 88407.

18 Statistical Inferential Analytical Tools Excel AVT SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 29 ANOVA df SS MS F Regression E E Residual Total E+12 Coefficients Standard Error t Stat P-value Intercept -11,755, Closed Date Stories 10, LMT (DOM) ASF # Rms -19, Bsmt. Beds Bsmt. Beds Beds 80, # Full Baths 52, # Half Baths Yr Blt 2, Recent Rehab 28, # Garage Spaces -129, Lot Sq Ft # Interior Fireplaces 114, Basement Finish 33, Subdivision -39,

19 Statistical Inferential Analytical Tools

20 Model Result Comparisons Excel AVT Statwing Redstone Pairs R Square Standard Error 88,407 79,060 87,923 Observations Variable Adjustment Closed Date Stories 10,887-4,412 42,936 LMT (DOM) ASF # Rms -19,716-35,518 4,104 Bsmt. Beds 206, , ,837 Beds 80, ,589 # Full Baths 52,338 35, ,583 70, ,313 # Half Baths ,851 22,286 Yr Blt 2,891 3,728 7,515 1,505 6,205 Recent Rehab 28,249 51,582 35,520 98,617 # Garage Spaces -129, ,268-56,703-38,332 Lot Sq Ft # Interior Fireplaces 114, , , ,149 Basement Finish 33,916 22,584 28,356 Subdivision -39,226-31,136-59,280

21 Case Study Subject Sale 1 Sale 2 Sale 3 Sale 4 Sale 5 Sold Pr $ 312,000 Adjustment $ 367,000 Adjustment $ 382,500 Adjustment $ 423,250 Adjustment $ 430,000 Adjustment $ Per Sq Foot LMT Closed Date 4/29/2014 1/9/2015 8/1/2014 4/30/2014 7/16/2014 # Rms Beds # Full Baths # Half Baths ASF ,243 2,187 2,200 2,367-36,000 2,220 Yr Blt Bsmt Full Full Full Partial Partial Full Bsmt Finish Unknown Unfinished Unfinished Rec Room Unfinished Bed,den,office Lot Size ,451 29,000 10,800 10,296 11,851-8,500 11, Garage Spaces Type 2 Stories 2 Stories 2 Stories 2 Stories 2 Stories 2 Stories Other Patio Patio Patio Screen Porch Patio Patio Condition C3 C3 C3 C3 C C3 Net/ % 23,200 7% (30,100) -8% (48,700) -13% (57,400) -14% (94,900) -22% Gross/% 74,800 24% 30,100 $ ,700 13% 57,400 14% 94,900 22% Adjusted Price $ 335,200 $ 336,900 $ 333,800 $ 327,850 $ 335,100 Avg Regression Adj Average Overall Price per square foot from sales comparison approach Conditiion 38,000 See what is in RED, think we have a GLA Adjustment Error? Bsmt Finish 28,000 Land 7.00 Our R Squared says GLA is 59%, therefore should it be $ and not the average of the Regression Models? Year Built 4,300 Marshall & Swift Depreciated Cost New $ Avg GLA $Adj 176 Average of R square and Depreciated cost New $ rounded to $114 Match Pair 1 to 2 Bed Adj 20,000 Avj Adj $ of 4 comps compared to 5 24,000 If we change the GLA adjustment to $114 per sq ft we get a new adjusted value for comp 4 at $340,252. Screen Porch Avg Adj comps compared to 3 9,600

22 Appraisal Uniformity Vertical Equity Comparison Method Appraisal Zestimate NAR AVM Value $335,560 $457,633 $660,780 SER/COD A required component in ad valorem mass appraisal is demonstrating the appraisal uniformity or equitable treatment of identical or similar properties during the valuation process. A comparison method available that will reveal uniformity of values and equitable treatment is vertical equity analysis, with the caveat that the appraised value of the subject is substituted for the assessed value in the ratio analysis. This method provides users with an instant view of how the subject property is valued in comparison to other properties and how the adjustment rates affect the value

23 SUMMARY Appraisers are provided little guidance to help them with what the most comparable property actually means. As AVMs and CAMA become more universal, the idea of using statistical measures to evaluate comparability will be an acceptable alternative to the previous manual and subjective method. The Appraisal Industry and Mortgage Financial Services lack the education and experience to apply the methodology described; for the last 30+ years we have relied on illusionary false positives of appraisal quality using guidelines 10%, 15%, 25%, 6 months, 1 mile, 20% GLA variance Appraisers, Review Appraisers, Chief Appraisers & Underwriters all need to re-tool for these methods to become standard protocol. Many appraisers will use these tools without understanding the what, why and when associated with identifying credible results.

86 years in the making Caspar G Haas 1922 Sales Prices as a Basis for Estimating Farmland Value

2 Our Journey Begins 86 years in the making Caspar G Haas 1922 Sales Prices as a Basis for Estimating Farmland Value Starting at the beginning. Mass Appraisal and Single Property Appraisal Appraisal