模型验证指南（普华永道）

PwCWorking Together:PwC Model Validation Services GuideSlide 2 2008 PricewaterhouseCoopers.Scope of our AssistancePwC offers a full range of model risk management services to assist you in the following areas:Assistance with independent model validations for a wide range of model typesEvaluation of model governance programs for effectiveness and for consistency with regulatory expectations and industry leading practicesDesign and development of model governance programs including: Organizational design Development of model validation policies and procedures Resource planning and analysisModel Risk Management ServicesSlide 3 2008 PricewaterhouseCoopers.PwC model risk management capabilitiesHighly Experienced Team of Model Risk Specialists We have a broad and deep team of model risk professionals whose experience covers virtually all financial model types including those used to manage credit risk, market risk, operational risk, and compliance risk as well as those used for financial reporting, valuations, and economic capital estimation. Significant Exposure to, and Driver of, Industry Leading Practices Our unique combination of advisory and audit-related model validation work for many of the largest global financial services firms provides us with significant exposure to leading practices in the areas of model validation and model risk management. Additionally, PwC has been at the forefront of the professional services industry in developing and refining approaches to implement the Office of the Comptroller of the Currencys (“OCCs”) 2000-16 guidance on Model Validation for various model types.Extensive Thought Leadership PwC is an industry leader in sharing our point-of-view on model risk management issues through published articles in American Banker and Bank Accounting & Finance, through our FS Regulatory white paper series, and through invited presentations at industry conferences. PwC is uniquely qualified to deliver industry-leading model risk management servicesDeep Knowledge of Regulatory ExpectationsPwC has developed strong relationships with the U.S. bank regulatory agencies and has met regularly with the OCC to discuss both existing and emerging model risk issues. PwC has also provided training to both Federal Reserve and OCC examiners in the area of model risk and model validation.Slide 4 2008 PricewaterhouseCoopers.Our Model Risk Management Skills and ExperiencePwCs model risk specialists combine deep technical expertise with highly effective model validation approaches across a broad spectrum of model types. Additionally, our experience working in both internal and external audit environments ensures that our work is risk-focused, efficient, and well-documented.Design and Operating Effectiveness of Model ControlsRegulatory and Accounting Knowledge Basel II models (PD, LGD, EAD, RWA)Financial Reporting Models such as FAS 91 amortization, SOP 03-3, FAS 5 reserves, etc.Vendor models such as QRM, AdCo / AFT prepayment models, AdCo Loan Dynamics, LoanPerformance RiskModel, etc.Advanced Financial Valuation and Risk ModelsDeep Credit and Prepayment Model ExperienceMonte Carlo Simulation MethodsLarge-scale data processing and managementAdvanced degrees in Economics, Mathematics / Statistics, and FinanceDiverse Computer Programming and Platform ExperienceDerivative Valuation ModelsStatistical ModelingSpecialistsDeep Model Governance ExperienceCommercial and Consumer Lending ModelsExperience Working in anAudit EnvironmentDeep Regulatory RelationshipsPwC Model Risk ManagementSlide 5 2008 PricewaterhouseCoopers.PwC Technical / Industry ExperiencePwC has significant experience with a variety of computer systems and computing platforms:Our team has deepknowledge of thefollowing industrysectors / products:All MortgageClassesAuto Loansand LeasesMBS and Other Structured FinanceStudent LoansCommercialLoansCredit CardsDerivativesand OtherFinancial AssetsOur experiencecovers a widerange of modeltypes:Statistical / Mathematical /Econometric ModelsTransition Matrixand Roll RateModelsOption-based Valuation Modelsand ComplexDiscounted CashFlow ModelsCredit Scorecardsand Basel IIEconomic Capital ModelsVendor Credit,Prepayment,Valuation, andRisk ManagementModelsAccountingEngines / Models(FAS 91, SOP 03-3,FAS 5,etc.)MicrosoftACCESSMatLabSQLSASMicrosoft Excel / VBAC / C+Windows and UNIX Operating SystemsSlide 6 2008 PricewaterhouseCoopers.PwC model risk management specialists have significant experience with many third-party credit, prepayment, and valuation models and platforms for example: PwC Experience Third-Party Models and Platforms Strategic Analytics Bloomberg Swap Manager (SWPM) Bloomberg Credit Default Swap Tool (CDSW) SuperDerivatives Platform Principia Structured Finance Platform (SFP) Murex MX Platform PolyPaths Platform OpenLink Findur Platform FINCAD Platform INTEX Platform Blackrock Models MSCI Barra Models QRM Platform MIAC Accutape and WinOAS LoanPerformance Risk Model AdCo Prepayment and Credit Models AFT Prepayment Models SS&C Debt & Derivative System S&P LEVELS Moodys KMV Models Moodys M3 ModelSlide 7 2008 PricewaterhouseCoopers.PwC Model Validation ApproachPwC follows a structured and auditable approach designed to ensure that relevant model risks are identified and tested. We also can perform our work according to your companys specific guidelines and procedures.Issue Identification: Document and communicate to model owners any issues identified by the review Assess criticality of the issues and document the rationale for the assessment Assist model owners in identifying the best approach for resolving each issueRisk Assessment: Perform a walk-through of the model methodology with the model owners Construct the process flow chart from data extraction, to model estimation / calculations, to model outputs use Identify and document relevant risks associated with the modelScope and Approach: Determine high-level scope and goals Determine the appropriate testing approach for each model risk Design a detailed work plan for addressing the relevant model risksDocumentation stage: Document review work and findings in a detailed and auditable manner Provide value-added recommendations focusing on: Addressing identified issues Enhancing model accuracy and efficiency Reducing operational risks Strengthening controlsIssue identification and resolutionModel risk assessment ExecutionValidation scope and testing approachDocumentation and recommendationsSlide 8 2008 PricewaterhouseCoopers.PwC Model Validation Testing ActivitiesWhile PwCs specific model validation testing activities are customized to each model, the following presents an example of the types of testing activities we are frequently engaged to perform:Model Controls Evaluating reasonability of the process used to monitor the on-going quality of model estimates Evaluating completeness and accuracy of model documentation Evaluating the reasonability of governance processes around key model assumptions Evaluating effectiveness of model change management processes Evaluating effectiveness of model operational controlsModel Development Testing completeness / accuracy of internal and external model development data Evaluating the reasonability / consistency of the model design relative to typical industry approaches for the specific type of model Evaluating appropriateness and execution accuracy of statistical estimation (or financial valuation) techniques Evaluating sensibility and significance of estimated model relationships Evaluating statistical validity of estimated models and overall goodness-of-fit Performing back-testing and benchmarking to evaluate reasonability of model estimatesModel Implementation Evaluating consistency of model production application with final model development specifications (e.g., data sources, variable definitions, and mathematical equations) Evaluating reasonability of support for all model assumptions Testing completeness and accuracy of model production application by building an independent replica and performing a parallel test Testing completeness and accuracy of the models integration with upstream and downstream models and / or data warehousesSlide 9 2008 PricewaterhouseCoopers.PwC is an Industry-Leader in Model Risk Management Thought LeadershipSlide 10 2008 PricewaterhouseCoopers.Specific PwC Model Risk Management Thought Leadership “Automated Valuation Models: Changes in the Housing Market Require Additional Risk Management Considerations,” PwC White Paper, January 2008. “Why is a Model Validation Program Necessary in a Sarbanes-Oxley Environment?”PwC Banking Issues Update, Fall 2004. “Model Validation: A Key Tool for Sarbanes-Oxley Section 404 Internal Controls Assessment,” PwC Consumer Finance Update, Winter 2004. “Model Validation: Deficiencies, Common Problem Areas, and Reporting,” PwC Banking and Specialty Finance Roundtable. “Controls Over Complex Financial Models: Common Deficiencies,” PwC Banking and Capital Markets Issues, Spring 2007. “Managing Model Risk,” invited presentation at the Opal Housing Finance Summit, February 2007. “Model Validation for Safety and Soundness,” PwCs International Banking and Capital Markets Conference, October 2005. “Navigating Emerging Model Risks: Model Validation in the Current Economic Environment,” PwC White Paper and invited presentation at MBAA Accounting, Tax, and Financial Analysis Conference, November 2007. “The Challenge of Keeping Models Up-to-Date,” American Banker, January 2008. “The State of Model Risk Management,” Bank Accounting & Finance, June 2008.Examples of PwCs Model Risk Management Experience and Value PropositionSlide 12 2008 PricewaterhouseCoopers.Summary of Recent Technical Model Validation Work Loan-level Agency and Non-Agency fixed rate and ARM mortgage loan prepayment models Implementation of a vendor model LoanPerformance RiskModel for use in the Banks FAS 5 allowance for loan loss processLarge Regional Bank FAS 5 model used to estimate allowance for loan and lease losses Models used to estimate used vehicle auction values and lease-end vehicle return rates Engine used to calculate VIN-level accumulated supplemental depreciation on auto leasesLeading Captive Auto Finance Company Private loan default and severity models used to estimate FAS 5 allowance for loan losses Models to estimate FFELP student loan consolidation rates and borrower benefit eligibility rates FAS 91 amortization engine ABS collateral cash flow model used to value ABS residualsLeading US Student Loan Originator Pool-level Agency and Non-Agency fixed rate and ARM mortgage loan prepayment models Loan-level Agency fixed rate and ARM mortgage prepayment models FAS 91 mortgage loan amortization engine Complex Monte-Carlo based discounted expected cash flow model used to value the entire SF residential mortgage portfolio Weighted repeat sales index house price model Single-family and multi-family FAS 5 allowance for loan loss models Implementation of several vendor models including S&P LEVELS, Moodys M3, and LoanPerformance RiskModelConforming Mortgage Market Leader Model Validation ExperienceClientSlide 13 2008 PricewaterhouseCoopers.Recent Model Governance SolutionsFDIC examiners were complimentary of the Banks new Model Governance policy and procedures, and the company has started using the new Model Governance organization as a platform for disseminating best practices in model development, documentation, and testing throughout the enterprise.Value ProvidedPwC identified 15 key model governance elements such as the definition of a “model”, the role of senior management in model governance, the role of internal audit, etc. and summarized both known regulatory guidance in each of these areas as well as industry leading practices (based on our experience with peer institutions). We then compared the clients model governance program against these benchmarks and offered targeted recommendations to remediate significant gaps. PwC then assisted the client with a significant re-design and implementation of an enhanced Model Governance program prior to the FDIC exam which included: Designing and writing a new Model Governance Policy Designing a new Model Governance process, a new independent Model Validation Unit, and a new senior management Model Oversight Committee Designing and writing the independent validation procedures and standards for the Model Validation Unit Assisting individual business units with the development of detailed model risk management procedures Creating and delivering Model Governance training materials for company personnelPwC SolutionThe Client was notified of an upcoming FDIC exam targeting the Banks model validation and model governance processes. PwC was engaged to perform a pre-exam review with a focus on identifying gaps in the Banks existing model governance program relative to regulatory expectations and industry practices and offering recommendations for improvement. Client Issue Major U.S. Credit Card IssuerSlide 14 2008 PricewaterhouseCoopers.PwCs work led to a significant improvement in the quality of the Banks Model Governance Program and the program received positive feedback from the Banks OCC examiners.Value ProvidedPwC reviewed the design of the Banks existing Model Governance Program and interviewed numerous keystakeholders in order to (1) leverage as much of the existing model governance processes as possible, and (2) tailor our recommendations for improvement to the preferences of key personnel. Our key deliverables on this engagement were as follows: Providing a set of options associated with various features of a new Model Governance process including: Organization reporting structure, senior management oversight structure, staffing options, etc., Providing advantages and disadvantages for each option, and Offering our recommendations to the Bank. Designing and writing a new Model Governance Policy Designing and writing the independent validation procedures and standards for the Model Validation Unit Creating job descriptions for key positions within the Model Validation Unit Creating estimates of program cost under alternative staffing modelsPwC SolutionAs part of its Basel II initiative, the Bank engaged PwC to assist it with a complete overhaul of its existing Model Governance program in order to improve its consistency with regulatory expectations under Basel II and OCC 2000-16.Client Issue Large U.S. Regional BankRecent Model Governance SolutionsSlide 15 2008 PricewaterhouseCoopers.The Client made significant improvements to their model risk management policies and procedures, implemented a robust model change management process including a robust independent model validation approach.Value ProvidedAs part of PwCs integrated audit, PwCs model risk specialists performed recurring reviews of the Companys model validation policy and procedures including detailed reviews of the model validation work performed by the Companys independent model validation unit. PwC provided a number of recommendations for strengthening the model governance program, and provided significant assistance to the Company in the area of controls around analytical models and systems.PwC SolutionA PwC audit client had a Significant Deficiency in the areas of model validation, model operational controls, and model change management controls.Client IssueConforming Mortgage Market LeaderRecent Model Governance SolutionsSlide 16 2008 PricewaterhouseCoopers.The Client was able to identify and correct numerous deficiencies in the models, improve managements understanding of the model limitations, and enhance managements overall awareness of how to interpret the model results. Additionally, management was able to demonstrate that they had established the foundation for an effective model risk management program. Value AddedPwC reviewed the affected model, as well as other models used in the process. As a result, a number of additional issues were discovered in various models. PwC then assisted the client in defining a comprehensive model validation program and procedures. The client mobilized various teams to enhance model development/monitoring documentation related to all material credit risk models, and to independently validate the models. PwC performed additional reviews of the model documentation and validation reports, providing specific recommendations for closing gaps in the documentation and expanding the testing where appropriate. PwC SolutionDuring the quarterly execution of its loss forecasting process, our Client discovered an error in one of its models. The error was significant enough to call into question prior period financial reporting. Client IssueLeading Regional BankRecent Model Governance SolutionsSlide 17 2008 PricewaterhouseCoopers.After the model error was corrected by the vendor, the client noted that the mortgage servicing asset risk profile appeared more logical, and hedge effectiveness was improved. The client continues to use the mortgage servicing rights value change attribution analysis methodology developed by the PwC team on a daily basis and the model validation work plan template on an annual basis for re-validation purposes.Value AddedThe Client used vendor models to develop mortgage prepayment assumptions and value mortgage servicing rights. The PwC team leveraged its proprietary model validation work plan template to create a set of model validation procedures for the vendor models. PwC also identified an error in the methodology used by the vendor to create a forward curve for projecting mortgage rates used by the prepayment model. The PwC team showed the client how the model error was contributing to mortgage servicing rights hedge ineffectiveness by demonstrating the impact of the error with mortgage servicing rights value change attribution analysis. PwC SolutionOur Client required assistance with the development of a model validation program for mortgage servicing rights valuation and risk management models to address regulatory safety and soundness examination findings. Client IssueA Top-Ten Mortgage and Home Equity LenderRecent Model Governance SolutionsSlide 18 2008 PricewaterhouseCoopers.PwCs work led to a better model risk management governance approach including, specifically, clarity around the role of model owners and independent model validation, as well as enhancements to the policies and procedures.Value ProvidedPwC benchmarked the banks model risk management policies, procedures, and guidelines to applicable regulatory guidance and our knowledge of regulatory expectations. We also reviewed the clients model documentation and model validation documentation for a sample of consumer and commercial PD and LGD models. Based on our benchmarking analysis, we identified a number of issues related to: The overall design of the clients model risk management program The programs policies and procedures Validation tests performed on, and interpretation of results for, certain models.We summarized these observations in a report of findings for the client, and participated in working sessions with the head of the model validation unit and senior management to discuss the findings and alternatives for enhancing the program. PwC SolutionAs part of its Basel II initiative, the Bank engaged PwC to assist it with an evaluation of its model risk management program relative to Basel II and regulatory guidelines / expectations.Client Issue Large International BankRecent Model Governance SolutionsPwC Model Risk Management Key ContactsRichard R. Pace PrincipalMcLean, VA(703) 918-1385Ric.PRic is a co-leader of the firms model risk management practice providing both strategic advice to our clients on the design of model risk management programs, as well as assisting our clients with model validations in the areas of credit scoring and automated underwriting, risk-based pricing, loan loss forecasting and reserving, economic capital, mortgage prepayment forecasting, asset valuation modeling, SOP 03-3, and SFAS 91 amortization.In addition to deep model risk management and control experience, Ric and his team have extensive technical expertise in the fields of statistical modeling, SAS programming, large scale data processing, and model auditing. Additionally, his team possesses a broad and deep range of industry knowledge including substantial mortgage banking and consumer lending experience, particularly in the areas of conventional, nonconforming, and subprime mortgages, auto loans, credit cards, student loans, and other consumer finance products. Prior to joining PricewaterhouseCoopers, Ric was an Assistant Professor of Economics and taught both undergraduate and graduate courses in managerial economics, finance, and banking. Ric has a PhD in Economics from the University of Rochester and a BS in Finance and Economics from the State University of New York College at Oswego.PwC Model Risk Management Key ContactsDoug Summa PartnerNew York, NY(646) 471-8596Doug.SDoug is a co-leader of PwCs model risk management practice and specializes in addressing a variety of valuation and risk measurement issues for a broad range of fixed income, foreign exchange, equity, credit and commodity products. He has over 20 years of experience advising clients on a spectrum of financial and risk management issues and has assisted numerous global financial institutions with the development of risk management frameworks and in the successful implementation and execution of a standard industry-practices approach to risk management and measurement. Dougs clients include investment banks, global and super-regional commercial banks, insurance companies, asset managers, government sponsored enterprises, mortgage companies and energy merchant trading firms.Doug has an MBA in Finance and Accounting from Columbia University.PwC Model Risk Management Key ContactsSteve Robertson Managing DirectorMinneapolis, MN(612) 596-4438Steve.RSteve provides specialized audit support and advisory services for model-based critical accounting estimates such as valuation of mortgage servicing rights, loans held for sale, and securitization residuals as well as loan loss allowances, repurchase loss reserves, and credit card membership program cost reserves. Steve has over 15 years of professional experience designing, developing, and validating statistically-based and computationally-based economic and financial models. Steve is recognized as one of PwC s