In June 2022 the PRA published CP6/22, its consultation on Principles for Model Risk Management (MRM). The PRA’s core concern driving the publication of the principles is that senior executives and boards are not fully aware of the extent to which models drive management decisions in banks. The CP proposed five principles for MRM, along with a very broad definition of what constitutes a model for the purposes of the principles.

Figure 1 shows the PRA’s definition of a model from the Supervisory Statement, and the high-level description of the five principles for MRM, none of which is materially changed from the CP. A more complete assessment of the CP can be found in our previous blog.

The PRA received considerable feedback from industry in response to the CP and has reflected that in the Policy and Supervisory Statements. 

The most significant change in the Supervisory Statement is in the scope of application, as the policy will only initially apply to banks with internal model (IM) approvals for regulatory capital purposes when it comes into force in May 2024. Banks that are applying to become IM banks will have 12 months from the date of approval of their IM application to demonstrate compliance with the principles.

The PRA will update the industry as to how the principles will apply to non-IM banks once it has progressed its policy work on Simpler-regime firms, albeit the PRA notes that all firms, irrespective of size, are required to manage their model risks and that non-IM firms which are subject to existing supervisory expectations around models (such as self-assessments and attestations) should continue to comply with them.

Changes to the content of the policy predominantly see the PRA stepping back from the somewhat prescriptive nature of the CP and allowing banks greater scope to interpret some of the requirements with proportionality to their own business complexity and size. This was a common theme in the feedback the PRA received.

However, less regulatory prescription inevitably means reduced clarity as to what constitutes a compliant approach: industry and the PRA will likely have ongoing conversations on this point, particularly in respect of where the definition of a model stops, and whether banks’ assessments of a proportional implementation are sufficiently rigorous.

The content changes the PRA has made to its original proposals include:

• clarifying the responsibilities of the Senior Management Function (SMF) accountability and making clear that more than one SMF may be appointed;
  • the changes remove some of the language in the initial CP that respondents felt made the SMF for MRM responsible for both first- and second-line activities. They allow for the SMF to delegate some activities, while retaining accountability for the overall MRM framework;
  • the SMF holder(s) will have to provide initial and ongoing annual attestations of compliance with the principles. Such attestations require significant effort to prepare and sign off. Second- and third-line teams will need to play a material role in the process;
• replacing a reference to “accounting” with “financial reporting” to clarify that the intent is to ensure MRM reporting is available to the audit committee and not to mandate a change in audit committee responsibilities;
• reducing the prescriptive nature of some of the wording on model tiering, clarifying that firms can select their own relevant factors to determine model complexity;
• clarifying that subsidiaries using models developed by their parent or group may leverage the outcome of the group’s validation of the model so long as the principles on validation are satisfied;
• combining some clauses on expectations for models that recalibrate dynamically;
• clarifying the expectations of model documentation for vendor (external) models;
  • the PRA acknowledges that where banks use models provided by external vendors, the documentation the vendors provide will not disclose proprietary information. The PRA has clarified that it expects banks to ensure they receive sufficient information to enable them to validate their use of the external model;
• acknowledging that post model adjustments (PMAs) are an important risk management tool, and making changes to recognise the need for proportionality in PMAs; and
• making the principle around escalation processes less prescriptive and more principles-based.

The PRA noted in the Policy Statement that it received considerable feedback on the application of the principles to Artificial Intelligence and Machine Learning (AI/ML) models, in particular:

• the cross-functional nature of some AI/ML systems including data, models, and technology, which may mean a firm-wide approach to MRM with greater collaboration across relevant areas would be beneficial;
• AI/ML models can be highly complex, and so explaining how they produce outputs can be difficult. Firms could benefit from the PRA giving practical examples of the level of explanation expected;
• where AI/ML models are dynamic by design i.e. they change and/or recalibrate frequently, this may present additional challenges around ensuring adequate oversight and review;
• as AI/ML model complexity increases, monitoring of model performance becomes increasingly important (and challenging); and
• the use of AI/ML models can raise ethical challenges including fairness and bias – which could increase conduct and reputational risks. Better management and oversight of such risks may be needed in the MRM process where this risk is identified.

The PRA observed that there are similarities between the feedback received from CP6/22 on MRM and that received on DP5/22, the joint Bank of England/PRA/FCA discussion paper on the use of Artificial Intelligence and Machine Learning in financial services. The PRA notes that it will assess the feedback from DP5/22 as well as the responses to the 2022 Machine Learning Survey to inform any further policy actions.

In the meantime, as there is no specific reference to AI/ML in the supervisory statement, banks should proceed on the basis that the MRM principles will apply in full to AI/ML models that meet the PRA’s definition, absent further policy publication by the PRA.

Implications

Banks with existing IM permissions already have significant ongoing effort in their modelling teams, with work underway in several areas, including:

• implementing remaining model changes from the IRB roadmap;
• reviewing and revising IFRS 9 models;
• incorporating climate into modelling approaches for risk management and stress testing;
• preparing for the implementation of Basel 3.1; and
• assessing the model risk implications of the Consumer Duty.

Banks will have to address a number of challenges in meeting their obligations around MRM:

1. Designing and implementing a revised model governance process that retains existing capabilities and adds the capacity and expertise to oversee the significant expansion of models subject to oversight:
   • Ensuring that models which are not constructed similarly to “traditional” credit or market risk models, but which are still material – such as Anti-Money-Laundering models – are able to be appropriately challenged through model governance processes.
   • Additional training for modelling and validation teams, as well as for members of model governance fora, is likely to be necessary.
2. Designing a multi-tiered validation process that provides a review and validation regime that is fit-for-purpose for a significantly broader model population:
   • Not all validation efforts will require the same validation process or expertise as a “traditional” credit or market risk model – banks will need to think creatively about how to meet the requirement for oversight of different types and sizes of models.
   • There will be opportunities to apply technology to this process, particularly for models that are at the lowest levels of materiality and complexity.
3. Ensuring that appropriate reporting on the breadth of the model inventory is available and shared with executive management and the board (or a board sub-committee):
   • The reporting should focus on issues with the most material models and allow the executive and board to answer questions from supervisors about the implications of poor model performance for business decisions, and explain remedial or mitigating activities underway.
4. Identifying similarities and/or commonalities across the streams of model work underway:
   • Opportunities to undertake work that delivers common benefits across IRB repair, IFRS9 and Basel 3.1 will be relatively easy to identify. Identifying similar commonalities across Climate, MRM and Consumer Duty may be more challenging. Banks that are able to identify similarities and exploit them to reduce the workload for their scarce model resource will reap benefits.

The PRA sees model risk as a risk that should be treated in the same way as other material risks in banks: it should be part of risk appetite and should be monitored and managed as seriously as any other material risk. The PRA’s intent in putting the principles for MRM into the supervisory framework is to drive a change in banks’ culture around MRM.

The policy comes into force on May 17, 2024.

Co-Authors: Ian Wilson, Richard Tedder, Alexander Marianski, Justin Le Blanc, Yung Chong and Jyoti Makolski Deloitte.