The Imperative of a Holistic MLOps Framework for Financial Institutions

The digitization of the financial services industry, accentuated by artificial intelligence (AI), has ushered in an era of dramatic transformation. The surge in the scale, diversity, and complexity of models demands reimagining organizational choices, addressing model growth, scaling, and risk management of these models. This article discusses the challenges faced by Financial Institutions and the opportunity to address them leveraging MLOps platforms to transform Data Science Operating Models for both Financial and AI models.

SUBSTANTIAL GROWTH AHEAD

According to the McKinsey State of AI in 2022 report, a real gap can now be observed between 8% early adopting corporations that see massive AI benefits with 20% bottom-line impact, and the large mass of “laggards” that see more hurdles, including the why? to the investment.

However, the dazzling advent and penetration of the latest technology – Generative AI Large Language Models – could be a turning point, as a self-fulfilling prophecy, or virtuous circle as one would prefer seeing it.

Early adopters, among which major corporations are already making significant investments in using LLMs in their daily processes. That comes as no surprise, as most of us, as individuals, are already impacted through the major search engines, and given very easy tools to become Citizen developers. Individuals will naturally be looking for similar capabilities in a professional set-up.

Corporations that are in a wait & see mode, only tipping their toes in the cold water, may experience dire and overwhelming moments, in waiting further to seriously take the AI turn, let alone the Generative AI one.

Another important angle to look at is talent: this growth will partially be carried out by the impressive growing number of Data Scientists – the United States Bureau of Labor Statistics plans a 36% annual growth until 2031 for that profession vs 0.5% on average for all professions.

WHAT CONSTRAINTS?

Significant Model Risk Management transformation

Traditionally, model risk management (MRM) was focusing on financial and risk models only, with the US regulators setting the tone for what model risk management entailed. However, over the last 2 years US regulators, followed by UK regulators have also put an emphasis on the integration of AI models into the Model Risk Framework (1).

The MRM transformations needs to address both the nature of the AI Models, as well as processes linked to their permanent evolution and their explosive growth.The nature of AI models: (i) explainability and interpretability of results avoiding black boxes, (ii) incorporation of ethical and sustainability considerations, with the former making the headlines across the industry, the legislators and world leaders (2).

The processes linked to AI models MRM: (i) continuous model validation processes due to the very rapid evolution of AI models vs. point-in-time processes for financial models, (ii) the handling of private data by model developers requires adherence to stringent data privacy regulations (iii) the adaptation to the significant growth of AI Models.

Legacy IT Inefficiencies and Scaling Costs

The dream of a unique architecture often clashes with the reality of regulatory timelines, modeling needs, and IT implementation complexity.

The reliance on distinct modeling tools creates data redundancy and compliance complications, often leading to costly regulatory remediation.

Cloud computing, while enabling previously unthinkable analyses, incurs substantial costs, necessitating efficient resource allocation and dynamic management.

Fragmented Operating Models

The management of data science with its ubiquitous nature is a complex task.

The increasing fragmentation across multiple teams like IT, Data, Business, and Risk & Compliance can result in inefficiency and lack of ownership.

Furthermore, integrating new teams focusing on AI adds to the complexity. The sharp increase in models and use cases significantly exacerbates existing problems.

WHAT SOLUTIONS?

[1] SR 11-07, SR 15-18 and SR 15-19 further modified by the Tailoring rules in 2021. In addition, OCC introduced new guidance on Model Risk Management including AI models through its Comptroller’s handbook. In the UK, the Bank of England published the 6/22 Consultative Paper on Model Risk Management that also covers AI models.

[2] G7 in Hiroshima agreed on 5/20/23 to “advance international discussions on inclusive artificial intelligence (AI) governance and interoperability to achieve our common vision and goal of trustworthy AI”, while the US Senate hearing saw Open AI CEO calling for AI regulation on 5/16/2023