Meredith Piotti of Wolf & Company, P.C., explains why model verification is a critical component of cannabis banking risk management.
Meredith “Merry” Piotti works with clients to provide full internal audit services, supplement internal audit capabilities, assist with specialized reviews such as model validations, and develop risk-based audit plans. Shield spoke with Merry about model verification best practices for financial institutions serving cannabis banking customers.
What is model validation?
Merry Piotti (MP): Model validation is a review designed to identify if the information and systems financial institutions rely on to make strategic business decisions are providing an accurate and complete picture. It includes three primary components:
1.) Is all the data that should be incorporated being provided to the model completely and accurately?
2.) Are the calculations/logic used within the model based on current expertise on the subject?
3.) Are the calculations/logic within the model functioning as designed and producing accurate and necessary results?
Why is model validation important for financial institutions providing cannabis banking services?
MP: The banking platforms used to address FinCEN guidelines and evolving regulatory expectations would be considered models. They are designed to alert management to potential risks within cannabis-related businesses (CRBs) and their transactions. Given the higher level of scrutiny over the cannabis industry, it is even more important that models related to these services produce complete and accurate results.
How does validation differ based on the type of model used?
MP: Models are either rules-based (like Shield Compliance) or behavior-based. A rules-based system will identify a criterion or set of criteria to generate alerts. A very simple example of this would be to identify all consumer customers with three or more wire transactions totaling over $50,000. Behavior-based models typically incorporate some type of additional layer of analytics into the parameters. For example, it may start with a rule saying it wants to identify cumulative wires over $50,000 but only if that activity is 20% higher than the historical transactions for that customer. When a model utilizes such an algorithm without disclosing the underlying logic, this is referred to as a black box. When models incorporate algorithms that are not fully disclosed to the user, it makes it even more complicated to validate. This often requires specialized data analytic software.
When should a model verification be performed?
MP: Some of the metrics that should be incorporated in the risk assessment to determine model validation frequency include the types of transactions, the volume of transactions, the complexity of the model, and the level of regulatory scrutiny. A model should receive a new validation whenever there are significant changes to the system(s) supplying data or the logic within the model. It is important for financial institutions to understand the expectations of their examiners as it pertains to validating models based on the asset size of the institution or the size of the cannabis portfolio.
What is the value/benefit of using a 3rd party for model validation?
MP: Depending on the complexity of the model and the skills within the financial institution, model validation can be performed internally or by a third party. The more complex or critical the model is, the more likely a third party may be required to review the model to ensure the three components listed above are properly evaluated.
It is important that the individual performing the validation not only has the expertise to validate the model but is also independent of the process. This will ensure that all components of the model are evaluated and issues are properly identified which is why institutions often rely on third parties to perform model validations. Another reason an institution will find benefit in outsourcing model validation is that third-party specialists will have more insight into how other institutions are utilizing models including current regulatory hot topics, industry-leading practices regarding the functionality of the models, and model functionality that may be more susceptible to errors.
For financial institutions serving the cannabis industry, model verification is a significant part of BSA/AML compliance. Shield’s rules-based compliance management tools help bankers gain greater insights into customer accounts and mitigate against risk.