Program & Model Validation


Model Validation for Actimize, Mantas, SAS, Detica-Norkom and other AML Transactions Monitoring Systems:

Validation is a structured  analytical process during which a documented evidence & assessment of the  effectiveness and accuracy of Data, Scenarios, Models and Rules, Population Segments, Thresholds and other Tunable Parameters is gathered, measured & documented.   Validation analyses and assesses currently used system settings, configurations, parameters  & key system components, such as:

  • Data (is the required data available?  How is the data sourced, normalized, transformed, mapped & ingested).

  • Watch Lists, Filters and Exclusions.

  • Scenarios, Models and Rules (are the Scenarios, Models and Rules aligned with AML Risk Assessment).

  • Segmentation / Population Groups and their adequacy and effectiveness.

  • If the system output (Alerts or Risk Events) correspond with the Scenario’s intended monitoring coverage (alert records are sampled).

  • Alert Productivity Rates are measured.

Vesta LLC - Tuning, Optimization & Independent Model Validation Services:  Our Approach    

  • Structured, Disciplined, Systematic & Scientific.

  • Sustainable & Repeatable.

  • Scalable & Right-Sized.

  • Experience and Expertise.

  • Objective and Independent.

  • Proven Risk-Based & Sustainable Programme Testing Methodology, Process and supporting automated Tools.

  • Validates Alignment and Conformance with AML Risk Assessment, Applicable Local, Regional & Group Level Rules, Regulations, Policy and Procedures.

  • Identifies & Documents Data, Watch Lists, Filters, Scenario, Model & Rule Monitoring Coverage, Thresholds & Tunable Parameters, Segmentation / Population Groups and Alert Risk Scoring GAPs, Issues, Concerns as well as  Control Weaknesses.

  • Supports OCC 2011-2012 Model Validation Framework & Sound Practices for Model Risk Management.

  • Supports NYDFS Rule 504 Programme Effectiveness Certification Requirements including Data Management and Data Controls.

  • Expertly Documented Current Programme State & Status, Controls, GAPs and Associated Risks,  Notes Continuous Improvement Roadmap, Recommended Changes and Remediation Steps.

Model Tuning & Optimization:

Tuning & Optimization is a process that enables the analysis &  testing of alternative settings for your AML System, Model, Scenario & Segment settings, configurations and tunable  that will improve Systems Performance and Scenario’s Monitoring Coverage. Each alternative setting is tested & measured, above & below the line testing is conducted, results are documented in the Tuning & Optimization Report.  Results and Findings are discussed with Stakeholders and the optimal segments or populations, setting, thresholds, configuration, parameters may be selected and implemented. Tuning & Optimization Analysis includes the following:

  • Data.

  • Segments / Population Groups.

  • Watch Lists & Filters.

  • Scenarios, Models & Rules and their tunable parameters (such as Scenario’s Look Back Period) and Thresholds.

  • Alert Productivity Rates.

  • Alert Risk Scoring Rules / Alert Risk Scoring Model.

There are numerous Threshold Tuning Methodologies and every Financial Institution needs to first understand its Risk Tolerance (aka “Risk Appetite), gather 12+ months of history (transactions, Alerts & Cases) and discuss its internal tuning and segmentation capabilities before the optimal Tuning Methodology can be defined and implemented.   In addition, as the Tuning and Optimization Function matures, Financial Institutions may need to also assess and change or enhance their Tuning Methodology. Also, its important to note that in addition to Tuning Methodology, every FI should also have a well defined Sampling Methodology and also ATL and BTL Testing protocols (ATL – Above the Line, BTL – Below the line.