Artificial Intelligence (AI) systems are fundamentally transforming industries and are becoming increasingly integral to decision-making processes in fields ranging from healthcare and finance to transportation and security. They are used for multiple tasks and are not without risks. Ensuring these systems function correctly and comply with relevant regulations is important. AI system assessment is the systematic process of confirming that an AI system performs as intended, is reliable, fair, and meets all regulatory requirements. This white paper explores the key steps and considerations involved in AI system assessments.
We look at this topic from the perspective of the insurance industry in the Netherlands and Belgium. The regulations of those markets are considered leading when it comes to governing the use of valuation and risk models. Processes similar to existing model validation processes can be applied to the development and use of AI systems. In this paper, we provide more insight around our assessment approach and how it can be used to meet requirements from financial supervisors and prepare for compliance with industry standards related to AI.
Discussion points include the following.
- Risks associated with AI systems: Legal, reputational, strategic, financial, operational, technological, environmental, and model risks.
- Compliance with AI regulation, frameworks, and guidance: This report's reliance on supervisory expectations and general principles set out in the European regulators and industry organisations such as the National Institute of Standards and Technology and National Association of Insurance Commissioners.
- Risk categorization in the European AI Act: Unacceptable risk, high risk, limited risk, and minimal/no risk.
- Risk classification of AI systems: Likelihood factors and severity factors.
- AI system categorization: Relevant data categories, model functions, and technical approaches.
- AI system assessment approach: Business objectives, data, model, use, and periodic reassessment.
- Assessment criteria for data, model, and use: Examples of qualitative criteria into thresholds and measurable criteria.
- AI system assessment policy: Clear definitions of assessment objectives and scope, assignments of roles and responsibilities, and establishment of standardised procedures.
- Assessment process overview: Some aspects that will remain challenging.