Vendors
日本語
search
  2. //
  3. Reports & insights
  4. //
  5. Best Practices to Train an AI/ML Model

Best Practices to Train an AI/ML Model

Create a vendor selection project
Click to express your interest in this report
Indication of coverage against your requirements
A subscription is required to activate this feature. Contact us for more info.
Celent have reviewed this profile and believe it to be accurate.
We are waiting for the vendor to publish their solution profile. Contact us or request the RFX.
Projects allow you to export Registered Vendor details and survey responses for analysis outside of Marsh CND. Please refer to the Marsh CND User Guide for detailed instructions.
Download Registered Vendor Survey responses as PDF
Contact vendor directly with specific questions (ie. pricing, capacity, etc)
18 April 2023
Max Ang

What are the best practices to train an AI/ML model? In this blog, we will provide some insights on aligning business objectives with the development of an AI/ML model.

1) Consideration for data management, data security, and compliance policies: Data management should include considerations for security during production, because the experimental approach of AI/ML may fit badly with data compliance policies in the industry. Data management includes considerations for incoming data streams and datasets The process for data discovery, obtaining raw data, and processing requires access approval and tools. Therefore, insurers need to develop a strong data management standard before diving into the applications of AI/ML. And the ownership of data AI/ML initiatives must coincide with project sponsorship by the business unit. Data management also includes the development of data literacy in the organization, which imbues in the organization an appreciation of data which will translate into data innovations in production.

2) Defining the problem and business objective. Before beginning the model training process, it is important to clearly define the problem you to solve and the objectives to achieve. This includes identifying the type of data needed, the performance metrics you will use to evaluate the model, and the resources required to build and train the model. Select appropriate use cases such as underwriting and claims for AI/ML and MLOps initiatives.

3) Source, collect, and preprocess data. A good model is dependent on the quality and quantity of data used to train it. It is important to have a representative set of data to accurately reflect the problem to be solve. Preprocessing of data will be crucial to remove noise and inconsistencies that can negatively impact the model’s performance. Have request procedures to determine the right quantity and quality of data to be used, with transparency of data lineage and data anonymization in place. Keep data for future use but do not collect data you cannot use. Follow regulations on data usage and adjust the sensitive variables in the dataset to protect the consumer and prevent bias. The goal is to collect data that is useful and representative of the chosen use case.

4) Choose the appropriate algorithm and model architecture. Select the appropriate algorithm and model architecture based on the nature of the problem to solve, the size and complexity of the data, and the performance metrics to achieve. It is important to experiment with different algorithms and architectures to determine the best approach for the specific problem, with consideration for computing resources as well.

5) Train and validate the model. After selecting the appropriate algorithm and model architecture, model training can begin. It is important to split the data into training, validation, and testing sets, and to use techniques such as cross-validation and regularization to prevent overfitting. During training, it is important to monitor the model's performance and adjust the hyperparameters as needed.

6) Evaluate and fine-tune the model: After training, it is important to evaluate the model's performance on the testing set and adjust the model as needed. This includes tuning the hyperparameters, modifying the model architecture, or adjusting the data preprocessing pipeline. The model must be fit-for-purpose and to fulfil the business objectives.

7) Deploy and monitor the model: After the model is trained and validated, it can be deployed into production. Continuously monitor the model's performance and adjust as needed to ensure that it will perform well on new data.

8) Prepare model for scale. There can be a gap between the research environment/innovation labs and the operation unit responsible for the production pipeline This gulf between local innovation and scalability can be bridged on a centralized platform and workflow pipeline. The platform will handle the engineering aspect of model deployment, while allowing the data science team to focus on innovation that uses AI/ML techniques.

9) Data maintenance throughout the overall process: Data must be maintained for use across the preparation, training, and testing cycle. Data maintenance comes in the form of data/concept drift, which is the change in predictive variables as the model results change over time

10) Ensure explainable metrics and ethical usage. It is important to implement guardrails to ensure that the AI system is used ethically and responsibly. Insurers should develop policies and guidelines to govern the behavior of the AI/ML system and regularly audit its performance. Model explainability and governance policies will provide fair and unbiased model results. They also provide an avenue for the business and audit teams to understand what is under the hood, providing traceability and an audit trail. Explainable metrics can be visualized according to the feature contribution towards the model performance. Business users will understand the metrics (features) used in a model and collect appropriate data.

11) Managing AI Bias: Removing AI bias requires careful attention to the data used for training, as well as the algorithms and processes used to develop the model. To minimize bias in models, insurers need to consider data diversification, which will help provide a good representation of the data population and domain. Data can be evaluated regularly to monitor for bias and include analyzing the model’s outputs for discriminatory patterns. It is also good to choose models which are designed to minimize bias. This includes techniques like debiasing, fairness constraints, and adversarial training, and these techniques typically involve the human developers first and then the AI/ML to learn. And the involvement of the business users is important to ensure that the model development is ethical and transparent. Communication on the model’s limitations and potential bias to stakeholders and end users will help build trust that the model is used in a responsible manner.

These are some suggested best practices to train an AI/ML model that is accurate, reliable, and effective at solving the business objectives or problems. The key consideration for managing data well will also contribute to successful use case application for natural language processing, computer vision, and predictive decision making.

_____________________________________________________________

To learn more, Celent tracks this market and has research addressing it .If you would like to find out more, please feel free to get in touch with me.

Below are related reports contributed by Celent on this topic:

Business Data Strategy: Underwriting and Actuary Case Studies

The Data Force: Cultivating a Data-ready Organization

Data Management for Insurance: Best Practice and Solutions

Data Innovation and Management for the Future of Insurance

Data, MLOps, and IoT for the Next-Generation Insurance Industry

MLOps Part 2: Examples of Enterprise Machine Learning Deployment Providers

MLOps Part 1: From Machine Learning Innovation to Production

Introduction to Graph Data Design: Alternative Database and Tools

Securing Insurance Data: Confidential Computing and Data Lineage Use Case