On Wednesday, the International Medical Device Regulators Forum (IMDRF) finalized two highly anticipated technical documents outlining guiding principles for good machine learning practices (GMLP) and risk characterization in medical device software. These documents are designed to enhance global harmonization and assist medical device manufacturers in meeting compliance requirements for software-based and AI-driven technologies.
Good machine learning practice for medical device development
The Good Machine Learning Practice (GMLP) principles outlined in the IMDRF document are largely consistent, with only slight wording adjustments, to those jointly identified by the U.S. Food and Drug Administration (FDA), Health Canada, and the Medicines and Healthcare products Regulatory Agency (MHRA) in the UK. A notable distinction lies in the broader scope of collaboration and the target audience, making it particularly relevant for manufacturers worldwide.
The guidelines aim to ensure the creation of safer, more effective, and high-quality AI/ML-enabled medical devices.
Here’s a summary of the 10 principles of GMLP software:
- The device’s intended use and purpose are clearly understood, utilizing multi-disciplinary expertise throughout the total product life cycle.
- Robust software engineering, medical device design, and security practices are applied consistently throughout the total product life cycle.
- Clinical evaluations utilize datasets that accurately represent the target patient population.
- Training datasets are independent of test sets
- Chosen reference standards are fit-for-purpose
- Model choice and design are customized based on the available data and the device’s intended use.
- The device is evaluated with an emphasis on human-AI interactions in the intended use environment, focusing on the performance of the human-AI team rather than the device alone.
- Testing verifies the device’s performance under clinically relevant conditions.
- Users receive clear and essential information.
- Deployed models are continuously monitored for performance, and re-training risks are effectively managed.
The final version includes only slight modifications from the draft document released in July 2024. A key update is that manufacturers should ensure their software can reproduce results while discussing the importance of good software engineering, medical device design, and security practices throughout the product’s lifecycle.
The release of the N88 document represents a significant advancement in aligning ML principles for medical devices. Manufacturers of ML-enabled devices that incorporate Good Machine Learning Practices (GMLP) into their product development will likely experience better compliance, fewer regulatory challenges, increased product reliability, and greater customer trust.
Software risk characterization
IMDRF has released its final document regarding considerations for device and risk characterization specifically for medical device software, which includes several updates from the draft version published in January 2024. This new N81 document builds upon the risk categorization framework outlined in IMDRF/SaMD WG/N12 FINAL:2014, adapting to the varied landscape of software and its different interpretations across regions.
One notable update in the final document is the emphasis on the fact that software can evolve during a product’s lifecycle, highlighting the need to reassess risk characterization as software updates occur.
IMDRF also emphasizes that the N81 document is meant to enhance, rather than replace, the N12 framework. This document offers important insights on how to properly characterize MDSW and evaluate associated software-specific risks.
