Conversations That Matter
No deviation’s Takeaways from 2025 ISPE France Conference – Validation of AI in the pharmaceutical industry.
Presentation of a validation strategy for AI systems (hybrid and generative) in GxP environments, aligned with GAMP 5 2nd edition.
Speaker : Filipa Silva – Capgemini Engineering
Artificial Intelligence (AI) and Generative AI (Gen AI) are transforming many industries, including pharmaceuticals. Philip Silva, based in Portugal, presented how the industry can move AI validation from risk management to practical reality while ensuring compliance and, above all, patient safety.
Fundamentals: GxP, Risk, and Human Supervision
In pharmaceuticals, the starting point always remains the GxP (Good Practices) impact. It is essential to clearly identify which parts of a process you intend to entrust to AI and to determine the actual GxP impact of that specific step.
Every AI implementation must undergo a thorough risk–benefit analysis. While AI offers tremendous advantages, it also introduces additional risks such as bias, drift, extrapolation, and hallucinations. These risks must be clearly identified and mitigation strategies defined.
A key principle is that human-in-the-loop oversight is an absolute necessity. A human must decide when to proceed, and the system and process owner remains the person accountable to regulators. If AI is used to draft a report, for example, a human must still critically review and validate it before moving on to the next step.
The Critical Role of Data and Testing
The success or failure of any AI solution largely depends on the data used. Data selection is critical and requires:
Quality – Accurate and relevant data for the intended use case.
Diversity – Including both successful and failed use cases to help the AI understand what works and what does not.
Volume – A sufficiently large dataset to achieve reliable outcomes.
To ensure validation and avoid overfitting, it is essential to use separate datasets for training and testing. Including real rare and extreme examples is also recommended.
Moreover, especially in the pharmaceutical industry, data protection is paramount. Strict access controls should be defined and anonymization methods applied when necessary.
Transparency and Containerization Requirements
Any AI or Gen AI functionality integrated into pharmaceutical activities must include an explainability feature. Every decision or output provided by AI must be explainable, giving humans a clear basis to understand how the decision was made.
To ensure reproducibility and cybersecurity compliance, AI models must be static and containerized. Once a model version is selected, it should be packaged to guarantee that future updates will not affect the performance of the validated system. Capcha Mini recommends using local, containerized Large Language Models (LLMs) within the client’s environment so that GxP information never leaves a controlled space.
Adapting Traditional Validation (IQ/OQ/PQ)
The traditional validation approach must be adjusted for AI by combining classic concepts with new perspectives.
| Validation Step | Additional Requirements for AI |
| Installation Qualification (IQ) | Qualification of the AI infrastructure (cloud or on-premises), hardware (e.g., GPU), and the selected AI model. |
| Operational Qualification (OQ) | In-depth testing of the AI model’s interfaces with other platforms and systems, data pipelines, and confirmation of result reproducibility. |
| Performance Qualification (PQ) | Detailed and ongoing evaluation of the AI model’s performance in the specific context, continuous accuracy monitoring, re-assessment, and change control. |
Change control and versioning are essential. Model updates must not directly impact the validated solution and should be pre-tested. Likewise, retraining triggers should be defined during validation to anticipate performance drops or the release of new model versions.
The Challenge of Annex 22 and Deterministic AI
Annex 22, a new regulatory guideline, provides clarifications but also brings challenges. The current text strongly discourages the use of predictive, generative, and probabilistic AI for use cases directly critical to GxP.
Gen AI is still considered highly valuable for non-GxP-critical processes—those with no direct impact on the product or patient safety (for example, documentation support and information retrieval).
For activities with a direct impact on GxP/GMP, the focus is on more deterministic and static models that deliver a high level of certainty rather than probabilistic responses. The term deterministic is debated, since AI is often probabilistic by nature. It raises questions about what the authors of Annex 22 truly mean—perhaps distinguishing between classic AI (which provides binary or labeled outputs without altering the underlying information) and Gen AI (which can extrapolate and generate new content).
Successfully adopting AI in the pharmaceutical industry requires a hybrid approach, combining deep knowledge of compliance and business context with the new demands of AI validation.
