Artificial Intelligence in Life Sciences: Reliability of AI-Generated Evidence in Drug Discovery and Development

The life sciences sector is uniquely vulnerable to the inherent risks of AI-generated evidence in drug development and discovery. This client alert focuses on five core challenges: hallucinations, skewed predictions from unrepresentative training data, the opacity of AI reasoning, input sensitivity that undermines reproducibility, and adversarial manipulation of training data. To address these risks, companies should use experimental validation and human oversight as the common legal and regulatory safety net. This entails among other measures auditing AI outputs, characterizing training data, and structural oversight. Companies also should prepare for evolving frameworks and guidance from the U.S. Food and Drug Administration (FDA) and the European Union (EU) that address use of AI in drug discovery and development .  

Background 

The use of AI in drug discovery and development generates a category of legal risk distinct from ordinary scientific uncertainty: AI/ML systems can err in ways that are both difficult to detect and difficult to attribute after the fact. These features create legal exposure across multiple domains. A regulatory submission that relies on AI-generated data without independent human validation may not meet existing FDA data integrity standards; a clinical trial dataset whose outputs were distorted by adversarial manipulation of the AI tool’s training data may expose the sponsor to product liability and regulatory fraud risk; and AI-generated expert analysis offered in litigation may be vulnerable to challenge on reproducibility grounds under Federal Rule of Evidence 702.  

Across regulatory frameworks, international guidelines, and emerging liability regimes, the legal and regulatory backstop for AI-generated evidence is human review by a qualified person who can independently assess the accuracy and completeness of AI outputs. 

1. Hallucinations: False Data and Fabricated References 

What hallucinations are: An AI/ML model generates outputs by predicting the most statistically likely response based upon its training data and inputs provided. Thus, a model can produce an output that is misleading or completely fabricated yet presented with total confidence. This phenomenon is commonly referred to as hallucination.  

The drug development context: In drug discovery and development, an AI/ML model may generate a fabricated biochemical reference that appears authoritative, a skewed efficacy prediction that reflects the limits of its training data rather than the biology of the molecule, or a result that may not be replicated when the same inputs are run again. This occurs because of inherent structural features of how the model processes data, not experimental errors. 

Legal exposure: The legal consequences of undetected hallucinations vary by context. In regulatory submissions, AI-generated false data that is submitted to the FDA without independent verification may constitute a material misrepresentation under 21 U.S.C. § 331, irrespective of intent. In patent prosecution, fabricated prior art references submitted to the USPTO without verification abrogates the duty of candor under 37 C.F.R. § 1.56 — and a finding of inequitable conduct can render an issued patent unenforceable.  

2. Skewed Predictions: Incomplete and Unrepresentative Training Data 

The structural problem: An AI model’s predictions are bounded by its training data. A model trained on an incomplete historical dataset that overrepresents certain molecular classes, patient populations, disease presentations, or experimental conditions will generate predictions that systematically reflect those gaps. The skew is often invisible from the output alone: a model can produce highly confident predictions that are precisely wrong in ways that trace entirely to what it was and was not trained on. 

The regulatory consequence: FDA submissions that rely on AI-generated predictions derived from training data of unknown or uncharacterized provenance create data integrity risk. The FDA’s existing GCP and GMP frameworks require that data submitted in support of regulatory applications be accurate, complete, and traceable. The FDA-EMA Joint Guiding Principles of Good AI Practice in Drug Development (January 14, 2026), establish a unified framework for the entire product lifecycle. Through Principle 6 (Data Governance) and Principle 7 (Model Design), establishing that rigorous data provenance, processing traceability, and fit-for-use data are mandatory prerequisites to ensuring the generalizability and robustness of AI-driven evidence. ICH M15 — the final, harmonized international guideline on model-informed drug development adopted January 29, 2026 — provides a complementary framework, expressly covering AI/ML models used in drug development and establishing international expectations for model verification and validation that include characterization of training data limitations. 

3. The Black Box Problem: Traceability and Explainability 

The traceability gap: Unlike conventional statistical models, many AI/ML models do not generate step-by-step, traceable outputs. Instead, inputs pass through complex layers of learned parameters to produce a result without human-readable intermediate steps. This is the “black box” problem: the output may be correct, but the path from input to output lacks explainability. 

The regulatory dimension: The FDA’s existing expectation of traceable, auditable data — embedded in 21 C.F.R. Part 11, in GCP and GMP frameworks, and in the agency’s longstanding data integrity guidance — creates structural tension with AI opacity. ICH M15 addresses this directly, establishing harmonized international expectations for model transparency and documentation that include the obligation to characterize and disclose the limits of a model's explainability. The FDA-EMA Joint Guiding Principles similarly identify traceability as a core requirement for AI used in drug development, across the full product lifecycle. 

The product liability dimension: The revised EU Product Liability Directive (Directive (EU) 2024/2853) expressly links disclosure and explainability to liability. If a manufacturer does not disclose or cannot explain how an AI system reached a particular output, the PLD creates a presumption that the product was defective. Companies must comply ahead of the December 9, 2026 transposition deadline, when the PLD becomes law in EU countries.

4. Input Sensitivity and Reproducibility 

The reproducibility problem: AI/ML models can exhibit high sensitivity to small changes in input data, sometimes described as “brittleness.” A model that produces a specific output when given a particular input may produce a materially different output when such input is provided in a slightly modified format or version, not because of experimental variation but because of how the model’s learned parameters interact with the input at the margin. This creates a reproducibility problem. The experiment may have been run perfectly, but that the model’s outputs are structurally unstable at the input boundary. 

The regulatory consequence: FDA and ICH frameworks expect that results submitted in regulatory applications be reproducible; that is, that the same data, processed through the same methodology, should yield the same results. Where an AI model’s outputs cannot be reproduced from substantively identical inputs, the model may not meet these standards. 

The litigation consequence: Under Federal Rule of Evidence 702 and the Daubert standard, expert testimony, including AI-generated expert analyses, must be based on sufficient facts or data and must be the product of reliable principles and methods reliably applied. An AI-generated analysis that cannot be reproduced from the same inputs is vulnerable to a Daubert challenge on reliability grounds: If the method does not reliably produce the same result from the same data, it does not satisfy the reproducibility component of the reliability inquiry.  

Courts are already beginning to grapple with these issues. In In re Celsius Network LLC, a bankruptcy court excluded an expert valuation report under Rule 702 where the AI-assisted analysis could not be adequately explained or validated by the sponsoring expert. The Advisory Committee on Evidence Rules has proposed Federal Rule of Evidence 707, which would establish explicit admissibility standards for machine-generated evidence, addressing directly the risk that AI outputs are offered as standalone evidence to circumvent Rule 702 gatekeeping. As of mid-2026, the Advisory Committee has paused to consider significant revisions to the proposal, but the underlying concern driving it, that AI-generated outputs require reliability standards that existing rules were not designed to provide, is already shaping how courts evaluate expert testimony that relies on AI tools. 

5. Adversarial Manipulation of Training Data 

What adversarial manipulation means: Adversarial manipulation of AI training data, sometimes called a “data poisoning” attack, involves the intentional introduction of corrupted, falsified, or distorted data into an AI model’s training dataset, with the goal of causing the model to produce systematically skewed or false outputs. In the drug development context, the concern is that malicious actors could exploit security vulnerabilities in an AI tool or its data pipeline to distort the model’s predictions about molecular properties, efficacy signals, toxicity profiles, or clinical trial results. 

Why this is a distinct legal risk: Adversarial manipulation is legally distinct from ordinary data security failure and is a hidden minefield for life sciences companies. First, the manipulation may be undetectable at the output level: A poisoned model can pass standard validation checks while being systematically skewed in ways that only become apparent after regulatory approval and diverse event accumulation. Second, attribution is difficult: Tracking a failure to malicious tampering rather than model error, data quality failure, or experimental noise may require forensic audit of the training data pipeline that sponsors rarely control. Third, a single event may generate a chain reaction, concurrently exposing a company to product liability, regulatory fraud, and cybersecurity compliance breaches, all from a single manipulation event. 

Applicable frameworks: The EU AI Act (Regulation (EU) 2024/1689), which classifies certain AI systems used in medical and clinical contexts as high-risk, imposes cybersecurity and data governance obligations on providers of high-risk AI systems that directly address adversarial manipulation risk. The FDA-EMA Joint Guiding Principles similarly address security and integrity of AI systems across the development lifecycle. 

6. The Human Validation Obligation: Who Is Responsible and What Does It Require 

The common thread: Across all sub-risks addressed in this alert, the legal and regulatory safety net is rigorous human validation, meaning that a qualified expert must independently verify the accuracy and completeness of AI outputs before any consequential use. This is the application of existing data integrity, scientific validity, and professional responsibility standards to a nascent and highly unpredictable new class of tool. 

Role of Humans: The CIOMS Working Group XIV on Artificial Intelligence in Pharmacovigilance articulates two key operationally distinct models for inserting human oversight: human-in-the-loop (HITL) and human-on-the-loop (HOTL). In a HITL model, a human reviewer is integrated into the AI workflow at each decision point — the AI generates a candidate output, and a human approves, modifies, or rejects it before it is acted upon. In a HOTL model, the AI operates autonomously within defined parameters, with human oversight applied at the system level rather than the output level. Humans monitor aggregate performance and intervene when the system exceeds defined thresholds, but do not review individual outputs in real time.  

What the regulatory frameworks require: The FDA-EMA Joint Guiding Principles identify human oversight as one of their ten core principles for good AI practice in drug development, applicable across the full product lifecycle. ICH M15 establishes documentation and validation requirements for model-informed drug development that include human expert review of model outputs before they are incorporated into regulatory submissions.  

Practical implications: Three key questions follow that companies should be prepared to answer for any AI tool in their drug development stack: 

• Who is the responsible human reviewer for each category of AI output generated by this tool? 

• What does independent validation require in practice? Does it require re-running the analysis by hand, checking outputs against independent sources, or something else? 

• Who bears responsibility between the sponsor and the AI vendor if a validation failure occurs, and does the contract between them reflect that allocation? 

7. What We Are Watching 

• FDA finalization of guidance specifically addressing AI data integrity and human validation obligations for drug and biologic submissions — distinct from the device-focused August 2026 generative AI discussion paper. 

• EU PLD transposition by Member States ahead of the December 9, 2026, deadline — particularly whether any Member State transposes the disclosure and presumption-of-defect provisions more aggressively than the PLD's minimum floor. 

• EU AI Act enforcement actions bearing on high-risk AI systems used in clinical trial and drug development contexts. 

These developments will continue to be monitored in coordination with our EU colleagues, particularly as European regulatory and enforcement frameworks evolve. 

8. Practical Steps 

1. Audit AI outputs before submission. For every AI-generated output incorporated into a regulatory submission, patent prosecution filing, litigation document, or investor-facing material, establish a documented human review protocol that identifies the reviewer, the review methodology, and the outcome. 
2. Characterize your training data. Before relying on any AI model for a consequential regulatory or litigation purpose, document what the model was trained on, what known gaps or biases exist in the training data, and how those gaps may affect the model's predictions in your specific use case. 
3. Determine your HITL vs. HOTL model. For each AI tool in your drug development stack, determine whether it is operating on a HITL or HOTL basis, and document the governance and oversight structure that supports whichever model you are applying. 
4. Review AI vendor contracts. Ensure that contracts with AI tool vendors include data integrity representations, privacy and security standards, audit rights, and incident notification obligations. These terms should flow down to subcontractors. 
5. Monitor EU PLD transposition. If your products or submissions have EU exposure, confirm PLD transposition status in relevant Member States before December 9, 2026, and assess whether the transposed disclosure obligations may reach materials you consider privileged in U.S. proceedings. 
6. Do not rely on AI-generated references without verification. Any AI-generated literature citation, regulatory precedent, or case reference should be independently verified against the primary source before use in any submission, filing, or expert report.