When the Algorithm Misses

In March 2024, a case was filed in the Northern District of California that most people in medicine and technology should have read carefully and largely didn’t. The plaintiff — identified in initial filings as Jane Doe, a 47-year-old woman from Sacramento — alleged that an AI-assisted radiology tool failed to flag a lung nodule that a radiologist then also missed, and that the combined failure delayed her lung cancer diagnosis by fourteen months. By the time the diagnosis was made, stage I disease had progressed to stage IIIA.

The defendants were three parties: the radiologist, the hospital system, and the AI vendor. The case was settled out of court in late 2025 for a figure that was not disclosed, which means the legal questions it raised — who bears liability when AI and a physician both fail — were resolved with money rather than precedent. The liability question remains open, and it is being answered case by case, in a patchwork of settlements and early trial decisions, in ways that will shape the deployment of medical AI for the next twenty years.

The Standard of Care Problem

American medical malpractice law is built around the concept of standard of care — the conduct that a reasonably competent physician in the same specialty would provide under the same circumstances. Liability attaches when a physician deviates from that standard in a way that causes harm.

AI complicates this framework in at least two directions. First, if an AI tool is widely deployed and generally performs at a certain level, does using AI become part of the standard of care? The answer is probably yes, eventually, for tools with strong evidence bases. Diabetic retinopathy screening with AI is arguably already standard of care in primary care settings where ophthalmologists are unavailable. A primary care physician who fails to use an available, validated AI screening tool for a patient who develops preventable vision loss from undetected retinopathy may have a liability exposure that didn’t exist five years ago.

Second, and more unsettling: if a physician uses an AI tool that performs badly on their specific patient population — because the training data was from a different demographic, or the imaging equipment differs from the validation cohort — and the AI misses something they might have caught without it, has the physician’s standard of care changed by virtue of having deployed the tool? Several legal scholars have argued that AI-assisted diagnosis creates a “standard-of-care trap”: physicians who use AI are expected to catch what AI misses (because they are the final decision-maker), but cognitive research demonstrates that AI use changes how physicians process information in ways that can increase miss rates on AI-negative cases.

Related reading

The Learned Intermediary Problem

Pharmaceutical liability law solved an analogous problem decades ago with the learned intermediary doctrine: a drug manufacturer’s duty to warn runs to physicians, not patients, because physicians are the “learned intermediaries” who evaluate risk and apply professional judgment. The manufacturer is not liable for failure to warn patients directly because patients don’t prescribe.

Medical AI vendors have been attempting to import this logic into diagnostic AI liability. The argument goes: the AI tool is a software medical device; the FDA cleared it for use as a decision-support tool under physician supervision; the physician is the learned intermediary who makes the final call. If the physician fails to catch what the AI missed, liability attaches to the physician (and their employer), not to the AI vendor.

Courts have been receptive to this argument in the narrow cases where it has been tested, but only partially. In Petrov v. Aidoc Inc. (D. Ariz., 2025), the court declined to dismiss the vendor entirely, finding that the vendor had an ongoing duty to warn about known performance limitations — specifically, the well-documented performance degradation in non-white patient populations — that it had not adequately disclosed in its labeling or training materials. The case proceeded to discovery before settling, but the ruling created a precedent that AI vendors cannot fully insulate themselves from liability through learned intermediary doctrine when they had knowledge of systematic performance deficits.

The Disclosure Gap

The Petrov decision points to a disclosure problem that is industry-wide. The FDA’s 510(k) clearance process for AI-based devices requires validation data, but does not require vendors to proactively notify customers when post-market performance surveillance reveals that real-world performance differs materially from the validation dataset. A vendor who clears a device based on performance on a predominantly white, male, academic medical center dataset and then deploys it to community hospitals serving different demographics is not legally required — under current FDA rules — to disclose the performance gap.

This is changing. The FDA’s 2024 AI/ML-based Software as a Medical Device action plan included provisions for “predetermined change control plans” and enhanced real-world performance monitoring requirements. Several states, led by California (AB 2088, signed 2025), have enacted state-level disclosure requirements that go further than federal standards, requiring AI diagnostic vendors to provide health systems with demographic performance breakdowns as a condition of procurement.

The gap between what vendors know about their tools’ performance limitations and what they tell deploying hospitals remains significant. A 2025 survey by the American College of Radiology found that 63 percent of radiologists who used AI tools in their practice were not aware of the demographic performance characteristics of those tools. They were using something they didn’t fully understand, on patients who hadn’t consented to be the test subjects for its real-world generalization.

Consent and Transparency

Patient consent in AI-assisted diagnostics is almost entirely notional. The typical patient who has a chest X-ray read with AI assistance has signed a general consent for diagnostic services that says nothing about AI. They have no mechanism to opt out, no information about which tool was used, and no access to the AI’s output as a separate artifact from the radiologist’s report.

In the context of civil liability, this doesn’t create immediate problems — a patient generally doesn’t need to have consented to AI use in order to have a negligence claim if they were harmed. But it creates a deeper accountability problem. When AI influences a diagnostic decision, and that decision causes harm, the patient has no record of what happened and therefore no basis for inquiry beyond what the medical record reflects. The AI’s output is often not preserved in the medical record. The AI vendor’s logs, which would show what the tool saw and how confident it was, are not part of the discoverable medical record in most current deployment contracts.

Related reading

Several plaintiff’s attorneys have begun demanding AI audit logs in discovery in medical malpractice cases involving AI-assisted diagnosis. The legal battles over whether those logs are discoverable, and whether they fall under work-product or trade-secret protection, are now being fought in multiple jurisdictions simultaneously.

The International Comparison

The European approach is attempting to resolve this through product liability rather than medical malpractice. The EU’s updated Product Liability Directive (2024) explicitly covers AI systems and software, creating a strict liability path for harms caused by defective AI products. Under this framework, a patient harmed by an AI diagnostic tool can sue the vendor directly as a product liability claim, without needing to prove physician negligence.

This is a fundamentally different liability allocation than the American system, and it creates different incentives. Under European strict liability, AI vendors bear more direct exposure for product failures, which should incentivize better validation, disclosure, and monitoring. Under the American system, where liability currently channels primarily to physicians and hospitals, the incentive structure for vendors is weaker. They compete on clearance and feature set, not on comprehensive post-market liability exposure.

The American plaintiffs’ bar, historically innovative in developing new liability theories when they perceive undercompensated harm, is watching the European approach carefully.

What Should Actually Change

The AI diagnostic liability question is not a technical problem — it’s a governance problem, and governance lags technology in medicine the way it lags technology everywhere. Several specific changes would reduce both harm and litigation:

Standardized disclosure requirements, at the federal level, for demographic performance breakdowns before procurement and as updates when real-world data diverges from validation performance. Preservation of AI decision artifacts in the medical record as part of the patient’s clinical history, available for review and discovery. Informed consent language in general consent forms that names AI-assisted tools in a patient’s care when they are used, not comprehensively but meaningfully. And a clearer FDA post-market surveillance framework for AI medical devices that creates the same ongoing vigilance expectations that exist for pharmaceutical products.

None of this is technically difficult. All of it is politically contested among parties whose financial interests are not aligned with the patients who will need the law to work correctly when the algorithm misses.

The Insurance Dimension

Medical malpractice insurance is a leading indicator of how the liability landscape is shifting. Several major medical malpractice insurers have begun asking hospitals whether they use AI diagnostic tools as part of their underwriting questionnaires — a practice that barely existed in 2022 and is now becoming standard. The insurers are not yet penalizing AI use; the data is too thin to establish whether AI-assisted practice increases or decreases malpractice risk overall. They are collecting data that will eventually inform pricing, which will eventually drive hospital behavior.

Related reading

The dynamic is analogous to how automotive insurers responded to the introduction of advanced driver assistance systems (ADAS). Initially, it was unclear whether ADAS reduced or shifted accident risk. As actuarial data accumulated, the insurers built that data into pricing models. Hospitals will eventually face the same analysis: does using AI diagnostic tools reduce their malpractice exposure (by catching more findings) or increase it (by introducing new failure modes and reducing physician vigilance)? The answer probably differs by tool, by indication, and by implementation quality — which means the insurance models that emerge will be granular in ways that create strong incentives for hospitals to deploy and monitor AI responsibly.

The Patient Perspective

There is a dimension of the liability question that legal frameworks are poorly structured to address: the patient who was harmed by AI never knew they were interacting with AI. When an AI tool misses a finding and the radiologist confirms the miss, the patient’s medical record says “normal chest X-ray — reviewed by Dr. [name].” There is no entry for the AI tool, no indication that the radiologist’s review was AI-assisted, no information about which tool was used or what its known performance limitations are.

This opacity is not accidental. Clinical documentation practices evolved before AI-assisted diagnosis existed, and they haven’t been reformed to reflect it. A patient seeking to understand why their diagnosis was delayed has no documentary basis for discovering that an AI tool was part of their care pathway — unless their attorney subpoenas the hospital’s system logs in discovery, which requires first having reason to suspect that AI was involved.

The fundamental principle of informed consent — that patients have the right to understand what is being done to them and to participate in decisions about their care — is being systematically violated in AI-assisted diagnosis not through malice but through inertia. Fixing it requires updating documentation standards, consent language, and patient communication practices in ways that the healthcare system has been slow to prioritize.