Your AI vendor promised privacy. Its agents did not get the memo.

The Privacy Promise in Your AI Contract Assumes Someone Is in Control

Every AI vendor selling into healthcare makes the same promises: your data will be protected, it won't be misused, and you'll be told if something goes wrong. Those promises sit in business associate agreements and data processing addenda, and most of us read them assuming the threat is an outside attacker. Last week's disclosure from OpenAI suggests we have been reading them too narrowly.

OpenAI said agents in its research environment had posted 53 user-provided images to image-hosting sites, and while the links were unlisted, the images could still be discovered. This wasn't a typical hack. An AI agent operated by OpenAI accessed internal training data and then posted it directly to the open internet. And OpenAI can't tell the people affected, because the anonymization process that strips names, contact details and metadata from training data made it difficult to trace the original users.

A promise needs a promisor in control

A contract promise is only as good as the promisor's ability to keep it. "We will not disclose your data" means one thing when a vendor's people and systems do what they are told, and something less when the vendor's own autonomous agents can find data, decide it is useful, and send it elsewhere. Nothing here suggests bad faith; OpenAI disclosed the incident and worked with hosting providers to remove most of the exposed content. That is exactly the point. A vendor acting in good faith still could not stop it.

The healthcare stakes are obvious. Patients and clinicians upload photos of wounds, rashes, test results and insurance cards to AI tools every day. OpenAI has not publicly identified the images, their upload dates, or the people involved, so we don't know what these 53 showed. But the notification failure should stop every privacy officer. Breach notification assumes you can identify whose data was exposed, and a photo of a face or a wound can identify someone long after the metadata is gone.

Questions your contract should answer

Before relying on an AI vendor's assurances, ask what they actually cover. Does the agreement address actions taken by the vendor's own AI agents, not just its employees and subcontractors? Is your data ever placed in research, training or evaluation environments, and what can agents in those environments reach? What stops an agent from sending data outside, and are those actions logged?

Ask, too, what happens when the vendor can't tell you who was affected. Your notification obligations don't disappear because a vendor's de-identification made the answer unknowable, and if the contract is silent, the risk is yours.

The privacy promises in AI contracts were written for a world where vendors controlled their own systems. Until a vendor can show it controls its agents, read those promises as intentions rather than guarantees.

Timothy M. Wogan, JD, CHC, is a healthcare compliance attorney and founder of Wogan Health Law — Counsel for Healthcare. He advises healthcare organizations on compliance, privacy and incident response, and writes the daily brief at Our Office Next Door.

Every morning: the compliance developments that affect your organization, and what to do about them. Start free, no card needed, at ourofficenextdoor.com