Hugging Face says ‘autonomous AI agent’ hacked its data pipeline

Hugging Face says the attack 'was driven, end to end, by an autonomous AI agent system'

MANILA, Philippines – Open-source coding and research community platform Hugging Face disclosed it had been the victim of a security incident in which its data pipeline was attacked by an “autonomous AI agent system.”

According to its security disclosure on July 16, the attack against Hugging Face “was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system — and we detected and dissected it largely with AI of our own.”

It said, “A malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker.” The threat actor made its way up through Hugging Face’s system, harvesting cloud and cluster credentials, and moving into several internal clusters.

Checking the data

Hugging Face said it is still assessing if any partner or customer data was affected, but will contact affected parties directly as required.

It said it “found no evidence of tampering with public, user-facing models, datasets, or Spaces, and our software supply chain (container images and published packages) was verified clean.” It has also fixed the root vulnerability and removed the attacker’s access across what was affected.

Hugging Face also said it was working to investigate what happened with cybersecurity forensics specialists and reported the incident to law enforcement.

Learning from the incident

Hugging Face was learning from the incident.

Because it did not know what model powered the attackers’ agentic AI — “the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried” — it was a practical lesson for those defending against such types of attacks: “to have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment.”

The incident also pointed to AI-driven attacks no longer being theoretical, Hugging Face explained: “It lowers the cost of running a broad, patient, multi-stage campaign, and it operates at machine speed.”

It added, “Defending an online platform now means treating the data and model surface as a first-class attack surface, and using AI on defense to keep pace.” – Rappler.com