modelpoısonıng Español

Malicious models on Hugging Face: how a download can run code

Poisoning a model does not always mean touching its training data. Sometimes the file you download does the work the moment you open it.

Published 23 October 2026, 6 min read

Why loading a model can run code

Python’s pickle format does not just store data. It stores instructions for rebuilding Python objects, and those instructions can name any function to call while the file is being loaded. A malicious file can therefore tell Python to run a shell command, download a program or open a network connection, and it does so before the model has produced a single output.

Many model files are pickles underneath. A PyTorch .bin or .pt file is a ZIP archive with a pickle inside it. Loading one with an old or misconfigured tool is, in effect, running a program written by whoever uploaded it.

Real cases

2024: about a hundred models with real payloads

In February 2024, JFrog’s security research team scanned PyTorch and TensorFlow Keras models on Hugging Face and reported “around 100 instances” of models with malicious functionality (JFrog). In one, uploaded by an account called baller423, loading the model opened a reverse shell: a connection that gives a remote attacker a command line on the victim’s machine. The repository has since been deleted.

2025: a broken file that the scanner skipped

In February 2025, ReversingLabs described two models on Hugging Face that it called nullifAI (ReversingLabs). The files were compressed with 7z instead of the usual ZIP and the pickle inside was broken near its end. Picklescan, the open source tool Hugging Face used to inspect pickles, first checked whether a file was valid; these files failed the check, so the dangerous code was never flagged. Python, however, runs a pickle’s instructions in order, so the malicious part at the start executed before the corruption caused an error. Hugging Face removed the models within 24 hours and updated Picklescan.

Later that year, JFrog reported three critical flaws in Picklescan itself that allowed malicious models to pass its checks undetected; they were fixed in version 0.0.31 (Infosecurity Magazine).

2026: a fake OpenAI release at number one

In May 2026, a repository called Open-OSS/privacy-filter copied the model card of a genuine OpenAI release almost word for word and reached number one on Hugging Face’s trending list within 18 hours (CSO Online). It showed about 244,000 downloads, figures that the security firm HiddenLayer, which found the malware, said were almost certainly inflated to make it look legitimate.

This attack did not need a malicious pickle. The README told users to run a script, loader.py or start.bat, which fetched and installed malware that stole browser passwords, cryptocurrency wallets and other data. Hugging Face removed the repository.

When the payload targets the model itself

The cases above attack the computer. In 2024, Trail of Bits described a subtler use of the same weakness, which it called Sleepy Pickle (Trail of Bits). The malicious code runs when the file is loaded and modifies the model in memory: it can patch the weights to insert a backdoor or hook the model’s methods to tamper with what goes in and comes out.

This is model poisoning without touching the training data. The weights stored in the file can look normal and pass a comparison with the original, because the change only exists once the model is running.

The name you trust can change hands

Pulling a model by name, such as author/model, assumes that name always points to the same publisher. In September 2025, Palo Alto Networks’ Unit 42 showed that this is not guaranteed (Unit 42). When an author or organisation is deleted from Hugging Face, its name can be registered again by anyone. An attacker who recreates a model path under an abandoned name can reach every pipeline that still downloads it by name. The researchers used this to get code execution on managed AI platforms run by Google and Microsoft.

How common is it?

Hugging Face scans uploads with several tools, including its own and, since 2024, Protect AI’s. By April 2025, Protect AI had scanned 4.47 million model versions in 1.41 million repositories and flagged 352,000 unsafe or suspicious issues across 51,700 models (Hugging Face). Not every flagged issue is an attack, but the numbers show why the hub treats files as untrusted until scanned, and why you should too.

What to do before you load a model

  • Prefer safe formats. Safetensors and GGUF store weights without executable code. In the transformers library, use_safetensors=True makes sure those are the files that load.
  • If you must load a pickle-based file, restrict it. torch.load only accepts plain weights by default since PyTorch 2.6 (weights_only=True). Keep PyTorch up to date: versions up to 2.5.1 had a flaw that allowed code execution even with that setting (NVD, CVE-2025-32434).
  • Pin the exact version. Download a specific commit with the revision parameter instead of whatever the name points to today. It protects against both later changes and namespace reuse.
  • Read before you run. Never run a script because a model card tells you to, and read the Python files of any model that needs trust_remote_code=True.
  • Do not trust popularity. Downloads, likes and trending positions can be inflated, as the fake OpenAI repository showed.
  • Load untrusted models in a sandbox without network access or credentials, as recommended in the defence checklist.

Our model checker does the first checks for you: it shows what Hugging Face’s scanners flagged, which files use pickle, whether the model needs custom code, and the commit to pin.

Sources

Read next