Supply Chain Attack: Fake OpenAI 'Privacy Filter' Pushing Rust Stealer on Hugging Face
Just saw the alert regarding the Open-OSS/privacy-filter repository hitting #1 on Hugging Face. It’s crazy that a typosquatting attempt managed to rack up 244k downloads before being taken down. The repo was a near-perfect clone of the legitimate openai/privacy-filter, but the payload is a Rust-based information stealer targeting Windows.
This highlights a massive gap in how we handle ML model supply chains. Everyone is vetting pip packages now, but who is checking the .safetensors or hidden scripts in a model repo?
From an analysis perspective, the malware is written in Rust, which usually means smaller binary size and harder reverse engineering. If you have machines that pulled this specific repo, you might want to check for suspicious child processes or unexpected network connections.
I've whipped up a quick Python snippet using the huggingface_hub library to validate repository authors before pulling. It's a simple sanity check to prevent this specific flavor of typosquatting:
from huggingface_hub import HfApi
repo_id = "openai/privacy-filter" # The intended repo
expected_author = "openai"
api = HfApi()
model_info = api.model_info(repo_id)
if model_info.author != expected_author:
print(f"[!] Alert: Author mismatch! Expected {expected_author}, found {model_info.author}")
else:
print(f"[+] Author verified: {model_info.author}")
Given the surge in AI adoption, is anyone else looking into integrating model registry scanning into their existing CI/CD pipelines, or are we treating these repos differently than standard code packages?
Solid catch on the author verification. We're seeing a lot of shops just git clone or snapshot_download without checking metadata.
On the detection side, since this drops a Rust binary, standard sig-based AV might struggle initially. We've had luck hunting for the absence of ETW (Event Tracing for Windows) events often associated with custom Rust malware, but that's noisy.
If you suspect infection, checking for PowerShell web requests in your logs is a good start, as stealers often phone home using Invoke-WebRequest or similar:
Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-PowerShell/Operational'; ID=4103} | Where-Object {$_.Message -match 'DownloadString|Invoke-WebRequest'}
This is essentially the new PyPI typosquatting wave but for data. The scary part is the scale; 244k downloads in no time.
We implemented a pre-commit hook that checks requirements.txt and model configs against an internal allow-list. It’s tedious to maintain initially, but it stops the accidental pull of 'open-ai' vs 'openai'.
Also, don't forget to hash your models. Even if the author is valid, a repo compromise can still happen. Pin the SHA256 of the model file.
From a pentester's perspective, this is a goldmine for initial access. Developers often run these models locally on high-privilege workstations with GPU access.
If you're blue teaming, look for unusual process trees spawning from Python or Conda environments that shouldn't be executing binaries outside the sandbox. A simple parent-child process query in Sentinel can help surface this if you have EDR data flowing in.
The sheer velocity of that download count is terrifying. Until we see widespread cryptographic signing for ML artifacts, we need to treat these repos like untrusted USB drives.
For immediate defense, consider auditing the file manifest before loading. This quick bash scan flags non-standard executables hidden inside model folders:
find . -type f \( -name "*.exe" -o -name "*.dll" -o -name "*.bat" \)
It’s a crude filter, but it stops the obvious stealer drops.
It’s scary how easily PE files slip into these model artifacts. Since standard scanners might miss non-standard file extensions, I’d recommend running a quick static analysis on any downloaded .safetensors files before loading them. You can grep for binary signatures like the MZ header to flag embedded executables immediately.
Here is a quick bash one-liner to check for embedded Windows binaries:
strings -a model_file.safetensors | grep -E '^MZ\x90\x00'
This simple check catches a lot of these 'trojanized' models before they even hit memory.
Great thread. To catch these cross-platform payloads in a Linux CI environment, I've added a simple file type check to my pipeline. It’s crude but effective for spotting PE files masquerading as model data.
find ./model_cache -type f -exec file {} + | grep -i "PE32"
If that returns anything, the build fails immediately. It’s a low-effort way to prevent accidental execution of the stealer if someone tries to run the payload via Wine or similar tools during testing.
Beyond static analysis, we need to focus on runtime containment. Since these payloads often rely on C2 callbacks, strict egress filtering is a must. I recommend running inference in isolated containers with dropped network capabilities if the task allows.
docker run --network none -v $(pwd):/data python:latest python inference.py
This creates a fail-safe that prevents the stealer from exfiltrating data even if it executes.
Verified Access Required
To maintain the integrity of our intelligence feeds, only verified partners and security professionals can post replies.
Request Access