Two researchers highlight that certain Hugging Face ML models can execute code on load, enabling attackers to take control and implant backdoors. The report analyzes malicious models, their serialization-based exploit methods (notably Pickle in PyTorch and Lambda layer in TensorFlow/Keras), and recommended mitigations like safer model formats and MLOps security checks. #HuggingFace #baller423
Keypoints
- NSFOCUS and JFrog researchers identify malicious Hugging Face ML models capable of code execution when loaded, enabling attacker control and backdoor persistence.
- At least 100 malicious AI model instances were found on Hugging Face, including baller423/goober2, which can directly execute code and provide persistent access.
- Approximately 95% of malicious models are PyTorch-based, with the remaining 5% using TensorFlow/Keras, exploiting serialization to trigger code execution.
- In PyTorch, attackers embed malicious data in Pickle-based model files (data.pkl) that, when deserialized, execute arbitrary code via mechanisms like __reduce__.
- In TensorFlow/Keras, Lambda layers in models can execute code during deserialization (marshal.loads), enabling similar exploitation via HDF5 stored models.
- Hugging Face has introduced Safetensors to store model data securely and employs malware, pickle, and secret scanning, but these scans do not block downloads outright, only mark unsafe content.
- Mitigation emphasizes reviewing external models in MLOps pipelines and adding security checks, such as Python Pickle static analyzers, during model loading and deployment.
MITRE Techniques
- [T1203] Exploitation for Client Execution – Malicious code is triggered during model loading; “This malicious ML attack technology uses the process of loading models in the Transformers library to trigger malicious code execution.”
Indicators of Compromise
- [IP Address] Context – Four IPs extracted by NSFOCUS Threat Intelligence (NTI) associated with the activity: 192.248.1.167, 136.243.156.120, 136.243.156.104, and 210.117.212.93
Read more: https://nsfocusglobal.com/ai-supply-chain-security-hugging-face-malicious-ml-models/