Garak is NVIDIA’s open-source LLM vulnerability scanner that automates hundreds of attack probes—prompt injection, DAN-family jailbreaks, encoding bypasses, data leakage, toxicity, and more—against any model reachable via common APIs. It runs repeated generations, scores responses with detectors, and outputs structured JSONL reports so teams can map vulnerabilities and prioritize fixes. #Garak #NVIDIA
Keypoints
- NVIDIA’s Garak automates LLM red-teaming with a CLI-first, plugin-based scanner.
- It probes models across 37+ modules including dan, encoding, promptinject, leakreplay, and packagehallucination.
- Each probe runs multiple generations and uses detectors to produce a JSONL audit trail of prompts, responses, and verdicts.
- Findings map to known vulnerability classes (e.g., LLM01:2025, LLM02:2025) to guide remediation priorities.
- Generators support OpenAI, Hugging Face, AWS Bedrock, Mistral, Ollama, NVIDIA NIM, and custom REST APIs for cloud and local model scanning.
Read More: https://www.toxsec.com/p/garak-llm-vulnerability-scanner