garak
FeaturedAn open-source LLM vulnerability scanner that probes a model or LLM app for prompt injection, leaks and jailbreaks.
garak is a command-line LLM vulnerability scanner backed by NVIDIA and the community. It checks if an LLM-backed application can be made to fail in a way you do not want. Its probes cover hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks and other weaknesses, with static, dynamic and adaptive probes. The project compares itself to nmap or Metasploit, but for LLMs. It supports models on the Hugging Face Hub, the OpenAI API, AWS Bedrock, LiteLLM and more. The code repository is under the Apache 2.0 licence. Security and red-team leads use it to collect repeatable test evidence before a release.
- Price
- Free, open source (Apache 2.0)
- Model
- free
- Platforms
- CLI, Python
- Category
- AI red teaming
- Link checked
Published by AIGovernanceTools. How we check each AI governance tool.