promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Promptfoo runs your prompts against test cases across models, catching regressions before they reach production, and includes red-teaming checks.
From the README
npm install -g promptfoo promptfoo init --example getting-started
- Test your prompts and models with automated evaluations
- Secure your LLM apps with red teaming and vulnerability scanning
- Compare models side-by-side (OpenAI, Anthropic, Azure, Bedrock, Ollama, and more)
- Automate checks in CI/CD
- Review pull requests for LLM-related security and compliance issues with code scanning
- Share results with your team
Excerpts from the project README on GitHub. Copyright and licensing remain with the respective authors.
Claim its page: indexed whatever its rank, translated into six languages, and enriched with what you write yourself.
Get an email alert on its next release or when it starts trending — never miss the moment.
Free · no card · unsubscribe anytimeTest your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
promptfoo has 24.7k stars on GitHub. It has been forked 2.3k times. promptfoo is written mainly in TypeScript. It has been in active development since 2023. promptfoo is available under the MIT license. Its main topics are ci, ci-cd, cicd, evaluation.
Read the full guideWhat is promptfoo?
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Is promptfoo open source?
promptfoo is an open-source project. It is released under the MIT license.
Is promptfoo free?
Yes. promptfoo is free and open source — you can use, modify and self-host it.
What license does promptfoo use?
promptfoo is available under the MIT license.
What language is promptfoo written in?
promptfoo is written mainly in TypeScript.
Add this live badge to your README — your GitHub stars and directory rank, refreshed daily.
[](https://olud.ai/project/promptfoo-promptfoo.html)
Measured from GitHub topics shared by both projects, weighted by how rare each topic is.