evals
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Evaluate and test large language models with existing benchmarks or create your own custom tests.
From the README
pip install evals
- Walking through the process for building an eval: build-eval.md
- Exploring an example of implementing custom eval logic: custom-eval.md
- Writing your own completion functions: completion-fns.md
Excerpts from the project README on GitHub. Copyright and licensing remain with the respective authors.
Claim its page: indexed whatever its rank, translated into six languages, and enriched with what you write yourself.
Get an email alert on its next release or when it starts trending — never miss the moment.
Free · no card · unsubscribe anytimeEvals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
evals has 19.3k stars on GitHub. It has been forked 3.1k times. evals is written mainly in Python. It has been in active development since 2023.
What is evals?
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Is evals open source?
evals is an open-source project.
Is evals free?
Yes. evals is free and open source — you can use, modify and self-host it.
What language is evals written in?
evals is written mainly in Python.
Add this live badge to your README — your GitHub stars and directory rank, refreshed daily.
[](https://olud.ai/project/openai-evals.html)
