llm-evaluation

37 projets partagent ce topic GitHub

Les traits relient les membres réellement apparentés entre eux. La taille des points suit les étoiles.

🧬 Membres

🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground,…

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to…

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare…

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive…

The LLM Evaluation Framework

AI Observability & Evaluation

the LLM vulnerability scanner

chinese-llm-benchmark

非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个…

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23…

🐢 Open-Source Evaluation & Testing library for LLM Agents

LLM-Engineers-Handbook

The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps…

AutoRAG: Now your agent can find anything in your computer. It gets smarter if you are using it frequently.

The open-source workspace for building and running AI agents. Build agents through chat, share them with your…

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

A full-stack AI Red Teaming platform securing AI ecosystems via OpenClaw Security Scan, Agent Scan, Skills…

Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in…

Evaluation and Tracking for LLM Experiments and AI Agents

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview…

A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers…

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications.…

Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty…

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and…

AI equity research agent with resilient workflows, Redis Lua single-flight, pgvector RAG, versioned reports,…

A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs,…

Awesome-LLM-Eval

Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models,…

agent-skills-eval

A test runner for agentskills.io-style AI agent skills

Awesome-LLM-in-Social-Science

Awesome papers involving LLMs in Social Science.

Open-source benchmark for browser AI agents on daily tasks.

continuous-eval

Data-Driven Evaluation for LLM-Powered Applications

EnterpriseRAG-Bench

Dataset and benchmark for RAG on company internal documents.

llm-leaderboard

A comprehensive set of LLM benchmark scores and provider prices. (deprecated, read more in README)

Build, Improve Performance, and Productionize your AI Application

UltraEval-Audio

Your faithful, impartial partner for audio evaluation — know yourself, know your rivals.…

Python SDK for running evaluations on LLM generated responses

Evaluate & benchmark AI coding agents and Claude Code skills — sandboxed, reproducible YAML eval suites for…

ai-reliability-copilot

Turn a production incident into a structured 9-section LLM response (severity, root cause, mitigation,…

🔗 Familles voisines

llm-observability 25 evaluation-framework 12 llm-eval 7 agent-evaluation 9 evals 17 llm-security 27 llama-index 23 observability 66

Mesuré à partir des topics GitHub communs aux deux projets, pondérés par leur rareté.