llm-evaluation

42 projetos partilham este topic do GitHub

Linhas conectam membros que estão mensuravelmente relacionados entre si. O tamanho do ponto reflete estrelas.

🧬 Membros

🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground,…

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to…

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare…

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive…

The LLM Evaluation Framework

Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in…

AI Observability & Evaluation

the LLM vulnerability scanner

chinese-llm-benchmark

非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个…

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23…

A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra…

🐢 Open-Source Evaluation & Testing library for LLM Agents

Agent OS: Stop prompting. Start specifying. A Socratic interview gates the spec on an ambiguity score, then…

LLM-Engineers-Handbook

The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps…

AutoRAG: Now your agent can find anything in your computer. It gets smarter if you are using it frequently.

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

AI-Engineer-Headquarters

A collection of scientific methods, processes, algorithms, and systems to build stories & models.

Evaluation and Tracking for LLM Experiments and AI Agents

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview…

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications.…

A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers…

Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty…

Trace-native CI/CD for AI agents — production failures become regression tests that block the PR.…

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and…

AI equity research agent with resilient workflows, evidence-grounded RAG, versioned reports, and automated…

A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs,…

Trace-native CI/CD for AI agents — production failures become regression tests that block the PR.…

agent-skills-eval

A test runner for agentskills.io-style AI agent skills

Awesome-LLM-in-Social-Science

Awesome papers involving LLMs in Social Science.

Open-source benchmark for browser AI agents on daily tasks.

EnterpriseRAG-Bench

Dataset and benchmark for RAG on company internal documents.

continuous-eval

Data-Driven Evaluation for LLM-Powered Applications

The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests,…

A long-horizon benchmark harness: give a coding agent a real game engine, professional conditions and time,…

llm-leaderboard

A comprehensive set of LLM benchmark scores and provider prices. (deprecated, read more in README)

Evolutionary multi-agent runtime that breeds, evaluates, and improves autonomous agents across reproducible…

Compile real-world Claude Code and Codex trajectories into verified, tradable post-training assets.

★ 196 · GitHub ↗

Engineering deterministic, production-grade systems around non-deterministic LLMs — FSM, durable execution,…

ai-eval-platform

ai-reliability-copilot

Turn a production incident into a structured 9-section LLM response (severity, root cause, mitigation,…

Evaluate visual models on your own images, JSON Schema, and production constraints with LangGraph, Pareto…

🔗 Familias relacionadas

llm-observability 26 agent-evaluation 9 llm-security 32 evals 19 observability 68 rag-evaluation 8 llama-index 17 mlops 59

Medido a partir dos tópicos do GitHub compartilhados por ambos os projetos, ponderado pela raridade de cada tópico.