llmops

120 progetti condividono questo topic GitHub

Le linee collegano membri che sono misurabilmente correlati tra loro. La dimensione dei punti riflette le stelle.

🧬 Membri

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data.…

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format…

🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground,…

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to…

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to…

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare…

☁️ Build multimodal AI applications with cloud-native stack

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive…

<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build,…

RagaAI-Catalyst

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm…

Supercharge Your LLM Application Evaluations 🚀

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast &…

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and…

AI Observability & Evaluation

Build, Manage and Deploy AI/ML Systems

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model…

⚙️🦀 Build modular and scalable LLM Applications in Rust

Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any…

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent…

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode,…

agent-starter-pack

Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/CD,…

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23…

An awesome & curated list of best LLMOps tools for developers

🐢 Open-Source Evaluation & Testing library for LLM Agents

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by…

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

LLM-Engineers-Handbook

The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps…

llm-twin-course

🤖 𝗟𝗲𝗮𝗿𝗻 for 𝗳𝗿𝗲𝗲 how to 𝗯𝘂𝗶𝗹𝗱 an end-to-end…

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Agent Skills as a Memory Layer

Evaluation and Tracking for LLM Experiments and AI Agents

The platform for LLM evaluations and AI agent testing

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version…

AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and…

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

An open-source visual programming environment for battle-testing prompts to LLMs.

second-brain-ai-assistant-course

Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems…

Turn expensive prompts into cheap fine-tuned models

Community maintained hardware plugin for vLLM on Ascend

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and…

🤖Self-Modifying Framework from the Future 🔮 World's First AMS

Bionic is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining…

No-code multi-agent framework to build LLM Agents, workflows and applications with your data

A Unified MCP Server Management App (MCP Manager).

AICI: Prompts as (Wasm) Programs

Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙…

Standards for building agents, better

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications.…

The SmythOS Runtime Environment (SRE) is an open-source, cloud-native runtime for agentic AI. Secure,…

A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic…

awesome-ai-sdks

A database of SDKs, frameworks, libraries, and tools for creating, monitoring, debugging and deploying…

Dynamiq is an orchestration framework for agentic AI and LLM applications

The web framework for building LLM microservices [deprecated]

⛓️ Serving LangChain LLM apps and agents automagically with FastApi. LLMops

🧠 Motorhead is a memory and information retrieval server for LLMs.

Interactive CV with AI chat integration. Built with React 19, TypeScript, Claude API. Chat with my AI avatar…

LLM全栈优质资源汇总

Waku Waku! Waku agent is your personal AI agent, on your own laptop, in code you can read in an afternoon —…

★ 633 · GitHub ↗

END-TO-END-GENERATIVE-AI-PROJECTS

End to End Generative AI Industry Projects on LLM Models with Deployment_Awesome LLM Projects

The only fully local production-grade Super SDK that provides a simple, unified, and powerful interface for…

LLM-Agents-Ecosystem-Handbook

One-stop handbook for building, deploying, and understanding LLM agents with 60+ skeletons, tutorials,…

See what your coding agents did and what it cost. Breaks each task down into work steps — tools used, files…

★ 538 · GitHub ↗

Deprecated historical repo. Superlinked now develops SIE, a self-hosted inference engine for embeddings,…

EnterpriseAgentFramework

面向 OA、ERP、CRM、工单等企业系统快速构建智能体，让 AI…

🕵️‍♂️ Library designed for developers eager to explore the potential of Large Language Models…

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM)…

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs…

LLMs and Machine Learning done easily

Govern & Secure your AI

Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based…

Framework to bring LLM applications to production

The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests,…

Tooling to build LLM applications: prompt templating and composition, agents, LLM memory, and other…

llm-leaderboard

A comprehensive set of LLM benchmark scores and provider prices. (deprecated, read more in README)

Open-source, OpenAI-compatible LLM gateway you run yourself. One endpoint for 40+ providers, with virtual…

Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.

Open-source sandboxes where coding agents build and deploy. Spin up isolated environments where Claude Code,…

Open-source AI gateway for local agent clients: publish fusion-style multi-model workflows as…

Free, local, open-source LLM cost analyzer - see where your LLM bill leaks, on your machine.

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test &…

Local proxy that compresses your LLM API requests so you pay less, with no change to the answers. Trims…

Production LLM call layer for AI agents and tools: keep OpenAI/Anthropic/AI SDK/LiteLLM, hot-swap models with…

Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage,…

Local-first observability your agent reads back. TMA1 records every LLM call, then routes what it sees into…

An orchestration runtime for multi-agent AI systems. Declare agents, tools, and policies as YAML; Orloj…

Point your existing SDK at one URL and reach every LLM vendor — with real failover, not a try/except. One…

awesome-ai-tokenomics

A map of what AI tokens actually cost, and where they're wasted vs. well spent. Tools, research, practices,…

★ 105 · GitHub ↗

Unified CLI for running AI coding agents in isolated containers. Includes built-in local metrics collection,…

★ 104 · GitHub ↗

Selfhost modern LLM stacks. Run the whole fleet from your terminal

AI Engineering Specially Topics- Agentic AI & GenAI Explanation

半人马环 Centaur Loop：AI 员工的最小工作单元框架。把复杂岗位拆解为可由 AI…

agentic-rag-financial-parser

Enterprise RAG ecosystem managing 32000+ semantic chunks. Features hybrid parsing (LlamaParse/PyMuPDF) and…

Self-hosted, Docker-first OpenAI-compatible LLM gateway with provider routing, fallbacks, API keys, and usage…

★ 86 · GitHub ↗

A secure file system for your agents to execute code

Local outcome-loop + reward layer for coding agents — remember what worked, ship the next move, measure how…

Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.

Continuum — the agent runtime by ShyftLabs. Build, orchestrate, ship.

Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic,…

awesome-ai-gateway

⚡ Awesome AI Gateway — pick an AI gateway from 100+ (LiteLLM, OpenRouter, Portkey, Kong, Higress,…

Agent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude,…

AI-ML Companion: an interactive platform to learn AI & ML by watching it work - 22 tracks, 300+ modules, live…

An open source, Native AI Gateway and LLM proxy built in Rust

Auto-delegation, status truth, and cost-aware model routing for OpenClaw agents

The "Economy Mode" for AI Compute

★ 56 · GitHub ↗

Economical human-readable logs and structural (event pattern) retrieval for agentic trajectories

Six Claude Code skills that harden Opus 4.8 toward frontier behavior — written by Fable 5, pressure-tested…

★ 53 · GitHub ↗

Bring your own key (BYOK) relay to enable lightweight frontend-only apps to use LLMs using the user's API…

★ 53 · GitHub ↗

Zero-code LLM security & observability proxy. Real-time prompt injection detection, PII scanning, and cost…

★ 52 · GitHub ↗

Self-hosted AI gateway and LLM proxy. OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock and 60+…

★ 47 · GitHub ↗

Bounded, inspectable LLM inference pipelines from declared YAML — runs offline against Ollama or any…

★ 45 · GitHub ↗

llm_cost_tracker

Rails-native LLM cost ledger: track spend by provider, model, and feature with self-hosted storage and budget…

★ 44 · GitHub ↗

Self-hosted LLM router. Cost effective, deterministic, and fast. Secure and private by default.

★ 43 · GitHub ↗

Turn scattered knowledge, operational data, and history into source-linked context that your agents can…

★ 42 · GitHub ↗

Lightweight cost observability for every API call in your stack. Track LLM tokens, database ops, and external…

★ 42 · GitHub ↗

Make AI coding agents safe to scale autonomously: assign work, cap spend, enforce policy, verify output, roll…

★ 41 · GitHub ↗

🔗 Famiglie affini

observability 58 mlops 51 ai-gateway 46 llm-gateway 19 llm-observability 22 llm-evaluation 33 llm-inference 60 gateway 11

Misurato dai temi di GitHub condivisi da entrambi i progetti, ponderato in base a quanto è raro ciascun tema.