ai-safety

31 projects share this GitHub topic

ai-safety — iFixAi ★46.3kai-safetyAI-Infra-Guard — ★7.6kagent-governance-toolkit — ★6.6kPyRIT — ★5kawesome-machine-learning-interpretability — ★4kFormal-Verification-RSI — ★2.9ksafe-rlhf — ★1.6kcc-safety-net — ★1.5kawesome-gpt-6-astra — ★1.2kkeystone-bench — ★1.1kawesome-rsi — ★1.1kAgentLens — ★1kTracekit — ★734langtest — ★561box — ★557PromptInject — ★528floe-guard — ★410tiger — ★403martin-loop — ★394prismor — ★347Aegis — ★343orbit — ★310wrench-board — ★267AgentGuard — ★160orloj — ★123guardian-sdk — ★121awesome-llm-agent-papers — ★114cerbia — ★52llm-firewall — ★47TeaLeaves — ★44proofagent-harness — ★41AI-Infra-Guard★ 7.6kagent-governance-toolkit★ 6.6kPyRIT★ 5kawesome-machine-learning…★ 4kFormal-Verification-RSI★ 2.9ksafe-rlhf★ 1.6kcc-safety-net★ 1.5kawesome-gpt-6-astra★ 1.2kkeystone-bench★ 1.1kawesome-rsi★ 1.1kAgentLens★ 1kTracekit★ 734langtest★ 561box★ 557PromptInject★ 528floe-guard★ 410tiger★ 403martin-loop★ 394prismor★ 347Aegis★ 343orbit★ 310wrench-board★ 267AgentGuard★ 160orloj★ 123guardian-sdk★ 121awesome-llm-agent-papers★ 114cerbia★ 52llm-firewall★ 47TeaLeaves★ 44proofagent-harness★ 41

Lines connect members that are measurably related to each other. Dot size reflects stars.

🧬 Members
iFixAi
Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in…
★ 46.3k
AI-Infra-Guard
A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra…
★ 7.6k
agent-governance-toolkit
AI Agent Governance Toolkit — Policy enforcement, zero-trust identity, execution sandboxing, and…
★ 6.6k
PyRIT
Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive…
★ 5k
awesome-machine-learning-interpretability
A curated list of awesome responsible machine learning resources.
★ 4k
Formal-Verification-RSI
Formal verification for AI: a Lean 4 + Dafny proof checker that catches fake AI proofs (reward hacking), and…
★ 2.9k
safe-rlhf
Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback
★ 1.6k
cc-safety-net
A pre-execution guard for AI coding agents. It blocks destructive Git and file system commands, plus common…
★ 1.5k
awesome-gpt-6-astra
Evidence-backed use cases, prompts, integrations, evaluations, and safety notes for OpenAI GPT-6 Astra.
★ 1.2k
keystone-bench
A clinical decision-evidence benchmark: pull the one fact an answer rests on, and see whether the assistant's…
★ 1.1k
awesome-rsi
A curated research map of Recursive Self-Improvement (RSI): models, agents, harnesses, embodied systems,…
★ 1.1k
AgentLens
Agentlens is a trusted agent trading platform. Here, you can quickly find the Agent that meets your needs,…
★ 1k
Tracekit
Tamper-evident tracing, policy gates and cross-checks for AI agents: see what the agent did, and why.
★ 734
langtest
Deliver safe & effective language models
★ 561
box
Run AI agents in a sandbox that restricts what they can execute, read, write, and reach on the network. Box…
★ 557
PromptInject
PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of…
★ 528
floe-guard
The spend meter, cost/margin, & budget gate for AI voice agents. Meters STT + TTS + LLM + telephony per call,…
★ 410
tiger
Open Source LLM toolkit to build trustworthy LLM applications. TigerArmor (AI safety), TigerRAG (embedding,…
★ 403
martin-loop
Run coding agents without babysitting them. Keep jobs focused, bounded, checked and accountable from start to…
★ 394
prismor
Self-hosted runtime control plane for AI agents. Observe or HITL approve or Block rogue tool calls before it…
★ 347
Aegis
Runtime policy enforcement for AI agents. Cryptographic audit trail, human-in-the-loop approvals, kill…
★ 343
orbit
Self-hosted, OpenAI-compatible AI gateway for private RAG, natural-language data access, and tool-calling…
★ 310
wrench-board
A senior microsoldering technician, available to every repair shop from the seasoned pro to the apprentice.…
★ 267
AgentGuard
AgentGuard: Zero-Trust Security Foundation for AI Agents
★ 160
orloj
An orchestration runtime for multi-agent AI systems. Declare agents, tools, and policies as YAML; Orloj…
★ 123
guardian-sdk
Ethicore Engine® is an AI safety, ethics, and compliance platform. This repo consists of the open-source…
★ 121
awesome-llm-agent-papers
LLM agents: curated papers on planning, memory, tool use, multi-agent systems, evaluation, and safety.
★ 114
cerbia
A modular threat detection library for AI and LLM pipelines. It provides language-agnostic guardrails through…
★ 52
llm-firewall
AI Firewall & LLM security toolkit - protect your AI applications from prompt injection, jailbreaks, PII…
★ 47
TeaLeaves
End-to-end pipeline for seeing how LLMs actually process your prompts. Capture attention across every layer,…
★ 44
proofagent-harness
Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios…
★ 41
🔗 Related families

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.