prompt-injection

Open-source security scanner for AI agents, MCP servers, and agent skills. It auto-discovers installed agent components and checks them for prompt injection, tool poisoning, secrets, malware payloads, and related risks.

An OWASP incubator project that protects AI agent memory from prompt injection, secret leakage, and tampering. It includes a Python library, policy-based controls, benchmarks, and integrations for agent frameworks like LangChain and AutoGen.

Free hands-on labs for prompt injection and LLM security training, with real models, a context trace view of assembled prompts, and a multi-level Gauntlet challenge.

Belay is an open-source, local-first security layer for AI coding agents. It blocks dangerous commands, secret leaks, prompt injection, and risky MCP tool calls, with human approval flows and support for multiple agents.

Open-source authorization engine for AI agents that adds confidence-based gating, human review, policy-as-code, and audit logging. The repo shows quickstart code, local demo steps, SDK installs, and self-hosting options.

NILScript is an open standard and CLI toolkit for letting AI agents act on real systems through a gated propose → approve → commit → rollback flow. The page shows a live playground, quickstart commands, and benchmark claims around unauthorized writes.

A hands-on kit for authorized red teaming of locally run LLMs, with jailbreak techniques, prompt-injection scenarios, streaming attack visualization, and support for OpenAI-compatible local model servers.

Open-source runtime guardrails for AI agents that scans for prompt-injection weaknesses and blocks unsafe tool calls at execution time with a decorator.

Open-source runtime guardrails for AI agents that intercept unsafe tool calls before they execute, aiming to block prompt-injection-driven exfiltration and other policy violations.

Open-source input scanner for AI agents that strips hidden instructions and scans text, files, images, PDFs, QR codes, audio, and video before they reach an agent.

A prompt-injection capture-the-flag arena for testing how AI agents behave under attack. Users can play challenges, create guarded agents, add optional guards and sandboxed tools, and track scores on a leaderboard.

An interactive CTF-style challenge where users try to break or exfiltrate data from sandboxed AI agents running in Declaw’s isolated runtime.

Open-source authorization engine for AI agents that gates actions by confidence, policy, and human review, with prompt-injection detection and full audit logging.

A blog post describing Annex, a local-AI security tool that encrypts selected files during agent sessions to keep sensitive data away from local AI tools. The post explains the threat model, encryption flow, and the stack used to build it.

A Blue41 case study on how a banking AI assistant could be abused through indirect prompt injection, and what mitigation layers help reduce the risk.

Jo is a secure programming language designed to help catch prompt injection and other unsafe behavior at compile time by enforcing explicit capability boundaries. The repository shows the language’s security model, code examples, installation flow, and current project status.

An ICML 2026 research project page arguing that prompt injection comes from how LLMs misread roles, with an extended writeup, examples, and links to the paper, code, arXiv, and BibTeX.

A Noma Security research post detailing a prompt-injection flaw in GitHub Agentic Workflows, with attack flow, proof-of-concept evidence, and security recommendations.

VAIBot is an AI governance product focused on agent security. This page explains its prompt-injection threat model and positions the product as a circuit breaker that gates tool calls and other egress actions before they execute.

An open-source experiment that adds a small zero-initialized overlay layer to a frozen GPT-2 so its behavior can be adjusted at inference time without retraining the base model.