LLM Security / Guardrails
Tool | Category | Segment | Platform / Tool | Plan / License | Monthly Price USD | Pricing Model | Free Tier / OSS | Included Usage / Limits | Threat Coverage / Policies | Runtime Enforcement / Guardrails | Red Teaming / Evaluation | Integrations / Frameworks | Deployment / Hosting | Security / Privacy | Team / Governance | Best Fit | Main Limits / Caveats |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
No tagline | LLM Security / Guardrails | LLM security monitoring metrics | WhyLabs LangKit | Apache-2.0 | $0 software; observability platform optional | Open-source text metrics toolkit | ✓ | No software usage cap; optional WhyLabs/whylogs logging infrastructure separate | Text quality, relevance, sentiment, regex patterns, jailbreak similarity and prompt injection similarity signals | Metrics can feed monitors, alerts or app decisions; enforcement is application-managed | Metrics support drift/security monitoring and benchmark dashboards | whylogs, WhyLabs, notebooks, Python apps and LLM observability pipelines | Local Python toolkit or logged to observability stack | Can compute locally; logging destinations determine data retention and sharing | No standalone team governance in toolkit; WhyLabs platform adds monitoring governance separately | Monitoring prompt/response security signals in production and experiments | Last release in captured source is older; similarity metrics are signals rather than hard security guarantees |
No tagline | LLM Security / Guardrails | Cloud content safety API | Azure AI Content Safety | Azure AI service | Free tier then usage-based Standard | Text and image records; exact regional pricing via Azure calculator | ✓ | Free tier stops usage when transaction limit is reached; S tier text record is up to 1,000 Unicode code points | Harmful content detection for text and images, severity scoring, safety studio and review workflows | API can flag content before or after model calls; app decides blocking, review, or escalation | Can be used with Azure AI Studio / Foundry workflows; not a full adversarial scanner by itself | Azure AI services, Azure OpenAI, REST APIs, Studio and enterprise Azure stack | Azure cloud; disconnected container commitment tiers are documented for some deployments | Azure security, compliance and customer tenant controls apply | Azure resource, IAM, billing and content review governance | Teams already standardized on Azure AI that need content moderation and review operations | Prompt-injection and agent-tool security require additional controls; pricing page may need calculator for exact regional rates |
No tagline | LLM Security / Guardrails | Cloud provider guardrails | Amazon Bedrock Guardrails | AWS managed service | $0.15 per 1k text units for content filters / denied topics; other filters vary | Usage-based per enabled guardrail policy and text unit | No durable free tier captured | Content/denied topics $0.15 per 1k text units; sensitive info and grounding $0.10; automated reasoning $0.17; regex sensitive filters and word filters are free | Content filters, denied topics, word filters, PII/sensitive information filters, contextual grounding checks, prompt attacks and automated reasoning checks | Guardrails can evaluate prompts and responses and can be applied to Bedrock, self-hosted and third-party model flows via ApplyGuardrail | No built-in red-team suite, but guardrail evaluations support safety/privacy control testing | Amazon Bedrock models, Agents, Knowledge Bases, ApplyGuardrail API and AWS SDKs | AWS managed service | AWS account/IAM, regional Bedrock service controls and selected model/data policies apply | IAM, account guardrails, AWS billing, service quotas and policy versions | AWS-native teams needing centralized safety controls across Bedrock applications | Charges accrue per configured policy; blocked response still incurs model inference cost up to the point of evaluation |
No tagline | LLM Security / Guardrails | Managed LLM firewall | Lakera Guard | Commercial SaaS / self-hosted | Start free; paid pricing not public | Managed guardrails with enterprise SaaS and self-hosted options | ✓ | Docs say get started for free; public page does not itemize monthly quotas or paid tiers | Prompt attacks, data leakage, PII, content violations, malicious links and custom detectors | Screens messages and reference content in real time, with policy thresholds from L1 to L4 and block/flag behavior | Threat intelligence and daily managed-guardrail updates; customer-specific adaptation and audit logs | REST API, OpenAI-style message payloads, SIEM/monitoring integrations and enterprise stacks | SaaS or self-hosted product | SOC2, EU GDPR and NIST claims on product pages; self-hosted option gives more data-path control | Web UI policy configuration, platform API, activity logs and enterprise support | Production GenAI apps needing strong prompt injection and data leakage defenses across languages | Pricing and quotas require sales/account confirmation; custom guardrails may be beta access |
No tagline | LLM Security / Guardrails | LLM firewall / eval engine | Arthur Shield / Arthur Engine | Commercial plus open-source eval engine | $0 Free; $60/mo Premium; Enterprise custom | Plan-based AI monitoring and safety platform | ✓ | Pricing page lists Free with monitoring for up to 4 use cases and unlimited seats; Premium is $60/month for up to 100 use cases | PII leakage, sensitive data leakage, toxicity, hallucinations, prompt injection and custom rules | Shield validates prompts and responses through two endpoints and can block or flag rule failures | Arthur Evals Engine and platform metrics support quality and safety evaluation | Any LLM, cloud providers, APIs and Arthur platform components | SaaS, managed cloud and on-prem options | Enterprise includes dedicated/managed VPC options, SSO, SLAs and BAA options | Unlimited seats on Free, enterprise SSO/SLA/support; use-case limits by plan | Organizations wanting LLM firewall behavior plus eval/monitoring in one vendor stack | Product naming has evolved from Shield to Engine; exact Shield feature availability should be confirmed for the target plan |
No tagline | LLM Security / Guardrails | Managed AI guard API | Pangea AI Guard | CrowdStrike/Pangea service | Free account path; paid pricing not captured | API service with recipes/detectors | ✓ | Quickstart starts with a free Pangea account; public docs do not itemize AI Guard production quotas in the captured source | Prompt injection, malicious content, PII/confidential data, secrets and other AI traffic risks through recipes | API detects, blocks or redacts content before it reaches or leaves the model | Recipes and logs can be used to validate policy behavior; not a full adversarial scanner | Pangea SDKs/APIs, recipes, dashboards and LLM app middleware | Pangea cloud service | Pangea account security and service data handling apply | Console recipes, audit/logging and project governance | Developers wanting a security API for prompt guard, PII and malicious content screening | Exact production pricing and limits need account/sales verification; recipe quality must be tuned per use case |
No tagline | LLM Security / Guardrails | Validation framework | Guardrails AI | Apache-2.0 / open source | $0 software; model and validator hosting costs separate | OSS validation framework with optional remote/hosted inference paths | ✓ | No software cap; validators may run local models, remote validators or provider LLM calls | Validators for PII, toxicity, provenance, summaries, schemas, custom policies and many Guardrails Hub checks | Guards validate LLM inputs/outputs and can apply on-fail policies such as exception, fix, reask or filter | Validators and metadata support targeted evals; not a broad vulnerability scanner by default | Python, JavaScript, OpenAI, Anthropic, LangChain, LlamaIndex and Guardrails Hub | Runs in app code or server mode; remote validators can be hosted separately | Self-hosting controls validator path; remote inference and third-party validators have separate terms | OSS governance through code/config; hosted options require separate review | Validation-heavy LLM pipelines where retry, reask and structured checks are central | Validator selection can add latency and model downloads; public hosted pricing was not encoded without current confirmation |
No tagline | LLM Security / Guardrails | Configurable guardrails SDK | OpenAI Guardrails Python | MIT / preview OSS package | $0 software; paid OpenAI API calls may apply | Open-source Python package wrapping OpenAI clients with configured checks | ✓ | No software usage cap; guardrail checks such as moderation can call OpenAI or third-party services and may incur charges | Built-in checks include moderation, PII, URL filtering, hallucination detection, jailbreak, NSFW text, off-topic prompts and custom prompt checks | Drop-in GuardrailsOpenAI client runs checks across input, output and pre-flight stages with tripwire handling | Includes an evaluation tool for labeled datasets, benchmarks, ROC curves and latency comparisons | OpenAI Python client, Responses API, Chat Completions, OpenAI Agents SDK, Presidio and custom checks | Runs in Python application code; configured through files or wizard | Developers are responsible for sensitive content storage and third-party service terms; API calls follow provider policies | Configuration files and app controls; no standalone SaaS governance unless paired with OpenAI org controls | OpenAI-centric Python apps needing quick guardrail wiring without replacing the application stack | Preview package; some checks rely on model calls or third-party services, so latency and cost need testing |
No tagline | LLM Security / Guardrails | LLM interaction security toolkit | Protect AI LLM Guard | Open source | $0 software; model/runtime costs separate | Open-source scanners for LLM prompts and outputs | ✓ | No software usage cap; advanced scanners can require extra model dependencies | Prompt injection, secrets, PII/anonymization, toxicity, code, language, regex, token limit, invisible text and many other scanners | Input and output scanners sanitize, validate and risk-score prompt/response content before use | Scanner outputs and risk scores support application-level tests; not a full red-team orchestrator | Python package, API deployment examples, OpenAI/ChatGPT examples and custom app middleware | Local application code or self-hosted API | Can run locally; external model or scanner dependencies change data handling | No SaaS control plane in OSS; app owns policy and logs | Developers wanting a practical scanner toolkit for prompt injection, leakage and moderation | Scanner coverage and false positives need tuning; some advanced functionality adds dependencies and latency |
No tagline | LLM Security / Guardrails | Programmable guardrail framework | NVIDIA NeMo Guardrails | Open-source Python package | $0 software; model/NIM/infrastructure costs separate | OSS library; production microservice is part of NVIDIA NeMo platform | ✓ | No software usage cap for library; microservice/container platform and NIM usage may have separate enterprise costs | Input, output, retrieval, dialog and execution rails; content safety, jailbreak detection, topic control, PII handling and agentic security | Colang/YAML rails intercept prompts, responses, retrieval and tool execution; microservice provides OpenAI-compatible inference endpoints | Detailed logging/tracing and safety models support testing; not a red-team generator by itself | NVIDIA NIM, OpenAI, Azure, Anthropic, Hugging Face, LangChain, LangGraph and custom providers | Self-managed Python library or Kubernetes microservice with Helm | Data path depends on selected LLM/provider and deployment; security guidelines emphasize isolating auth and validating tools | Portable configs, app-level governance and enterprise platform controls for microservice deployment | Teams building programmable guardrails around agents, tools and RAG flows | Can add latency and complexity; microservice and NVIDIA platform economics need separate validation |
No tagline | LLM Security / Guardrails | AI gateway guardrails | Cloudflare AI Gateway Guardrails / AI Security for Apps | Cloudflare service | Plan/add-on dependent | Gateway/WAF security controls around AI traffic | Guardrails available through Cloudflare AI products; AI Security for Apps is a paid add-on in WAF docs | AI Gateway guardrails intercept prompts and model responses; WAF AI detection fields vary by plan and paid add-on availability | Harmful content moderation, prompt/response guardrails, PII detection, prompt injection scoring, unsafe topics and custom topics depending product path | Proxy layer can flag or block AI traffic across providers | Gateway logs and WAF detections support audit and compliance workflows | OpenAI, Anthropic, DeepSeek and other provider traffic routed through Cloudflare Gateway/WAF | Cloudflare edge network and WAF / AI Gateway | Cloudflare account, edge proxy and logging policies apply | Cloudflare dashboard, WAF rules, logs and plan governance | Teams wanting model-agnostic edge controls for public AI endpoints | Exact capabilities and pricing vary across AI Gateway, WAF and paid add-ons; verify plan fit before production |
No tagline | LLM Security / Guardrails | Provider moderation API | OpenAI Moderation API | API feature | $0 for endpoint; model API calls separate | Free safety endpoint for OpenAI API users | ✓ | Moderation endpoint is documented as free to use; broader OpenAI API usage and rate limits still apply | Text and image moderation categories through omni-moderation-latest; legacy text-moderation-latest remains older text-only path | Returns category flags and scores so the app can block, route, review, or log unsafe content | OpenAI safety best practices recommend adversarial testing and human review; endpoint itself is not a full red-team suite | OpenAI SDKs, REST API, custom applications, policy workflows | Hosted OpenAI API | OpenAI API data handling and organization controls apply | Org, project, key, usage and policy governance in OpenAI platform | Apps needing a no-extra-cost baseline content moderation layer around OpenAI usage | Focuses on policy/content moderation, not prompt injection, PII redaction, tool authorization, or custom business rules |
No tagline | LLM Security / Guardrails | Agent framework guardrails | OpenAI Agents SDK Guardrails | Open-source SDK feature | $0 software; model token pricing applies | Guardrail hooks included in the Agents SDK | ✓ | No separate guardrail software cap; underlying agent runs and check models are billed by selected providers | User input checks, final output checks and tool guardrails for delegated workflows | Tripwires can stop runs; input guardrails can run in parallel or before model/tool execution | Tracing exposes guardrail results; custom guardrail functions support app-specific checks | OpenAI Agents SDK Python/JS, tools, handoffs, tracing and custom functions | Runs in application code | Data path depends on guardrail implementation and selected model/provider | Framework-level controls; broader org governance comes from application and provider setup | Agent apps that need checks around first input, final output and function-tool calls | Agent-level input guardrails do not run at every workflow hop; tool guardrails are needed for per-tool enforcement |
No tagline | LLM Security / Guardrails | Provider safety filters | Vertex AI Gemini Safety Filters | API feature | No separate feature fee captured; Gemini/Vertex usage applies | Built-in configurable filters with Gemini API calls | Included with model usage | Configurable thresholds for harm categories; finish reasons expose SAFETY, SPII, PROHIBITED_CONTENT and other block causes | Unsafe content filters, non-configurable CSAM/PII filters, citation/recitation filters and configurable harm thresholds | Model responses can be blocked or scored based on thresholds; apps can also use returned safety metadata | Useful for safety testing but not a standalone red-team or external firewall | Gemini API in Vertex AI, Google Cloud console and SDKs | Vertex AI managed service | Google Cloud data handling and Vertex AI project controls apply | Project IAM, model access, logs, region and policy configuration | Gemini applications needing built-in safety thresholds without another gateway | Some defaults vary by model version; BLOCK_NONE is restricted and configurable filters are not versioned independently |
No tagline | LLM Security / Guardrails | Cloud model firewall | Google Cloud Model Armor | Google Cloud service | Token-based; public simple price not itemized on overview | Standalone or Security Command Center integrated pricing | No public free tier captured | Prompt injection / jailbreak and responsible AI filters have 10k token limits; Sensitive Data Protection can process up to 130k tokens; text and files up to 4 MB | Prompt injection, jailbreak, responsible AI harms, sensitive data protection and malicious URL detection | Templates screen prompts and responses with inspect-only or inspect-and-block enforcement | Cloud Logging, templates and audit trails support validation; not a broad red-team generator | Google Cloud, Security Command Center, Sensitive Data Protection, Vertex AI and API workflows | Google Cloud managed service with regional processing options | Stateless processing; content is discarded unless customer logging is configured; TLS and regional data residency controls documented | Templates, Cloud IAM, Cloud Logging and Security Command Center governance | Google Cloud teams needing provider-level runtime filtering around prompts, responses and documents | Pricing requires Google Cloud/SCC pricing review; filter limits can skip or block depending over-limit behavior |
No tagline | LLM Security / Guardrails | AI gateway guardrails | Portkey Guardrails | Open source gateway plus SaaS plans | $0 OSS or Developer; Production $49/mo | Open-source self-hosting or recorded-log/request SaaS tiers | ✓ | Open source has no request limit; Developer is free with 10k requests/month and deterministic guardrails; Production includes 100k requests/month and LLM/partner guardrails | Regex, JSON Schema, code detection, prompt injection, moderation, partner guardrails and custom webhooks depending plan | Input and output guardrails run on the gateway with pass/fail verdicts, denial, retry, fallback, logging or dataset actions | Logs expose guardrail results; Enterprise adds advanced evaluation templates and centralized dashboard | Universal API gateway, OpenAI-compatible APIs, many providers, partner guardrails, webhooks and SDKs | Self-hosted gateway, Portkey cloud or enterprise deployment | Enterprise includes compliance, data isolation, VPC/private deployment options and custom BAAs | RBAC, SSO, service accounts, org-level guardrails and retention by plan | Teams already using an AI gateway and wanting runtime policy orchestration | Developer plan is not suitable for production; streaming output guardrails have limitations |
No tagline | LLM Security / Guardrails | Generative AI red-team framework | Microsoft PyRIT | MIT | $0 software; model/provider costs separate | Open-source framework for risk identification and red teaming | ✓ | No software usage cap; target and scorer model usage may be billed by configured providers | Jailbreaks, multi-turn attacks, prompt targets, scorers and datasets for generative AI risk identification | Primarily testing/orchestration, not runtime enforcement | Attack orchestration, scoring, memory, datasets, notebooks and custom scenarios | Azure OpenAI, OpenAI, local/custom targets and Python workflows | Local or customer-managed environment | Data path depends on target and scorer providers; local targets can keep data private | No hosted team governance in OSS; enterprise process controls are user-owned | Security teams running structured AI red-team campaigns and repeatable tests | Requires red-team expertise and careful scorer setup; not a drop-in production firewall |
No tagline | LLM Security / Guardrails | LLM red-team framework | DeepTeam | Apache-2.0 | $0 software; judge/model costs separate | Open-source framework with optional Confident AI platform integration | ✓ | No local software cap; FAQ says DeepTeam can be used purely locally, but attack/evaluation models may require API keys | Prompt injection, jailbreaks, PII leakage, bias, toxicity, SQL injection, misinformation, excessive agency and 40+ vulnerabilities | Includes production guardrails such as PromptInjectionGuard, ToxicityGuard and PrivacyGuard | Red-team framework maps to OWASP, NIST, MITRE and other safety/security frameworks | Python, DeepEval, Confident AI, custom model callbacks and provider models | Local Python framework; optional Confident AI platform | Local runs can avoid platform upload; provider judge/generator models may receive test data | Confident AI enterprise adds SSO, custom deployment and compliance; OSS is code-level governance | Python teams wanting combined red-team simulation and lightweight runtime guards | Requires model callbacks and judge models; results need calibration for target system and risk tolerance |
No tagline | LLM Security / Guardrails | Open AI security gateway | OpenGuardrails | Open source | $0 software; hosting/model costs separate | Open-source AI security gateway | ✓ | No software usage cap captured; enterprise/private deployment options should be verified in repository docs | PII cross-border transfer, sensitive data leakage, non-compliant content, prompt injection, adversarial attacks and policy violations | OpenAI-compatible gateway applies guardrails, multi-tenant configs and policy-based routing to each LLM call | Gateway logs/reports support security review; not primarily a red-team generator | OpenAI-compatible endpoint, model providers, enterprise gateway patterns and policy configs | Self-hosted gateway or private deployment pattern | Designed for private deployment; data handling depends on host and configured providers | Multi-tenant configs and org policies; exact RBAC/SSO maturity needs validation | Organizations wanting an open AI security gateway rather than per-app guard code | Newer project; license, maturity and production support should be reviewed before adoption |
No tagline | LLM Security / Guardrails | Agent security firewall | Meta LlamaFirewall | Open source / component licenses vary | $0 software; model/API costs separate | Open-source guardrail system for secure AI agents | ✓ | No software usage cap; required guard models may download from Hugging Face and alignment checks can require Together API | Prompt injection, agent misalignment and insecure code risks through Prompt Guard, Agent Alignment and Code Shield scanners | Scans messages or full conversation traces and returns allow/block decisions, reasons and scores | scan_replay can analyze conversation traces; examples cover integrations and demos | Python, OpenAI Agents SDK, Hugging Face guard models, Together API for some scanners | Local/customer-managed runtime | Can run locally for several scanners; external APIs and model downloads affect data path | No hosted governance; policy and trace handling are app-owned | Agent builders needing a runtime firewall around prompts, traces and code-producing agents | Some scanners require external models/API keys; project maturity and component licenses need review |
No tagline | LLM Security / Guardrails | AI security testing platform | Giskard | Open-source library plus commercial Hub | $0 OSS; Enterprise custom | Free local library; enterprise continuous red teaming and collaboration | ✓ | Free plan includes open-source library, local deployment and basic LLM vulnerability scan; Enterprise adds 50+ adversarial probes and collaboration | OWASP LLM Top 10, harmful content, reputation, legal/financial risk, misguidance, RAG quality and agent-specific vulnerabilities | Primarily scans/evaluates; remediation and custom guardrail consulting are enterprise services | Automated adversarial probes, risk reports, datasets and business-failure testing | Python library, Giskard Hub SDK/UI, RAG/agent testing workflows and CI/CD on enterprise | Local OSS; SaaS, private cloud or on-prem enterprise options | Enterprise lists data residency/isolation, 0-training policy, SOC2, HIPAA and GDPR | Enterprise SSO, RBAC, audit trails, versioning, alerting and SLA | Security and quality teams needing continuous LLM/agent vulnerability assessment | OSS vulnerability database is older/basic compared with Hub; scans can be token-intensive |
No tagline | LLM Security / Guardrails | PII de-identification SDK | Microsoft Presidio | MIT | $0 software; NLP/runtime costs separate | Open-source PII detection, masking and anonymization framework | ✓ | No software usage cap; can run via Python, Docker, Kubernetes or PySpark workloads | Names, locations, credit cards, SSNs, phone numbers, financial data, PHI-like entities, custom recognizers and image redaction | Can redact or anonymize sensitive content before prompts are sent and after outputs are returned | Recognizer scores and custom pipelines support privacy test cases; not an LLM red-team suite | Python, PySpark, Docker, Kubernetes, NLP models, custom recognizers and image redactor | Local/customer-managed infrastructure | Can run fully local; README warns automated detection does not guarantee all sensitive data will be found | Governance through code, recognizer configs and deployment controls | Teams needing a strong PII layer inside a broader LLM guardrail stack | Does not detect prompt injection or policy violations by itself; false negatives require defense-in-depth |
No tagline | LLM Security / Guardrails | Agent workflow security scanner | Agentic Radar | Open source | $0 software; optional LLM costs separate | Open-source scanner for agentic workflows | ✓ | No software usage cap; optional prompt hardening and runtime tests can require OpenAI or Azure OpenAI API keys | Tool identification, MCP server detection, vulnerability mapping, prompt injection, PII leakage, harmful content and fake news tests | Primarily scan/test; prompt hardening suggests better system prompts but does not enforce runtime policy itself | Static workflow visualization plus runtime testing for selected frameworks | LangGraph, CrewAI, n8n, OpenAI Agents and Autogen support matrix | Local CLI; generated HTML reports | Static scan runs locally; optional LLM features can send prompts to configured providers | No hosted team governance in OSS; CI artifacts and reports are user-managed | Developers securing multi-agent workflows and MCP/tool surfaces | Runtime testing currently supports fewer frameworks than static scanning; optional LLM features need keys |
No tagline | LLM Security / Guardrails | GenAI security toolkit | Cisco AI Defense / Robust Intelligence | Commercial enterprise product | Custom / contact sales | Enterprise AI security platform | No public free tier captured | Public product pages emphasize enterprise deployment rather than developer free quota | AI application discovery, model/application risk, red teaming, guardrails, prompt injection and data leakage controls depending product module | Runtime protection and policy enforcement are positioned for enterprise AI applications | Automated testing and red-team style validation are part of the AI security platform story | Enterprise security stacks, application workflows and Cisco security ecosystem | Enterprise SaaS / customer environment options need vendor confirmation | Enterprise security/compliance positioning; exact data path depends on deployment | Enterprise dashboards, policies, security team workflows and support | Security organizations standardizing AI risk management across many apps | Pricing, exact free tier and module packaging require vendor confirmation; less suitable for quick OSS experiments |
No tagline | LLM Security / Guardrails | Open safeguards and models | Meta Purple Llama Safeguards | Mixed: Llama Community licenses and MIT components | $0 software/model weights; hosting costs separate | Open safeguards and benchmarks for responsible generative AI | ✓ | No platform fee; model access and inference hosting are separate | Llama Guard moderation, Prompt Guard prompt injection/jailbreak detection, Code Shield insecure code filtering and CyberSec Eval benchmarks | Safeguard models and tools can filter inputs, outputs and code at inference time | CyberSec Eval suites measure insecure code, malicious compliance, prompt injection and cyber capabilities | Hugging Face models, Llama reference ecosystem, Python tools and custom deployments | Local or customer-hosted inference and tooling | Data stays local when models run locally; model licenses and acceptable-use terms apply | No hosted team governance; app teams own policy, logging and review | Open-model teams wanting first-party Meta safeguard models and cyber benchmarks | Licenses differ by component; safeguard quality depends on model version, language and deployment tuning |
No tagline | LLM Security / Guardrails | LLM vulnerability scanner | NVIDIA garak | Apache-2.0 | $0 software; target model costs separate | Open-source command-line vulnerability scanner | ✓ | No software usage cap; generator/API targets may incur provider costs | Hallucination, data leakage, prompt injection, misinformation, toxicity, jailbreaks and many other probes | Finds weaknesses; enforcement must be implemented with other guardrails or app controls | Static, dynamic and adaptive probes with detectors and reports | Hugging Face, Replicate, OpenAI, AWS Bedrock, LiteLLM, REST, llama.cpp/GGUF and many model families | Local CLI on Linux/macOS or CI environment | Data sent to scanned target and configured providers; local targets keep traffic local | No SaaS governance; reports and policies are user-managed | Model/application security assessment before deployment or after model changes | Scanner can be noisy and target-specific; probe results need expert interpretation and authorized testing |
No tagline | LLM Security / Guardrails | Red-team and eval CLI | Promptfoo | Open-source plus commercial | $0 Community; Enterprise custom | Community local testing with probe limits; paid enterprise/on-prem | ✓ | Community includes evals, vulnerability scanning and red teaming up to 10k probes/month at no charge | 50+ vulnerability types including jailbreaks, injections, RAG poisoning, OWASP/NIST/EU compliance and custom policies | Enterprise adaptive guardrails can turn red-team findings into filters; Community focuses on local red-team/eval | Automated red team scans, dynamic probes, reports, risk scoring, CI/CD and eval matrices | CLI, JavaScript/Python/custom targets, HTTP APIs, browser, RAG, agents and many providers | Local/self-hosted Community; cloud and on-premise enterprise options | Local scanner can test private endpoints; managed inference for red-team generation has Promptfoo privacy implications | Enterprise adds team sharing, dashboard, SSO, roles, scan history and support | Developer teams wanting test-driven LLM security and CI-friendly red teaming | Probe limit applies to open-source red teaming; advanced detection and dashboards require Enterprise |