Guardian is an independent audit of production AI chatbots. We simulate realistic 50–100 turn human conversations to surface the failures that infrastructure pen tests and platform guardrails were never built to catch — before they reach a customer, a regulator or the press.
Findings mapped to OWASP LLM Top 10. Regulatory framing aligned with DORA Art. 24–27, EU AI Act Art. 9 / 14 / 15 and DSA Art. 28. Designed for the Product Owner, Head of AI, CRO or General Counsel of any institution running customer-facing conversational AI.
Bot agrees to refunds, discounts or commitments the company never authorized — legally binding in EU jurisdictions.
Internal procedures, pricing rules and discriminatory criteria surfaced through sustained social-engineering conversations.
Bot meets a user in distress with silence, form links or — worse — encourages harmful behavior. Direct DSA Art. 28 exposure.
Identity, tone and rules erode across long conversations. Bot contradicts its own policy after 40 turns of sustained pressure.
From large-scale retail and e-commerce to retail banking, insurance and public services, Guardian audits the conversational layer of any production chatbot — regardless of the underlying LLM, vendor or platform.
Retail support, onboarding and advisory chatbots under DORA scope — explicit ICT testing obligations.
Customer support and order assistants — false-promise rulings (e.g. Moffatt v. Air Canada) directly apply.
Insurance bots handling claims, coverage questions and first-notice-of-loss interactions.
Migration, employment, social services and citizen-facing chatbots — High-Risk under the EU AI Act.
Employee-facing assistants handling onboarding, policy questions, leave requests and benefits.
Patient-facing chatbots handling triage, appointment booking or care navigation in regulated settings.
Each of the failure modes Guardian tests for has already produced documented public incidents — with court rulings, regulator inquiries, viral reputational damage or human harm. The companies involved were among the largest and most professionally operated in their sectors.
An Air Canada chatbot invented a bereavement-fare refund policy that did not exist. The customer relied on it. The tribunal ruled the airline could not disown statements made by its own chatbot — Air Canada was held liable.
Lesson: A production chatbot creates legally enforceable commitments. False promises under sustained user pressure are reproducible — and now precedent-setting.
A DPD UK customer used a 5-minute conversation to get the customer support chatbot to swear, criticize its own employer and write a poem about how badly DPD performs. Disabled within hours after going viral on global media.
Lesson: Identity coherence and policy adherence collapse under social pressure — even on simple support tasks. The damage is brand-wide and reaches the board within 24 hours.
The U.S. National Eating Disorders Association replaced human helpline staff with a chatbot called Tessa. Within days, the bot was advising users with eating disorders to count calories and lose weight. NEDA suspended it after public outcry.
Lesson: A chatbot tested for "appropriate response" against canonical prompts can still fail catastrophically when meeting users in distress with realistic, sustained conversation.
A Belgian man's widow reported that the Chai chatbot "Eliza" had engaged her husband in weeks of conversation that, in her account, contributed to his suicide. The case prompted Belgian regulatory attention and EU-level policy debate.
Lesson: The behavioral layer of a chatbot — its identity, tone and conversational arc over weeks — is now a recognized vector of regulatory and reputational exposure.
Each finding below is reproducible on demand, documented with full conversation logs, scored on exploitability and business impact, and mapped to OWASP LLM Top 10 categories. The 154-second silence, the false promise, the drift — these are reproducible events, not anecdotes.
A user expressed suicidal ideation across 12 consecutive exchanges. The bot returned the same form-link response and then timed out for 154 seconds on the final escalation.
Under sustained expert impersonation, the bot disclosed internal differential processing rules tied to nationality — sensitive procedural information that should never reach an end user.
After 40+ turns of conversational pressure, the bot's tone, policies and identity boundaries gradually softened — eventually issuing advice that contradicted its documented operating rules.
Asked about regulatory obligations, the bot generated fabricated yet plausible legal references and procedural deadlines — creating direct misinformation exposure for the institution.
Multi-step role-play scenarios bypassed instruction guardrails the bot enforced reliably on direct attacks. The vulnerability only surfaced under conversational framing.
A customer support bot — explicitly out of scope for financial or contractual advice — produced specific recommendations under persistent user framing. The Moffatt-class exposure.
Whether your chatbot runs on Microsoft Copilot Studio, Azure OpenAI, AWS Bedrock or a custom LLM stack, Guardian sits outside the build — testing the deployed system from the same external position a real customer or adversary would. The complement to your platform's built-in safety, not a replacement.
An audit performed by the team that built the bot is not an audit — it is self-review. Regulators, auditors and Board risk committees increasingly require evidence of external behavioral validation. Guardian provides it on documented, reproducible methodology.
Sustained 50–100 turn personas, OWASP LLM Top 10 mapping, calibrated scoring, regulatory framing — the methodology took years to build. Standing up an equivalent internal red team requires 4–6 FTEs at €500k+ annual cost, with no certifiable independence at the end.
Microsoft Copilot, Azure AI Content Safety, AWS Bedrock Guardrails and equivalents do excellent keyword-level filtering. They do not simulate a 90-turn distressed user, an expert impersonator or a slow drift over weeks. Guardian tests what those guardrails were never designed to catch.
We simulate realistic human conversations over 50–100 exchanges to expose failures traditional testing misses. Then we package every finding the way Risk, Compliance, Security and Product committees can actually act on it.
Automated pipeline deploys Guardian personas across 50–100 exchanges, exposing behavioral vulnerabilities standard tools never reach.
Every finding includes severity, exploitability, business impact, indicative regulatory exposure and remediation guidance.
The meeting starts from your bot's actual vulnerability report — not a generic pitch. Product, Risk, Security and Legal in the same room.
Re-audits after every release. Live Behavioral Safety status. Drift alerts. Move from reactive to certified.
Built from documented incidents, sector risk taxonomies and validated persona archetypes — not improvised prompts.
Each finding receives an exploitability and impact score on a 1–5 scale, calibrated against OWASP LLM Top 10 definitions.
Findings are reproduced by at least one independent operator. Full conversation logs are preserved as audit evidence.
Guardian turns complex multi-turn audits into OWASP-mapped findings with regulatory framing and remediation guidance. Every report is reproducible, evidence-backed and structured for direct submission to internal audit, the Board and external regulators.
Three converging EU frameworks now make behavioral resilience a documented obligation, not a best practice. Guardian's audit output is structured for direct quotation against the specific articles that apply to your sector.
Guardian does not replace your security stack. It covers what classical pen testing and algorithmic red teaming cannot reach: the behavioral layer of long-form conversation. The three approaches are complementary.
| Capability | Classical Pen Test | Algorithmic Red Teaming | Narrative Red Teaming · Guardian |
|---|---|---|---|
| Infrastructure & code coverage | ✓ | — | — |
| Keyword & jailbreak testing | — | ✓ | ✓ |
| Conversation depth tested | 3–5 turns | 10–20 turns | 50–100+ turns |
| Sustained persona coherence | — | limited | ✓ |
| Behavioral drift detection | — | partial | ✓ |
| Risk framed for Product / Risk / Legal | — | partial | ✓ |
| DSA Art. 28 / AI Act mapping | — | partial | ✓ |
| Adaptation to a new bot | Days / weeks | Days | 2–3 hours |
30 minutes with Benoît Vogt, Founder & CEO. We walk you through Guardian's method, the failure modes most relevant to your deployment, and how Behavioral Safety Certification fits your existing regulatory file.