Picaroon runs hundreds of adversarial scenarios against your live support, booking, refund, or quoting agent — scores them against OWASP LLM Top 10 and the CSA Risk Rubric v2 — and re-runs the library on every prompt change, model swap, or knowledge-base ingest. You get the written finding before customers find the failure.
Refund eligibility override via "manager approval" framing
−$1,240 / event
Warranty policy stealth — agent cites an outdated clause path
policy drift
Prompt injection through CSV attachment metadata
logging only
We do not ship a benchmark. We score each adversarial scenario against the canonical LLM-risk frameworks AND a vertical-specific failure catalog that compounds with every retainer.
Severity (P0–P3) is set by the framework rubric, not by us — so the report on day 90 is comparable to the report on day 1.
Active lens: OWASP LLM Top 10
We sell the finding, not the harness. Margin and recurring trigger both come from the same place.
A few hundred adversarial scenarios run against your live support, booking, refund, or quoting agent. We deliver a written report ranking every failure by severity with the prompt or guardrail that caused it.
Two-week turnaround. Closes the conversation about whether your agent is safe to ship.
The same scenario library re-runs on every prompt change, model swap, or knowledge-base ingest. Monthly failure report tells you what your agent broke before customers find it.
Triggered by release cadence, not calendar. A one-and-done audit instinct is fought off every contract.
The vertical library compounds across clients: every retainer feeds the catalog, and the catalog becomes a benchmark no general-purpose eval tool can replicate. That is the moat.
An agent that approves refunds under "manager authorization" framing in a way that overrides the eligibility policy costs real dollars per event.
OWASP LLM-06 / CSA RB.14
A verbose retrieval chain lets the agent cite an outdated clause. Looks harmless. Becomes a chargeback when it ships.
Vertical catalog · retail
A multi-turn refund flow that hugs the customer eventually agrees to a partial credit outside the original SKU.
Vertical catalog · SaaS
Clients leave each cycle with a report that points to the prompt, guardrail, or knowledge-base ingest that caused each failure — not a dashboard. The finding is the product.
Skim these before a scoping call. They cover the ones that have closed deals and the ones that have stalled them.
Send a brief description of your agent (vertical, model, rough traffic) and we will reply with a scoping outline within two business days. The first audit sells itself, or it does not — we would rather find out in week one.
Vertical-wedge engagements ship in two weeks. Retainers fill in release cadence.