AI EngineeringAI Quality Engineering & Evals
AI Red Team Sprint
A severity-ranked list of the ways someone can make the client’s live AI system leak data, take actions it should not, or say things it must not — each one reproduced with a working attack — plus a fix plan ordered by risk.
AI Red Team Sprint
$25K–$45K
- Duration: 2–3 weeks
- Tier: Build
- Format: Fixed scope
You leave withSeverity-ranked adversarial findings report with a remediation plan
Book a Free SessionSound familiar?
We shipped the assistant, and security signed off the app, but nobody has actually tried to break the AI part. A customer posted a screenshot of it saying something it never should have, and now the board is asking whether it could show one customer another customer’s data. Honestly, we don’t know — and our pentest firm said prompts were out of their scope.
What you leave with
Threat model
Format: Document
How we know it’s acceptableReviewed and accepted by the client’s security lead in week 1.
Findings report
Format: PDF and Word
How we know it’s acceptableEvery finding has reproduction steps a client engineer can follow alone, and a severity on the agreed scale.
Remediation plan
Format: Spreadsheet
How we know it’s acceptableEvery item has a severity, an effort estimate and a suggested owner; walked through at the readout.
Client-specific attack suite
Format: Test cases and scripts in the client’s repository
How we know it’s acceptableEvery finding has a case that reproduces it; a client engineer can re-run the suite without us.
Executive summary
Format: One page
How we know it’s acceptableReadable by a non-technical executive; approved by the buyer.
Readout session
Format: Live call, recorded
How we know it’s acceptableAttended by both the engineering and security owners.
How it runs
2–3 weeks
Week 1
Scope, rules of engagement, threat model
OutSigned rules of engagement, threat model, test plan.
Week 2
Attack execution, automated and manual
OutFindings log with a reproduction for each finding.
Week 3 (end of week 2 on the two-week version)
Severity triage, remediation plan, readout
OutFindings report, remediation plan, executive summary, attack suite merged into the client’s repository.
Is this right for you?
This is for you if
- Buyerthe person accountable for the security of the AI product — a CISO or head of security in a larger company, the CTO in a smaller one.
- Also has to say yesthe owner of the system under test (agrees the testing window and environment); whoever owns the hosting environment, who must give written authorisation to test; legal or compliance if any test touches customer or regulated data; and Finance.
This is the wrong answer if
- The system is not live or not stable in staging. There is nothing fixed to attack yet; the Agent Prototype Sprint or Eval Harness Sprint comes first.
- The client wants a compliance certificate (SOC 2, ISO 42001, an audit opinion). We are not an auditor, and a findings report will be read as one.
- The AI is a thin wrapper with no tools, no retrieval and no sensitive data. The attack surface is too small to justify the fee; the free Agent Reliability Scorecard covers it.
- The client will not give written authorisation or a production-like test environment. We do not test production systems informally.
- The real worry is the application or network, not the model. That is a conventional penetration test and we refer it out.
What’s included, and what isn’t
In scope
- one AI system or application
- one test environment
- direct and indirect prompt injection, data leakage, tool and permission misuse, jailbreaks and harmful output
- the findings report and remediation plan
Out of scope
- fixing the findings (that is a build sprint, priced separately)
- network, infrastructure or conventional application pentesting
- compliance certification or audit opinions
- load or denial-of-service testing
- social engineering of staff
- ongoing monitoring after the readout
Before we start
Before day one
- Written authorisation to test, signed by the system owner and the environment owner.
- Access to a test environment that matches production configuration (same prompts, tools and model), with representative but non-real data.
- The system prompt, tool definitions and architecture documentation.
- A named technical contact who can answer a question the same day.
- An agreed severity scale, and an agreed list of what is off limits.
- An incident contact we call the same day if we find something critical that is exploitable in production.
What we’ll need from your team
- A platform or AI engineer, about 4-6 hours a week, for access and questions about tools and prompts.
- The security lead, about two hours a week, to agree severity calls.
- The system owner, available within 24 hours if a critical finding needs action before the report.
Pricing
What we need to give you a quote
- An architecture sketch (models, tools, data sources).
- The list of actions the system can take, and which cannot be undone.
- Who the users are and whether it is multi-tenant.
- Which environment we can test against.
- Any regulatory constraints on the data (PII, health, financial).
- What is driving the timing — a launch, an audit or an incident.
After we hand over
What we measure
- Number of findings by severity if the agreement includes retest after fixes, then the attack success rate on our suite before and after fixes
- The share of critical and high findings closed at retest
What comes next
- Build sprint to fix the findings, where the client has no capacity to do it, to do it
- The Eval Harness Sprintto put the attack suite into CI
- Managed Services — Growthfor continuous evaluation.
- The AI Reliability Auditwhere ordinary reliability has not been measured yet.
Questions buyers ask us
A standard pentest tests the application and the infrastructure. It almost never tests what the model can be talked into doing, which is where prompt injection and data leakage live. Ask your pentest firm whether indirect prompt injection was in scope.
They handle the model in general. They cannot see your tools, your data or your system prompt, and that is where nearly all the serious findings come from.
Yes, and you should; we use them too. Scanners find the generic jailbreaks. The findings that matter come from someone reading your tools and data flows and chaining them together.
Fair. It tests the system as it is today. If you change models or prompts often, plan a retest before major releases, put the attack suite into CI with the Eval Harness Sprint, or run continuous evaluation under Managed Services — Growth.
We don’t need to be. We test a production-like copy with synthetic data, under written rules of engagement you sign before we start.