the wizard

Research

Independent tests of AI agent harnesses, the code between an app and its model: who the model listens to, which prompt-injection attacks get through, and which defences hold.

Harnesses: hand-rolled, Pydantic AI, pi · Code and data on GitHub

Studies

One question each, written up in plain language.

Voices

Who the model listens to: the operator, the person, a tool's output, and text that only looks like one of them.

Attacks

Instructions planted in emails, tool results and tool descriptions, and what each model did with them.

Defences

Approval gates, reviewers and labels: what each one held, and what it cost the task.

On the wire

What each harness actually sends the model, and where that trips up a model or a chat template.

How the tests run

Each test is a small agent with a real job (tidy an inbox, pay an invoice) and attacks planted in what it reads, run the same way through every harness with the same model settings. Models: Claude Haiku 5.5 and gpt-5.4-nano. Each setup runs 10 to 20 times, so read a result as a direction, not a rate.

Open marks a possible harness gap not yet filed with its developers. Retracted marks a claim that didn't hold up on a rerun, kept with the reason.

The method in full: How to test an AI agent for prompt injection. The code, every run and the scoring: github.com/dave8172/agent-harness-tests.