Evidence under attack

Can the record of an AI agent's decision be trusted by someone who does not trust the company that kept it? We attacked the records themselves, eleven ways, and compared four approaches under one scoring rule.

Seal passes 10 of 11. A signed log modelled on AWS CloudTrail passes 6, a plain log 1, no control 0. The one Seal fail, and the conditions behind the tenth pass, are below, stated plainly. This page reports the evidence; whether an agent can be stopped is the Arena results.

Results

"Not run": the cloud log's own verifier needs live AWS access, so the two-verifier test was not run for it, and is never counted as a pass. Counts are classes of attack, not a security score: classes differ in how hard and how serious they are.

The eleven attacks

• E1 Edit a record. Change one value in a record after it was issued.
• E2 Re-spell a record. Write the same signed content a different way so it still looks valid.
• E3 Forge a key. Sign a fake record with a key the issuer never published.
• E4 Issuer gone. Check offline after a key rotation; a record misdated around the rotation must fail.
• E5 Approval and use. Does the action match what the person approved, was it that approver, and was a single-use approval used twice?
• E6 Refusals. Edit a signed refusal into an approval.
• E7 Withheld records. The operator leaves a record out of what it hands over, or the system never writes one down.
• E8 Split view. Show two people two different histories.
• E9 Ordering. Reorder the history, or date a record earlier than it really happened.
• E10 Two verifiers agree. Two separately written verifiers give the same verdict on every item.
• E11 Ignored refusal. After a refusal, ask the system that carries out the action to do it anyway.

The four approaches: Seal (signed records, a published key list and a public log); a signed log modelled on AWS CloudTrail (hourly signed digests, built from AWS's documentation, in its most favourable configuration); a plain log (an ordinary decision log, unsigned); and no control.

What Seal fails, and why

• E7, a refusal never written. If the system never records a decision at all, and the action never reaches anyone who keeps their own log, nothing shows it is missing. No approach in this evaluation passes. The published methods we know of that catch it, such as signed acknowledgements kept by the sender, all need a second party that saw the request; that is the direction of our research. Two parts of E7 are now caught: a record the operator withholds (Seal signs an hourly checkpoint that counts every record, so a withheld one leaves a count that does not add up; the CloudTrail-style log catches this too), and an action that did run but was never recorded, when the system that carried it out keeps its own signed log. A refusal that never reached that system is still invisible.

The tenth pass, and its conditions

• E9, a backdated record: now a pass. Each record's claimed time is held to heads of the public log that an outside witness countersigned. A record logged more than an hour after the time written inside it fails. Backdating by less than that hour is not caught; the bound is the tolerance, stated in the result.
• It depends on the witness. The pass assumes an hourly witness with no gaps, stamped by a time authority Seal does not control. In this evaluation the witness is simulated with a signed statement; the live witness countersigns through Sigstore, which the published verifier does not yet check.
• Checkpoints are live. The production gateway has signed hourly checkpoints into the public log since 22 October 2026. They count action and approval-use records, not model-call receipts, and start from the hour they were switched on.

Seal moved from 8 to 9 of 11 on E5 on 21 October 2026, once it signed a record of each use of an approval, and from 9 to 10 on E9 on 23 October, once the verifiers that check checkpoints and witnessed heads were published.

Check it yourself

The verifiers are published, in Python and in TypeScript, and were written separately. The approval-and-use check is the chain profile; the checkpoint and witnessed-time checks are the complete and witnessed profiles, all three published as drafts in version 1.4.0.

pip install aqta-verify-receipt==1.4.0
npm install aqta-verify-receipt@1.4.0

aqta-verify-receipt bundle.json --profile chain --key <issuer key> --approver-keys <key>
aqta-verify-receipt bundle.json --profile complete --key <issuer key> --review-from <start> --review-to <end>
aqta-verify-receipt bundle.json --profile witnessed --key <issuer key> --witness-keys <key>

It answers PASS, FAIL or INCOMPLETE; missing evidence is never a pass. Try it on the published test bundles in attestation-spec, where both verifiers must return the same verdict on every one. In this evaluation they agreed on all 26,050 items.

The harness runs offline with no model calls, and every rerun gives the same bytes. It is not public yet; to rerun it now, write to hello@aqta.ai. Its result file has SHA-256 e69c9ef1bcd1fbb8f03abbe6392f3cd8b3d969d55020843c881cdfd79a8b295a.

Pre-registration

The attacks, the scoring rule and our predictions were written down before any result: on 21 October 2026, then the approval-and-use attacks again, split into five parts, before any of the new code, and on 22 October the withheld, never-written and backdated attacks again, before any of their code. Every scored result matched its prediction. All three registrations, with their dated deviations, are published unchanged: read the pre-registration.

Limits

• Run by Aqta and not yet rerun by anyone else. Every implementation here, including both Seal verifiers and the CloudTrail-style validator, was written by Aqta.
• The cloud approach is a model of CloudTrail's log file validation, built from AWS's documentation with a throwaway key. AWS's own tool was not run, and a real trail may differ in details we could not check offline.
• For the approval and the withheld, split and ordering attacks, Seal's records were signed by a simulated issuer in the production format with throwaway keys. The production gateway has signed use records since 22 October 2026.
• Who counts as an approver rests on the keys the reviewer supplies. The approval and use formats are published drafts and may change.