Gate evidence pack — G1–G5

Assembled 2026-08-19 at engine 8e0af26. This pack leads with what it does NOT prove, on purpose: the recurring failure mode across this build was a selected number becoming the headline, and every instance was caught only by looking. A pack that names its limits first cannot be read as overclaiming — and it is the version that survives a bank's security review rather than merely passing ours. Numbers reproduce from the committed gold + the local corpus (mirrored to the private bucket).

Bounds and honest limitations (read first)

• G3 real-event volume is DEFERRED to real-customer data (project decision, 2026-08-19). 438 real delegation events against a 2,000 target; the normalized-dataset target (≥10,000) is met at 18,450. The 2,000 was always a proxy for corpus richness bought from ourselves; the flywheel (customer-exported bundles, never background telemetry) supplies real volume once a design partner runs the loop. "Deferred to real-customer data", not "not met". The 25%-non-code G3 amendment is parked with it.
• Customer-domain enforcement is now shown on three frameworks (A3); customer-domain observe-sampling is ADK-only. customer-service + financial-advisor were observed/scored on ADK; CrewAI and LangGraph are additionally enforced live on a customer-domain (travel-booking) workload — real delegation, a held payments.transfer denied live, offline-verifiable, no divergence (docs/A3-FRAMEWORKS.md). Bound that remains: those two are configured crews/graphs, not scraped third-party apps (crewAI-examples target an old API; LangGraph apps are heterogeneous) — a real third-party customer-domain app on a non-ADK framework is a design-partner activity. G1/G5 derivation-quality numbers remain ADK-observed.
• The G1 hold-out over-provision metric rests on 13 clean rows. The express hold-out has 43 rows but only 13 survive the truncated/degenerate exclusion; the unused-scope and over-provision figures are computed on those 13. Small sample.
• The finance-advisory pack was curated from the very traces it is scored on. financial-advisor shows 100% curated because we wrote its pack from its own tool set — that is onboarding, not a blind test. The honest out-of-sample curation number is customer-service (0% → 100% after curation) and the day-0 coverage on datasets the catalog never saw (§G5, and the BFCL/hermes/ToolACE out-of-sample numbers).
• No live payment denial. Enforce is proven live single-agent and across a chain, but the specific "a booking agent's process_payment is denied live" run was not done (21-agent app, low marginal value). The deriver holding process_payment until --grant payments.transfer is pinned offline (tests/test_onboarding.py), not shown live.
• L3 (the LLM proposer) was never built. Its trigger — a domain whose day-0 path cannot produce a confident curated grant — was never met: on the two customer domains measured, L2 (catalog + curation) resolved everything. The architectural finding (templates are a code-agent artifact; the catalog generalises) is settled; L3 stays deferred until a real gap appears.
• All sampling is Haiku-class except the two live Sonnet enforce runs. Ceilings derived from Haiku over-exploration may be looser than a frontier model needs (a small frontier calibration slice is planned).
• The onboarding wall-clock was measured by the builder, not a naive operator. The ≤1h / ~minutes figure (G4/G5) inherits expert bias: I knew the vocabulary and the tools. A genuinely naive onboarding — the true G4 number — is a design-partner activity that solo work cannot manufacture. The judgement points are real; the time is a floor, not a representative user's.

Summary

G1 — derivation works

Hold-out (project-level, express held out; 13 clean rows), chain scoring: benign-deny 0.0 · unused-scope 5.8% · over-provision 3 · escalation 0 (thresholds ≤2% / ≤20% / esc 0). python -m attenu_derive.eval.g1 --holdout express --check

G2 — protects real projects

• Enforce (offline, 21 projects incl. 2 ADK customer-domain): 0 benign blocks; 32 role-violation over-reaches correctly blocked. python -m attenu_derive.eval.enforce --all
• Adversarial over-reach: 8,783 / 8,783 = 100%, scope-class 100%, 0 misses. python -m attenu_derive.eval.adversarial --all
• Task-string injection: 43,128 poisoned variants, 0 widened, 0 escalations (24 correctly narrower). python -m attenu_derive.eval.injection --all
• Live, single agent (Haiku + Sonnet): a real customer-service agent denied mid-run on a held scope; denial contract observed; ledger anchored. docs/LIVE-ENFORCE.md.
• Live, delegation chain (Haiku): analyst {web.fetch} ⊂ coordinator, denied web.search mid-chain; ledger anchored across the chain. docs/LIVE-ENFORCE.md (chain section).
• Structural, not model-dependent: Haiku ≡ Sonnet — enforcement does not inherit the model's judgment. (Bound: both are ADK.)

G3 — data flywheel

• Real corpus: 438 delegation events / 599 rows / 21 projects × 4 frameworks. The ≥2,000 real-event target is DEFERRED to real-customer data (project decision, 2026-08-19): 2,000 bought from ourselves was a proxy; real volume arrives via the flywheel — customer-initiated evidence-bundle export, never background telemetry (the custody story is the thing a bank buys). The 25%-non-code amendment is parked with it.
• Normalized datasets: 18,450 rows (BFCL + hermes + ToolACE; target ≥ 10,000 — MET).
• Versioned corpus + eval harness gate every release in CI (G1 + adversarial + injection gates + corpus lint). MET.

G4 — production grade

• Onboard in ≤ 1 hour: MET — travel-concierge to 100% curated in minutes following docs/ONBOARDING.md.
• Threat model, denial contract, strike policy, ledger anchoring (ADR-14): filed and tested.
• Packaging (Phase A1): installable wheel + attenu CLI (onboard/coverage/verify) — installs into a clean env with no source tree; onboarding re-measured through the packaged path at ~1s (docs/OPS-RUNBOOK.md).
• Internal security review vs the 5 threat-model invariants: all pass, 2 trust-boundary concerns with mitigations + Phase-B/C actions, labelled internal (docs/SECURITY-REVIEW.md). External review is Phase C.
• Perf: Deriver.propose p50 0.038 ms / p95 0.074 ms / max 0.178 ms over real events (budget <50 ms); model path N/A (L3 never built).
• Ledger contract for the console (2026-08-19, slice 1 / Plan A): held ≠ denied is now ON the ledger (disposition on every deny, in the model-facing denial, across all 12 adapters); products have an identity before a key (attenu init → .attenu/product.json, per-process boot id, assigned chain ids); the spool sink carries the ingest idempotency key; inside a product the anchor is signed with a product-local Ed25519 key and verified with the public key only (attenu verify --pubkey) — the HMAC test signer survives only outside a product and is labelled attenu-anchor-TEST. Repro: shim python tests/test_core_v02.py, python tests/test_sinks_identity.py, python tests/test_adapters_contract.py; engine python -m pytest -q tests/test_disposition.py tests/test_product.py tests/test_run_adk_enforce.py tests/test_cli.py.
• Delegation-graph UI and service mode live in the optional Attenu console (https://attenu.io): the graph with denials by disposition and the verified mark (integrity · monotonicity · containment against the product's public key), evidence download, and Decisions. The open engine needs none of it; attenu demo produces a real anchored ledger with no model and no key.
• Control with integrity (2026-08-19, slice 2): every change to what a product may do (grants, declared tools, policy) is a signed config revision — hashed, parent-chained, verified before a runner applies it, refused beyond the product's ceiling and on conflict (last-known-good stays); the cloud's revisions are signed by the Attenu signer the engine trusts offline; attenu config / attenu ceiling; the console shows "what I decided and what changed" with diffs and signatures. Custody options: product-local Ed25519 key (default) or KMS (attenu init --anchor kms; ES256; the key never leaves the HSM; auditors verify with ECDSAP256Verifier, no cloud SDK) — stub-tested until an AWS account exists. Out-of-band anchoring (AnchorScheduler) for long-running apps. Evidence report (attenu report, the console's Report button): a printable page rendered from the bundle + verification — the three checks, the anchor key, the chain as tree + table, denials in plain words, how to re-verify. Demo depth: attenu demo --scenario fanout (9 agents / 18 tools / every disposition / a strike revocation) and the real 21-agent travel-concierge run live as a 3-level chain. Two real bugs found by the richer app and fixed RED→GREEN (ADK transfer-back treated as a delegation; the enforce runner's sub-agent requests lacking their own descendants). G4: what remains for "production grade" is a hosted deployment and the slice-3 surface.

G5 — day-0 story

A brand-new app held out of all training (travel-concierge) gets safe derivation from the shipped kit alone: all payment tools withheld, unknowns fail-closed, reads granted heuristically — shadow-ready with zero unintended-payment risk. Curation then takes it to 100% with money tools held pending an operator grant. docs/ONBOARDING.md; tests/test_onboarding.py. Out-of-sample catalog coverage on public datasets the catalog never saw is the corroborating number (BFCL/hermes/ToolACE, catalog.coverage).

Operating cost (measured, A2c): across the 3 onboarded apps, 23 tools, 7 (30%) required an operator judgement call, 70% mechanical scaffold confirmations. The judgement calls are front-loaded on the tier-2 tools a bank wants a human deciding (payments, mail held requires_grant), so the curation burden scales with an app's distinct sensitive tools, not its traffic, and day-0 is safe before any of it. Pinned ≤50% by a test (tests/test_onboarding.py).

Reproduce everything

python -m pytest -q runs the full suite including the injection + adversarial CI gates and the corpus lint (schema-drift guard). python -m attenu_derive.corpus.lint → 0 violations.