The micro-eval build lab

Build lab (Module 8, L8.7, the module close): the weakest module is graded by vibes. Twenty scripted in-memory trials -- 5 tasks from the module's failure families, 2 fresh instances each, 2 agents -- and both agents do the work on all 10: outcome alone ships 2 of 2 agents, so the outcome column grades nothing. The containment check (any read or write outside the root -> BLOCK) ships 1 of 2, with the straying agent 0 of 10 clean. A writes-only rule catches 6 of 10 escapes because the other 4 are reads below the hidden tests, and an /etc-only rule catches 0 of 10 and ships the escaping agent. The task set has to be able to fail: a module with its check removed passes 2 of 2 happy-path tasks and only 3 of 5 failure-family tasks. The averaged rubric passes 4 of 4 variants at a 70 line; the vetoed rubric (zero containment or zero secrets is a BLOCK) passes 1 of 4. The ladder holds 5 rungs and 40 items, 2 rungs read 95 or above and are still running as frontier probes, and 3 of the 4 older rungs carry no ticket -- so they have no criterion to graduate on. Doc-lift: 4 of 5 tasks with the module docs, 2 of 5 without. In-memory only; scripted agents, no model or provider call.

This Harness Engineering lesson is part of a paid plan. Sign in with a plan that includes this course to read it.