Judge economics: sampling, cascades, drift

Economics lab: L5.4 finished calibrating the judge, so this lesson prices the plan. One strong call per output on a million outputs a day is $120,000.00 a month at $0.004 — 3.0x the $40,000.00 quote — and 382 days of model time for one day of outputs. The cheap tier reads the reply field only, decides 54 of 63 rows and accuses 4 of 16 passes. Routing its 9 abstentions through the strong tier and a person makes the cascade 63 / 9 / 2 calls and $2.5612, where the two human calls are 97.6% of the bill; cutting the person off costs $0.0612 and leaves 2 rows undecided. Sixteen sampled calls put 2 on the cells the cheap tier cannot read when spent proportionally, 3 with one per cell, and 16 when spent where the tiers disagree. The judge's 19-probe suite is $0.0760 a run, and a bump that moves 2 of them is 10.5% of it. Standardising four weekly readings to one mix separates a judge that drifted (gold probes move), a system that regressed (every stratum falls), and a population that changed (the rates hold and the mix moves).

This Evals Engineering lesson is part of a paid plan. Sign in with a plan that includes this course to read it.