A rule-import experiment shows limits of passive agent instructions
The current author report explicitly says scoring was not blind and run order was not randomized.
James Coombs’s current report describes nine ablation runs and 82 validation runs examining whether a coding agent imports mandatory rules. The current methodology says scoring was not blind and run order was not randomized. The original article incorrectly claimed both protections.
The current table reports average rubric scores out of 30: 26.7 with a prototype URL, 28.7 with a PDF screenshot, 16.4 with governance rules and 16.3 without guidance. These are not compliance percentages. The author also identifies a dark-mode scoring defect that awarded unearned points in some runs. Rule-import counts are a separate measurement. Without an archived version of the original source, this correction does not assert which values appeared on the source page on August 16.
Zero rule imports describes one governance ablation, not a combined zero-import result over all 91 runs. This distinction matters: compliance, attempted retrieval and import events are different observations.
The useful takeaway is to validate whether required rules are actually retrieved and enforced for a specific workflow. This small, openly qualified author experiment does not establish the population behavior of every coding agent.
Editorial correction (2026-10-13 UTC): Corrected the current source’s nonblind, nonrandomized method and denominators.