fix: address verified gadfly P5 findings (canary robustness)
executus CI / test (pull_request) Failing after 3s
executus CI / test (push) Failing after 1m9s

All 3 cloud models converged (all "minor" — example code, no blocking):
- Consolidate: a model whose every lens errored now reads "review incomplete",
  not a misleading "no issues found" (all 3 models). + test.
- Consolidate: swarm-cancelled (unattributed) cells now surface a "swarm
  cancelled — N cell(s) did not run" banner instead of vanishing (all 3). + test.
- main: io.ReadAll(os.Stdin) error is surfaced (all 3); a TTY stdin no longer
  hangs forever (TTY guard, minimax).
- providerOf: a bare tier name now keys its own PerKey bucket instead of all
  bare tiers collapsing onto "tier" (minimax, glm-5.2) — distinct tiers throttle
  independently.
- Review doc reworded (the closure, not fanout, carries per-cell errors).

Left as documented example-scope behavior: no per-cell timeout (caller supplies
ctx), unknown-severity → lowest rank (no crash).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-27 00:34:01 -04:00
parent ea9475da54
commit a87e7d2c72
3 changed files with 54 additions and 6 deletions
+26
View File
@@ -100,3 +100,29 @@ func TestConsolidateVerdicts(t *testing.T) {
t.Errorf("critical + errored lens header wrong:\n%s", got)
}
}
// TestConsolidateAllErrored: a model whose every lens errored must NOT be
// labelled "no issues found" (the gadfly P5 finding).
func TestConsolidateAllErrored(t *testing.T) {
got := Consolidate([]LensResult{
{Model: "m", Lens: "a", Err: context.DeadlineExceeded},
{Model: "m", Lens: "b", Err: context.DeadlineExceeded},
})
if !strings.Contains(got, "m — review incomplete") {
t.Errorf("all-errored model should be 'review incomplete', got:\n%s", got)
}
if strings.Contains(got, "no issues found") {
t.Errorf("all-errored model must not say 'no issues found':\n%s", got)
}
}
// TestConsolidateSwarmCancelled: dropped (unattributed) cells surface a banner.
func TestConsolidateSwarmCancelled(t *testing.T) {
got := Consolidate([]LensResult{
{Err: context.Canceled}, // dropped cell, no model
{Model: "m", Lens: "a", Findings: []Finding{{Severity: SevSmall, Title: "x"}}},
})
if !strings.Contains(got, "swarm cancelled") {
t.Errorf("dropped cells should surface a cancellation banner:\n%s", got)
}
}