feat(concurrency): provider-wide lens budget, drop the model cap
Concurrency was two multiplicative gates in two processes: entrypoint.sh capped MODELS-at-once per provider (GADFLY_PROVIDER_CONCURRENCY) while each model's binary separately capped its own lenses (GADFLY_LENS_CONCURRENCY). A model therefore held its whole model-slot until its LAST lens finished, stalling the next model even with idle lens capacity. Collapse to one throttle: a provider-wide lens budget shared across all of that provider's models. entrypoint now runs every model in a lane at once and seeds a single cross-process permit pool per lane (a dir of N flock files, sized by GADFLY_PROVIDER_LENS_CONCURRENCY -> GADFLY_LENS_CONCURRENCY). Each lens pass (review+recheck) acquires a permit before it runs and releases it after, so a model winding down immediately yields its freed permits to another model's queued lenses. flock auto-releases on process death, so a killed/crashed model can't leak budget. - cmd/gadfly/lenssem.go: the flock permit pool (+ lenssem_test.go). - main.go: runSpecialists holds a shared permit per lens; fanout sized to the budget so a lone model can use all of it. Falls back to the in-process limit when no pool is set (local runs, tests). - entrypoint.sh: drop provider_cap/DEFAULT_CONC; run_lane runs all models and seeds the per-lane pool. - GADFLY_PROVIDER_CONCURRENCY / GADFLY_CONCURRENCY are now ignored; the reusable workflow marks provider_concurrency deprecated and stops forwarding it. Docs (README, CLAUDE.md, examples) updated per the maintenance rule. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
@@ -55,14 +55,14 @@ jobs:
|
||||
# csv to choose; "all" for everything; or define custom ones via a repo
|
||||
# .gadfly.yml / GADFLY_SPECIALIST_<NAME>. See README "Specialists".
|
||||
GADFLY_SPECIALISTS: ${{ vars.GADFLY_SPECIALISTS }}
|
||||
# Lens fan-out (optional; default 1 = lenses run sequentially within a
|
||||
# model). Raise it to run a model's lenses concurrently so each model
|
||||
# posts its comment sooner. Total in-flight requests = (models at once)
|
||||
# × (lenses at once), so to fan out without oversubscribing a backend,
|
||||
# keep its model cap low and raise its lens cap. Per-provider configurable
|
||||
# via GADFLY_PROVIDER_LENS_CONCURRENCY (same lanes as the model map):
|
||||
# GADFLY_PROVIDER_CONCURRENCY: "ollama-cloud=1,m1=1"
|
||||
# Concurrency (optional; default 1 = fully sequential per provider). The
|
||||
# ONE throttle is a per-provider LENS BUDGET: the max lens passes (a lens =
|
||||
# one specialist's review+recheck) in flight at once for a provider, shared
|
||||
# across ALL that provider's models — every model in a lane runs at once and
|
||||
# its lenses draw from the shared budget. Raise it to overlap lenses; set it
|
||||
# per provider with GADFLY_PROVIDER_LENS_CONCURRENCY:
|
||||
# GADFLY_PROVIDER_LENS_CONCURRENCY: "ollama-cloud=3,m1=1"
|
||||
# (The old GADFLY_PROVIDER_CONCURRENCY model cap was removed and is ignored.)
|
||||
# GADFLY_LENS_CONCURRENCY: ${{ vars.GADFLY_LENS_CONCURRENCY }}
|
||||
# GADFLY_PROVIDER_LENS_CONCURRENCY: ${{ vars.GADFLY_PROVIDER_LENS_CONCURRENCY }}
|
||||
# Live status board (optional; ON by default): one consolidated comment
|
||||
|
||||
Reference in New Issue
Block a user