Repository navigation
feat!: remove the concurrency source label - #1607
Merged
Merged
Conversation
A build's concurrency is no longer described by the layer that set it:
since named jobs' limits combine with the target and project cap, one
source label no longer describes the limit a build uses.
- The verbose run context reads `concurrency 16 configured limit`.
RunContextFacts.concurrency_source and the config ConcurrencySource
are removed; EffectiveConcurrency keeps only its limit.
- A published job's resolved_concurrency is `{limit}`; its `source`
and the ConcurrencySource schema are removed from the API.
- Control store migration 0004 removes resolved_concurrency.source
from each stored job definition, so schedules, sensors and manual
runs read jobs published before the upgrade.
The job page's Concurrency panel reads "Inside a run: at most 4 of this
job's nodes at once" for a job, and "Members without their own limit:
at most 6 nodes at once" for a group, without naming the layer that set
it. A test reads it through the server's job read on the fake control
server. The TypeScript control client is regenerated without
ConcurrencySource, and the server upgrade docs say to let runs finish
or cancel them before upgrading, then restart every location.
The migration and pinned-document tests follow the scheduler test
layout, and load_project_setting_keys, whose only caller named the
project layer, is removed.
BREAKING CHANGE: a published job's `resolved_concurrency` is now
`{limit}`. Its `source` field and the `ConcurrencySource` schema are
removed from the control API, and `sqb build --verbose` no longer names
the setting the concurrency came from. Control store migration 0004
removes the field from each stored job definition. Let queued and
running runs finish, or cancel them, before upgrading: a group run or
retry pinned to a snapshot published by an earlier release fails with
`jobs: unknown field 'source'` and stops the scheduler. After upgrading
the server, restart every location so each publishes its project again.
…r upgrading Correction to 135016a: migration 0004 does not make every read of jobs published before the upgrade work. It removes resolved_concurrency.source from sqb_jobs, so job runs from schedules, sensors and manual requests work after it. The snapshot documents those jobs were published in keep the field, and 0004 can't rewrite them: until the location publishes again, its job pages, group runs and ad hoc runs fail with `jobs: unknown field 'source'`, and a group's schedule stops the scheduler. Runs pinned to a snapshot published before the upgrade fail the same way when they read it. The server upgrade docs now say to pause schedules and sensors, or keep the scheduler off, and to drain or cancel runs before upgrading, then to upgrade and restart every location after upgrading the server. The migration test stores the old form in sqb_jobs and the snapshot documents, and checks both what works after 0004 alone and that the rest works once the location publishes again. A new test shows a group's schedule failing the scheduler cycle until then. Comments no longer say the published concurrency names the layer that set it. BREAKING CHANGE: a published job's `resolved_concurrency` is now `{limit}`; its `source` field and the `ConcurrencySource` schema are removed from the control API, and `sqb build --verbose` no longer names the setting the concurrency came from. Before upgrading, pause schedules and sensors (or set `scheduler = false` under `[server]`) and let queued and running runs finish, or cancel them. Control store migration 0004 removes the field from each stored job definition. After upgrading the server, upgrade and restart every location: until a location has published again, its job pages, group runs and ad hoc runs fail with `jobs: unknown field 'source'`, and a group's schedule stops the scheduler.
…in runs
The server upgrade note no longer offers `scheduler = false`: the
scheduler loop launches queued runs and advances group runs, so with it
off nothing drains. It now gives the order to upgrade in and when to
resume schedules and sensors, and says that until a location has
published again a group's schedule or sensor stops the scheduler and a
job run that fails can't record its failure.
A test shows that failure: the worker's report that a job run failed is
refused, because deciding on a retry reads the snapshot's documents
before it checks for a retry policy, and the run stays running.
BREAKING CHANGE: a published job's `resolved_concurrency` is now
`{limit}`; its `source` field and the `ConcurrencySource` schema are
removed from the control API, and `sqb build --verbose` no longer names
the setting the concurrency came from. Control store migration 0004
removes the field from each stored job definition, but not from the
snapshot documents jobs were published in. Upgrade in this order:
1. Pause every schedule and sensor, and start no manual runs. Keep the
scheduler running.
2. Let queued and running runs finish, or cancel them.
3. Upgrade the server.
4. Upgrade and restart every location.
5. Once every location has published again, resume schedules and
sensors.
Until a location has published again, its job pages, group runs and ad
hoc runs fail, a group's schedule or sensor stops the scheduler, and a
job run that fails can't record its failure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A build's limit now combines the target or project cap with named jobs' limits, so one "source" label no longer describes it. Show the effective numbers plainly instead.
Changes
sqb build --verbosereadsconcurrency 16 configured limit. The source is removed from the run context and the config resolution.resolved_concurrencyis{limit};sourceandConcurrencySourceleave the API and the TypeScript client.Breaking changes
The API field is removed. Upgrade in this order:
Until a location has published again, its job pages, group runs and ad hoc runs fail, a group's schedule or sensor stops the scheduler, and a job run that fails can't record its failure. The server upgrade docs give the same order.
Verification
Invalid, then work. A failed job's report is refused (SQLite).make check-ci.