Repository navigation
feat(state): record each run in every schema it writes to - #1600
Merged
Merged
Conversation
A run appended its started and finished rows to _sqlbuild_runs in the target's default schema, creating that schema when missing. A build that writes only to other schemas then needed rights on a schema it never builds in, and could create a schema solely to hold state. Each run now writes both rows, with the same run ID, to every schema it writes to: the schemas of the models and seeds it runs (plan action not skip), its SQL functions and the sources it loads; for `sqb seed`, its seeds' schemas. The set is fixed from the plan before the first node runs, so each schema gets the pair. The writes run concurrently. As before, the started row sets each schema's runs table up (creating the schema, which the build would create anyway) and the finished row inserts directly, setting the table up again only when an ordinary error stops that insert; an interrupt still stops the run. A failed write warns naming that schema's table and never fails the run; the other schemas still record it. A run whose nodes write no relation records nothing. Readers merge the rows by run ID.
…e default schema Review follow-up to recording runs per schema. One physical schema spelled two ways, such as marts and MARTS on Snowflake or with and without the target's database, was written twice. Schemas now compare as the dialect resolves unquoted names (its native profile) with an unnamed database filled in from the target's, so each runs table gets one pair. Each phase opened a connection per schema, and a second on a first run. It now opens at most four, each writing its share of schemas in turn, and the create-and-retry reuses its connection: one login per phase for one schema, four for five or twenty. A failed schema still warns alone. Every writer runs on its own thread, outside the calling thread's context, so a build stopped by its execution deadline still records its finished row in each schema. An interrupt raised in a writer, or a Ctrl+C while the writers run, still stops the run once they finish. A run that writes no relation, such as only Python tasks or checks, now records in the target's default schema, resolved as audit results resolve theirs, instead of nowhere. Runs that write a relation still write only their own schemas. Tests cover case spellings, a skipped (disabled) model's schema, a function's schema, a build that raises across two schemas, and a task-only run; the killed-build test selects its crashing task alone again. The docs no longer claim the sync merges runs yet.
…em on Ctrl+C Review follow-up to recording runs per schema. Each run-record writer thread started from an empty context, so its statements left the build's event and recall stream. Writers now take a copy of the caller's context and run with every deadline suspended, through the existing execution_deadlines_suspended scope, now a native host entry. A build stopped by its execution deadline still records its finished row in each schema, and its run-record statements are recalled with the build again. Ctrl+C no longer waits for every writer to finish its share. A caught signal sets a stop flag that each writer checks before its next table and before a retry, and cancels the running statements as SIGTERM does. A further signal while the writers stop cancels again instead of staying pending. Writes cut short that way are not reported as failed, and the interrupt escapes once the writers stop. run_recorder now takes the command's pipeline and picks the build's or seed's schemas itself, which keeps the Python bindings crate within its code-line cap. Tests send a real SIGINT while a run-record insert hangs, and check that the build stops within seconds, raises KeyboardInterrupt and warns of no failed write; and that a build's run-record inserts are among its own recalled statements.
Run records and janitor planning both read the active target's database or schema as a location, naming none for preserve. They now share native_host's target_location.
kvlonge
enabled auto-merge
November 19, 2026 17:20
Native run-record writers resolve execution_deadlines_suspended through the sqlbuild._native_host name table, so it is a dead-code root like the other host targets.
…schema # Conflicts: # tests/integration/src/sqlbuild/cli/commands/main/_test_types.py # tests/integration/src/sqlbuild/cli/commands/main/helpers.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A run appended its
_sqlbuild_runsrows to the target's default schema, creating it if missing, so a role that builds only in other schemas needed rights on a schema it never builds in. Separately, a build that hit its execution deadline lost its finished run record: the row was written under the expired deadline and refused.Changes
sqb seedwrites its started and finished rows, under one run ID, to every schema it writes: its models' and seeds' that run, its SQL functions' and its loaded sources'. Readers merge the rows by run ID.Verification
make check-ci; one review plus follow-up.