Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Bundled copies of every page on the SQLBuild documentation site, matching the in
- [Seeds](concepts/seeds.md) (`concepts/seeds`) - Load static CSV data into your pipeline as tables.
- [Incremental Models](concepts/incremental.md) (`concepts/incremental`) - Cursor-based incremental strategies, microbatch execution, and backfill policies.
- [Snapshots (SCD Type 2)](concepts/snapshots.md) (`concepts/snapshots`) - Preserve row history over time using SCD Type 2 semantics with timestamp or check-based change detection.
- [Maintained Models](concepts/models/maintained.md) (`concepts/models/maintained`) - Let the warehouse keep a model up to date on a schedule, paused safely while builds run.
- [Functions](concepts/functions.md) (`concepts/functions`) - User-defined functions and table functions managed as part of your project.
- [Macros](concepts/macros.md) (`concepts/macros`) - Reuse SQL with typed SQL macros, or generate it with Python macros when it needs logic.
- [SQL Macros](concepts/macros/sql-macros.md) (`concepts/macros/sql-macros`) - Write typed, logic-free SQL fragments that the compiler checks and sees inside.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ own option. `--version` works only directly after `sqb`, as in `sqb --version`:
| `--defer-sources-to` | Read managed source data from another target |
| `--plan-from` | Read warehouse state `live`, or from the target's `synced_state` checked against the warehouse first; overrides the target's `plan_from`. See [Planning from synced state](../concepts/project-configuration.md#planning-from-synced-state) |
| `--fail-fast` | Stop on first failure and skip remaining nodes |
| `--ignore-maintained-grace` | Take over [maintained models](../concepts/models/materializations.md#during-a-build) another build holds, without waiting for `[maintained] heartbeat_grace`. Needs `allow_ignore_grace = true`, else fails with `A719` |
| `--ignore-maintained-grace` | Take over [maintained models](../concepts/models/maintained.md#when-a-build-dies) another build holds, without waiting for `[maintained] heartbeat_grace`. Needs `allow_ignore_grace = true`, else fails with `A719` |
| `--concurrency` | Number of worker connections (default: 1) |
| `--test-timeout <duration>` | Time limit for each unit test the build runs, such as `30s`, or `disabled`; see [Time limits](../concepts/testing-reference.md#time-limits) |
| `--verbose`, `-v` | Show lifecycle SQL inline after each model |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ state table. Models are not affected: their table type follows the model's `tabl

## Maintained models

A [maintained model](../models/materializations.md#maintained) is a Snowflake dynamic
A [maintained model](../models/maintained.md) is a Snowflake dynamic
table. `materialized materialized_view` fails with `A014`; use `materialized maintained`.

The build role needs these grants:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
<!-- generated-by: sqlbuild skills -->

# Maintained Models

> Let the warehouse keep a model up to date on a schedule, paused safely while builds run.

Online: https://sqlbuild.com/docs/concepts/models/maintained/

A maintained model is one the warehouse keeps up to date on a schedule you set, between builds. A
build creates it, changes it when its definition changes, and otherwise leaves the refreshing to
the warehouse; on Snowflake it is a dynamic table.

```sql
MODEL (
description "One row per customer per order date",
materialized maintained,
refresh (lag 1h),
);

SELECT customer_id, CAST(ordered_at AS DATE) AS order_date, SUM(amount) AS revenue
FROM __ref("stg_orders")
GROUP BY customer_id, CAST(ordered_at AS DATE)
```

Only Snowflake supports maintained models. See [Snowflake](../adapters/snowflake.md#maintained-models)
for the grants and change tracking they need.

## Settings

| Field | Default | Description |
|-------|---------|-------------|
| `refresh (lag <duration>)` | none | How far behind its inputs the model may fall: a fixed duration such as `5m`, `1h` or `1d`, at least `1m` on Snowflake and no shorter than the lag of a maintained model it reads. Without it, the model refreshes when a maintained model that reads it refreshes |
| `refresh_mode` | `incremental` | `incremental`, `full`, `auto` or `adaptive`. Snowflake rejects `incremental` for a query it can't refresh incrementally; use `full` for that query |
| `refresh_warehouse` | the connection's warehouse | Warehouse the scheduled refreshes run on |
| `refresh_on_build` | `false` | Refresh the model in every build that selects it, not only on its schedule |

## What the plan shows

| Change | Plan | What the build does |
|--------|------|---------------------|
| First build, or the dynamic table is missing | `create` | Creates it and fills it |
| Query or `refresh_mode` changed | `recreate, full recompute` | Replaces it; every row is computed again |
| `refresh` lag or `refresh_warehouse` differs from the warehouse | `change refresh settings in place` | Alters the schedule; rows stay |
| Nothing | `maintained` | Leaves it alone |

A model the plan keeps can carry one more line:

- `recomputes in full: stg_orders is rebuilt`: the build replaces a model it reads, so its next refresh computes every row again.
- `paused: scheduled refreshes are suspended`: Snowflake lists it as suspended, so it is not refreshing.

## During a build

A build pauses the maintained models it changes, and those that read a model it rebuilds, directly
or through views, so they never refresh from half-built inputs. It resumes them when it ends,
whether it succeeds or fails.

- Dynamic tables outside the project that read a paused model pause too, until the build resumes it.
- A model someone suspended before the build stays suspended, and the build warns about it.
- A suspend made by someone else during the build is undone when the build resumes the model.
- A kept model refreshes during the build when a model built later in the same build reads it.
- `refresh_on_build` refreshes the model in every build that selects it.
- A target with `pause_maintained = true` leaves its maintained models suspended between builds; see [Targets](../project-configuration.md#targets).

## When a build dies

- **One build at a time.** A build that needs a maintained model another build holds fails with
`A718`, naming that run. Wait for it to finish.
- **A stopped build leaves its models suspended.** A build killed before it resumes them, for
example by `SIGKILL`, can't resume them itself.
- **The next build resumes them.** The next build that pauses maintained models on the target
resumes them once the stopped build has been silent for `[maintained] heartbeat_grace`, 30
minutes by default; see [Maintained](../project-configuration.md#maintained).
- **Taking over at once.** Run `sqb build --ignore-maintained-grace` to take the models over
without waiting. It needs `[maintained] allow_ignore_grace = true`; without it the flag fails
with `A719`. You can also resume the models in Snowflake by hand.

## Limitations

- Writing out the default `refresh_mode incremental` recreates the model once.
- A snapshot rebuilt in full loses change tracking; run `ALTER TABLE <snapshot> SET CHANGE_TRACKING = TRUE` before a maintained model reads it again.
- Unit tests and scenarios treat a maintained model as a table.
- On a target with `schema = "preserve"`, builds see each other's pauses only when their maintained models share a schema.
Original file line number Diff line number Diff line change
Expand Up @@ -86,75 +86,17 @@ See [Snapshots](../snapshots.md) for timestamp and check strategies, historical

## Maintained

The warehouse keeps a maintained model up to date on a schedule you set, between builds. On Snowflake it is a dynamic table; a build creates it, changes it when its definition changes, and otherwise leaves the refreshing to Snowflake.
The warehouse keeps the model up to date on a schedule you set, and builds pause it while they change its inputs:

```sql
MODEL (
description "One row per customer per order date",
materialized maintained,
refresh (lag 1h),
);
MODEL (description "One row per customer per order date", materialized maintained, refresh (lag 1h));

SELECT customer_id, CAST(ordered_at AS DATE) AS order_date, SUM(amount) AS revenue
FROM __ref("stg_orders")
GROUP BY customer_id, CAST(ordered_at AS DATE)
```

| Field | Default | Description |
|-------|---------|-------------|
| `refresh (lag <duration>)` | none | How far behind its inputs the model may fall: a fixed duration such as `5m`, `1h` or `1d`, at least `1m` on Snowflake and no shorter than the lag of a maintained model it reads. Without it, the model refreshes when a maintained model that reads it refreshes |
| `refresh_mode` | `incremental` | `incremental`, `full`, `auto` or `adaptive`. Snowflake rejects `incremental` for a query it can't refresh incrementally; use `full` for that query |
| `refresh_warehouse` | the connection's warehouse | Warehouse the scheduled refreshes run on |
| `refresh_on_build` | `false` | Refresh the model in every build that selects it, not only on its schedule |

Only Snowflake supports maintained models. See [Snowflake](../adapters/snowflake.md#maintained-models) for the grants and change tracking they need.

### What the plan shows

| Change | Plan | What the build does |
|--------|------|---------------------|
| First build, or the dynamic table is missing | `create` | Creates it and fills it |
| Query or `refresh_mode` changed | `recreate, full recompute` | Replaces it; every row is computed again |
| `refresh` lag or `refresh_warehouse` differs from the warehouse | `change refresh settings in place` | Alters the schedule; rows stay |
| Nothing | `maintained` | Leaves it alone |

A model the plan keeps can carry one more line:

- `recomputes in full: stg_orders is rebuilt`: the build replaces a model it reads, so its next refresh computes every row again.
- `paused: scheduled refreshes are suspended`: Snowflake lists it as suspended, so it is not refreshing.

### During a build

A build pauses the maintained models it changes, and those that read a model it rebuilds, directly or through views, so they never refresh from half-built inputs. It resumes them when it ends, whether it succeeds or fails.

- Dynamic tables outside the project that read a paused model pause too, until the build resumes it. One that someone had suspended before stays suspended.
- A model that was already suspended when the build started stays suspended, and the build warns about it.
- If someone suspends a model during the build, the build's resume undoes that suspend.
- A kept model is refreshed during the build when a model built later in the same build reads it, so that model reads current rows. `refresh_on_build` refreshes it in every build.
- A target with `pause_maintained = true` leaves its maintained models suspended between builds and refreshes them only in builds; see [Targets](../project-configuration.md#targets).

Two builds can't pause the same models at once. A build that needs a model another build holds fails with `A718`, naming that run. When two builds pause a model at the same moment, the one that recorded its pause first keeps it, and the other fails with `A718` without suspending or resuming anything. A build that stops without resuming, for example because its process was killed, leaves its models suspended. The next build that pauses maintained models on the target resumes them once that build has been silent for `[maintained] heartbeat_grace` (see [Maintained](../project-configuration.md#maintained)). Until then, resume them in Snowflake by hand, or run `sqb build --ignore-maintained-grace` where the project allows it.

### Error codes

| Code | When | Fix |
|------|------|-----|
| `A013` | `materialized maintained` on an adapter other than Snowflake | Use `table` or `incremental` on that adapter |
| `A014` | `materialized materialized_view` on Snowflake | Use `materialized maintained` |
| `A015` | A `refresh` lag under `1m` on Snowflake | Set a lag of at least `1m` |
| `A016` | A model with no `refresh` that no maintained model reads, so it would never refresh | Add `refresh (lag 1h)` or another lag |
| `A716` | The adapter can't build a maintained model | Use an adapter that supports them |
| `A717` | No warehouse to refresh with | Set `refresh_warehouse`, or a `warehouse` on the connection |
| `A718` | Another build holds the models this build needs | Wait for it to finish, or for its hold to pass `heartbeat_grace` |
| `A719` | `--ignore-maintained-grace` without `allow_ignore_grace = true` | Set `[maintained] allow_ignore_grace = true`, or wait for the grace to pass |

### Limitations

- Writing out the default `refresh_mode incremental` changes the model's definition, so the next build recreates it once.
- A custom materialization named `maintained` is not used; the built-in one takes the name.
- Unit tests and scenarios treat a maintained model as a table.
- On a target with `schema = "preserve"`, builds see each other's pauses only when their maintained models share a schema.
- A snapshot rebuilt in full is created without change tracking. Before a maintained model reads it again, run `ALTER TABLE <snapshot> SET CHANGE_TRACKING = TRUE`.
See [Maintained models](maintained.md) for its settings, what the plan shows and what happens when a build dies.

## Custom

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ A target is a named build context - the database and schema you build into, plus
| `concurrency` | Worker limit for commands run against this target, replacing `[settings] concurrency` (see below) |
| `ignore_audit_failures` | Whether builds on this target let audit failures through, replacing `[settings] ignore_audit_failures`; see [Ignoring audit failures](audits.md#ignoring-audit-failures) |
| `plan_from` | Where `sqb plan` and `sqb build` read warehouse state: `live` (default) or `synced_state`; see [Planning from synced state](#planning-from-synced-state) |
| `pause_maintained` | `true` keeps this target's [maintained models](models/materializations.md#maintained) suspended between builds; each build refreshes the ones it selects. Default `false` |
| `pause_maintained` | `true` keeps this target's [maintained models](models/maintained.md) suspended between builds; each build refreshes the ones it selects. Default `false` |

```toml
[connections.warehouse]
Expand Down Expand Up @@ -699,7 +699,7 @@ old_name_views = "30d"

## Maintained

How builds pause [maintained models](models/materializations.md#maintained) while they run:
How builds pause [maintained models](models/maintained.md) while they run:

```toml
[maintained]
Expand Down
2 changes: 1 addition & 1 deletion website/src/content/docs/docs/cli/build.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ own option. `--version` works only directly after `sqb`, as in `sqb --version`:
| `--defer-sources-to` | Read managed source data from another target |
| `--plan-from` | Read warehouse state `live`, or from the target's `synced_state` checked against the warehouse first; overrides the target's `plan_from`. See [Planning from synced state](/docs/concepts/project-configuration#planning-from-synced-state) |
| `--fail-fast` | Stop on first failure and skip remaining nodes |
| `--ignore-maintained-grace` | Take over [maintained models](/docs/concepts/models/materializations#during-a-build) another build holds, without waiting for `[maintained] heartbeat_grace`. Needs `allow_ignore_grace = true`, else fails with `A719` |
| `--ignore-maintained-grace` | Take over [maintained models](/docs/concepts/models/maintained#when-a-build-dies) another build holds, without waiting for `[maintained] heartbeat_grace`. Needs `allow_ignore_grace = true`, else fails with `A719` |
| `--concurrency` | Number of worker connections (default: 1) |
| `--test-timeout <duration>` | Time limit for each unit test the build runs, such as `30s`, or `disabled`; see [Time limits](/docs/concepts/testing-reference#time-limits) |
| `--verbose`, `-v` | Show lifecycle SQL inline after each model |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ state table. Models are not affected: their table type follows the model's `tabl

## Maintained models

A [maintained model](/docs/concepts/models/materializations#maintained) is a Snowflake dynamic
A [maintained model](/docs/concepts/models/maintained) is a Snowflake dynamic
table. `materialized materialized_view` fails with `A014`; use `materialized maintained`.

The build role needs these grants:
Expand Down
Loading
Loading