Skip to content

docs: design 001, database development flow - #677

Open
ditadi wants to merge 15 commits into
mainfrom
stack/database-dev-flow/00-design
Open

ditadi wants to merge 15 commits into
mainfrom
stack/database-dev-flow/00-design

Conversation

@ditadi

@ditadi ditadi commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🥞 Stacked PR (generated by Stack)

Use this link to review incremental changes.


What did you change, and why?

Design 001 (designs/001_DatabaseDevelopmentFlow.md) describes database branch ownership, committed migrations, and deployment. This revision updates the application development flow to match the named-operation design and experience fixes in #687–#692.

  • Replaces the current API description based on generated CRUD routes with authored, authorized operations. Table declarations alone expose no HTTP endpoint.
  • Documents the three route configurations: conventional database() discovery, routes: false, and an explicit custom registry. Covers committed registry/contracts, development reloads for shared schemas and policies, first/last route changes, and source-less production builds.
  • Defines omitted input as no parameters, with no placeholder object or inferred input. Explicit schemas determine caller input. Method-based output defaults remain unchanged; input shape never determines result cardinality.
  • Explains query/mutation hooks, typed skipToken, operation/session-scoped state, late responses, refresh after writes, and safe field feedback that preserves failed drafts.
  • Records the distinction between policy-resource snapshots and atomic business rules: transactional rechecks, constraints, database-generated IDs, and audit writes.
  • Updates the scaffold-to-deploy walkthrough, API changes, compatibility guidance and links to the owning operation PRs.

The migration decisions remain: opt-in startup migrations, per-migration transactions, bounded advisory/history locks, branch guards, ownership handoff, private-column preservation, deployed verification and offline history checks. The tooling remains internal at @databricks/appkit/beta/database-tooling, and revealing the hidden plugin/commands remains a separate change.

The document explicitly distinguishes the migration chain (#654–#678) from the operation chain (#687–#692). Combined startup/deployment validation is a release integration step; the separate chains' checks do not establish that integration. PR #677 targets main independently, and the implementation chain begins at #654 on main.

The linked Google Doc remains available for comments and edits. This PR updates the Markdown design copy.

Review round, 2026-11-09:

  • Closes three open questions with the decisions taken in review: the app's service principal owns the deploy branch's objects, with manual recovery, and a shared owner role is rejected for now (Q1); a branch init forks starts with no copied app rows, and --schema migrate opts in to keeping them (Q5); the stack ships on drizzle-kit 0.31's journal v7 (Q9). The remaining questions are renumbered.
  • Adds a runbook for a migration that fails on deploy every time, and separates how long a migration waits for a lock (lock_timeout) from how long it holds one (unbounded), with the procedure for a stuck start.
  • Narrows security claims: branch guards are not an authorization boundary, a baseline does not prove the security setup matches, verify --deployed proves the deployed source rather than applied migrations, clean leaves SP-owned tables in public and cascades into dependents, and init's drizzle-kit install runs lifecycle scripts. Names the gap of a local start against the deploy branch before appkit db init runs, and spells out the branch-naming rules.

Fix round for fjakobs's review of #677, 2026-11-15:

  • The design now matches the fixes for fjakobs's review. The operation sections are a short summary that links to design 002.
  • appkit db init decides from the branch itself instead of an .env marker. Migrate on start is an identity check: PGUSER must be set and equal DATABRICKS_CLIENT_ID.
  • The deploy section describes what Apps does per compute type, with the 300 s health check. The failure runbook repairs data and never edits a merged migration, so edited and a diverged history refuse a deploy start.
  • Ownership recovery is delete-role --reassign-owned-to after a full inventory, and long backfills run as a batched DML job between an expand and a contract migration.
  • Open questions answered with evidence: client_connection_check_interval is allowed, no permission is narrower than CAN MANAGE, schema-only branches are not in the public API, and the facts about Genie App Builder.
  • Introspect keeps the .private() the old schema.ts declares and writes every other column and foreign key as the catalog has it. The private-by-default and the foreign key refusal existed only for the AutoCRUD routes, which the operation stack removes.

How do you know it works?

Documentation only. Checked the changed contracts against the current #687–#692 implementation and plugin guidelines; validated the syntax of all three TypeScript/TSX examples, balanced fenced blocks, operation PR references, and git diff --check. The branch's pre-existing Knip findings (one unused CLI file and three dev dependencies) also occur with the unchanged design; they are unrelated to this Markdown-only edit. No runtime or app files are included.

Review round, 2026-11-09: each fix has a failing-then-passing test in its PR, and every layer's appkit and shared tests ran against Postgres 14 after the sync. At the top of the stack, pnpm build, pnpm test (6502 passed, 1 skipped), pnpm -r typecheck, pnpm check, pnpm knip and pnpm docs:build pass.

Fix round for fjakobs's review, 2026-11-15: each fix has a failing-then-passing test in its PR, and every layer's database, plugin and CLI tests ran against Postgres 17 after the sync. A throwaway Lakebase fork confirmed that the developer role can write the service principal's rows but cannot alter its tables. At the top of the stack, pnpm test (6602 passed, 1 skipped), pnpm -r typecheck, pnpm build and pnpm docs:build pass, and oxlint and oxfmt are clean on every changed file.

Signed-off-by: ditadi <victordperd@gmail.com>
… 001

Signed-off-by: ditadi <victordperd@gmail.com>
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 44716149603 -R databricks/appkit -n appkit-template-0.98.0-pr.afb7cbf-stack-database-dev-flow-00-design-677 -D appkit-pr-677 \
  && unzip -o "appkit-pr-677/appkit-template-0.98.0-pr.afb7cbf-stack-database-dev-flow-00-design-677.zip" -d "appkit-pr-677" \
  && databricks apps init --template "appkit-pr-677"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi
ditadi requested a review from a team as a code owner October 26, 2026 02:04
@ditadi
ditadi requested a review from MarioCadenas October 26, 2026 02:04
ditadi and others added 4 commits October 30, 2026 10:44
The design doc promised behavior the corrected stack does not ship. It
now describes what the code does:

- The tooling export is @databricks/appkit/beta/database-tooling, beta
  and internal to the AppKit CLI.
- migrate, introspect, clean and init hold config/database/.appkit-db.lock.
  init holds one lock for its whole run, and init --schema none, seed,
  verify and clean --delete-branch take none. The doc says how to
  recover a stale lock file.
- init --schema none hands off nothing, and schema public is never
  handed off.
- Values in variable-overrides.json count for the deploy boundary, and
  an unresolved branch or database reference refuses every write.
- Introspect keeps .private() on declared columns, writes new columns
  private, and refuses a schema.ts that does not load or a foreign key
  it does not declare.
- verify --deployed compares the committed migrations with the source
  snapshot of the active deployment, and fails when it cannot read it.
- Migrations log one line per run. There is no span and no metric.
- #675 keeps the plugin and appkit db hidden. A reveal is a later,
  separate change.

The appkit db check section stays as it is until #678 settles.

Signed-off-by: ditadi <victordperd@gmail.com>
…he stack code

Signed-off-by: ditadi <victordperd@gmail.com>
… and prompts

Apply the review of the design doc. The pooler refusal belongs to migrate on start, the migrator reconciles without the lock before it takes it, the deploy guard reads each target with and without variable-overrides.json, and init asks only for --schema clean and an ambiguous rename. The reveal also moves the database export into the generated barrel, check refuses a file that is not a drizzle-kit snapshot, and the project is not a databricks.yml variable.

Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi
ditadi requested a review from fjakobs November 1, 2026 23:27
Signed-off-by: ditadi <victordperd@gmail.com>
… database dev flow design

Record the review decisions: the app's service principal owns the deploy
branch's objects with manual recovery, a new fork starts without copied app
rows, and the stack ships on drizzle-kit 0.31's journal. Renumber the open
questions and point the doc at them by number.

Add a runbook for a migration that fails on deploy, separate lock
acquisition from lock holding, and give the procedure for a stuck start.
Narrow the claims about clean, branch guards, baselines and
verify --deployed, and describe the behavior this fix round adds across
the stack.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi
ditadi requested a review from atilafassina November 9, 2026 17:04

@fjakobs fjakobs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an AI-generated review (Claude Code). Treat each comment as a lead to check, not a verdict.

This PR only adds the design doc, so each inline comment marks a place where the doc contradicts itself, design 002 on main, or the code at the top of the two stacks: stack/database-stable/06-serve-operations for operations and stack/database-dev-flow/08-db-check for migrations. Comments marked "checked" are backed by reading that code; the four marked "from reasoning" come from how Postgres and the platform behave and weren't reproduced. Nothing was run: no CLI, no deploy.

Two comments point at code bugs, not just doc drift:

  • init.ts installs drizzle-kit only for the migrate and introspect starts, but clean is now the default for a new fork, so a new app's first appkit db migrate fails.
  • --schema none on a new fork skips the rebuild but still clears APPKIT_DB_NEW_BRANCH, so the copied production rows stay on the branch.

Smaller items without inline comments:

  • databricks apps run-local with exported M2M credentials migrates as an identity other than the app.
  • The journal-ownership rule as written would refuse the service principal's own journal in public.
  • The production registry claims ignore the Jiti source fallback.
  • The v0.87/v0.88 references are stale; main is at v0.91.
  • The "preserves existing beta startup behavior" heading is wrong: an upgrade without schemaName fails the startup check.
  • The operation sections repeat design 002 and have already drifted from it; linking to design 002 would avoid that.
  • The listNotes example has no limit, and getNote lets any caller read any note.
  • The pg_cancel_backend runbook doesn't name a role with the rights it needs.
  • The branch-limit message lists only your own branches, so a new developer has nothing to delete.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated

Output defaults stay independent of input:

| Method | Output when `output` is omitted |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (checked by reading the stack code): This table contradicts the operation runtime and design 002. With no output, noBody() in operation-runtime.ts (#692) throws INTERNAL for any result other than undefined: only undefined gives 204, null on GET/PATCH/PUT gives 404, and anything else is an opaque 500. The doc's own getNote example (lines 111-118) has no output and returns db.notes.find(id), so every found note would answer 500. A DELETE that returns true would also be a 500, not the 204 this table promises. "Output defaults stay" at line 657 has the same problem.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed against noBody (operation-runtime.ts:118-127). Without output, only undefined answers 204, null on GET, PATCH or PUT answers 404, and anything else is a 500. The table and "Output defaults stay" on line 657 predate explicit outputs. Both go with the duplicated section, in favor of the link to design 002. Fix in #677.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated
removes the journal and its merge conflicts, but moving to it changes the layout of every app's committed migrations.
That move is future work, in a format-migration proposal of its own.

`appkit db init` installs drizzle-kit as a dev dependency when it is missing, since only database apps run it. A

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (checked against init.ts on stack/database-dev-flow/08-db-check; this is a code bug too): init installs drizzle-kit only for the migrate and introspect starts (init.ts:703-705). This round made clean the default start for a newly forked branch, so the default path never installs it, and drizzle-kit is only an optional peer dependency of appkit. A new app with schema.ts runs appkit db init (start clean, no install), then appkit db migrate fails with "Generating migrations needs drizzle-kit, which is not installed", and so does appkit db check in CI. That breaks the six-step flow at lines 596-601.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and it's my bug from the last round. Fix in #672: init installs drizzle-kit for every start except none. appkit db check needs it too, because it diffs schema.ts with drizzle-kit. The install adds it to devDependencies, so CI gets it.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated
`.env` then points at the branch. When `databricks.yml` binds a database that the fork does not have, init refuses and
names the database and the branch, instead of pointing `.env` at `databricks_postgres`.

A Lakebase fork copies every row of its parent. A branch that init creates still starts with no copied app rows. In the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (checked against init.ts on stack/database-dev-flow/08-db-check; this is a code bug too): --schema none, and an explicit --schema introspect, on a new fork keep every copied production row, but this paragraph and the security limits (line 701) name only --schema migrate as the way to keep them. rebuild = found.fresh && options.schema === undefined, so any explicit --schema skips the rebuild, yet init.ts:752 still clears APPKIT_DB_NEW_BRANCH. A later plain appkit db init then treats the branch as reused and defaults to migrate, so the copied rows stay on the developer's branch indefinitely, with no warning.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed. Instead of patching the marker, #672 removes it. init now decides from the branch itself. If the app schema still belongs to the service principal, the branch is an untouched copy and init rebuilds it. If the schema is yours, init migrates. --schema none touches nothing, so the next plain init still rebuilds. An explicit --schema migrate or introspect keeps the copied rows because you asked for it.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated

export const listNotes = defineOperation({
name: "notes.list",
entity: "notes",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (checked by reading the stack code): OperationFields (#688-#692) has no entity property, so both defineOperation examples fail to typecheck ("Object literal may only specify known properties"), and line 92 lists entity as an operation field. In untyped files defineOperation silently drops it, which suggests operations are bound to an entity when they aren't.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. OperationFields has no entity (operations.ts:125-135), so both examples fail the typecheck. I'm replacing the operation sections of 001 with a short summary that links to design 002, so 001 stops restating that contract. Fix in #677.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated
| `DELETE` | Status 204 with no body when the callback returns `true`. |

A list, aggregate, nullable result or joined response declares its output schema. An `{ id }` input does not determine
whether the result is one row or an array. Private columns are omitted by default; explicit outputs project only the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (checked by reading the stack code): Operations have no default output, and .private() doesn't filter an operation's output, so "Private columns are omitted by default" doesn't hold. Line 558 says the same about introspect keeping .private() for default operation outputs. A developer who relies on .private() and then writes an output schema that names the column, or uses passthrough or z.record, sends the value to the browser; design 002 says this explicitly. The FK-refusal rationale at lines 560-561 ("adds a relation that authored operations can include") also looks left over from AutoCRUD.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. .private() does not filter an operation's output, because the Zod output schema is the projection, as design 002 says. I'm removing the claim and the "default operation outputs" wording on lines 558-561. I'll restate the foreign key refusal in terms of the server entity client, or drop the refusal if it no longer earns its place. Fix in #677.

The check compares the snapshots with each other and with `schema.ts`. It does not read the SQL, so a hand-edited
`.sql` is not checked against its snapshot.

A journal on `main` that passes the check and only grows can never strand an entry on the deploy branch, because the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (checked by reading the stack code): This assumes every deploy comes from main, but step 5 (line 607) has developers run databricks apps deploy from their own checkout. Say someone deploys an unmerged branch carrying 0002@T1, then main merges a different 0002@T2 and the feature is regenerated as 0003. The deploy branch now holds an ahead row plus DDL that main never reviewed: the regenerated 0003 fails with "already exists", or an older entry is stranded and every start fails. The doc gives no recovery for either.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed. The doc will state the precondition: deploy targets deploy commits from main, and a feature deployed early merges its migrations unchanged before main deploys again. #654 will refuse a start whose history has unrecorded committed entries and recorded rows the checkout lacks at the same time. That combination means the histories diverged; a rollback only produces the second. The error names both tags and the recovery.


### When a migration fails on deploy

Some migrations fail on the deploy branch every time, though they passed on the dev branch. For example, a `SET NOT

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (checked by reading the stack code): New dev branches are now rebuilt with no rows, so data-dependent migrations can't be tested on them, and the breaking-change warnings (lines 323-325) miss common cases such as ADD COLUMN ... NOT NULL without a default, ADD UNIQUE and CHECK constraints. Those apply cleanly on the empty dev branch with no warning and fail on the deploy branch every time, which makes this failure runbook the normal path. Old instances' INSERTs also break during the rollout.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly. ADD COLUMN ... NOT NULL without a default is already flagged (migrations.ts:283-298); the doc lagged. ADD UNIQUE, CHECK, foreign keys and unique indexes are not, and you're right that empty dev branches hide failures that depend on data. Fix: #666 adds a second warning class, "may fail on existing rows", separate from the rollout warning. The doc recommends testing such a migration on a session branch created with --schema migrate, and the failure runbook (L424) repairs data instead of editing the migration.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated
grants `CREATE` on the database. On a branch init created in this run, init drops the fork's app schema and rebuilds
it from the committed migrations, or from the baseline introspect writes, so no copied row stays in it. Limits that
the docs repeat:
- Until the rebuild runs, the copied rows are on the fork at the Lakebase level. An init interrupted before the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (checked by reading the stack code): This looks out of date. init writes APPKIT_DB_NEW_BRANCH before forking, so a plain rerun after an interruption still treats the branch as fresh and rebuilds it. It doesn't default to migrate, and --schema clean isn't needed; on a branch init treats as reused, --schema clean triggers the confirmation prompt the default path avoids. APPKIT_DB_NEW_BRANCH is also missing from the list of .env keys init writes (lines 663-666).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, and the marker is gone in #672. Before taking the schema over, init tags the copied journal with one row, and dropping the schema removes that row in the same transaction. A rerun from any checkout reads the tag and finishes the rebuild. appkit db migrate refuses a branch that still carries it. The design text will describe that rule instead of the .env key.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated

- The failed migration rolled back and is not recorded.
- The migrations before it in that start stay committed and recorded.
- The new instance does not serve. The old instances keep serving, because each migration works with the previous

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (from reasoning, not reproduced): "The old instances keep serving" doesn't hold for single-instance apps or for apps that scale to zero (line 55), so this isn't a no-outage state for them (see also lines 336-338). An app at zero instances that cold-starts after a deploy whose SET NOT NULL fails on deploy-branch data has its only instance exit 1, and it stays down until a fixed migration is redeployed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and the Apps runtime makes it worse. A deploy succeeds once the process has lived 2 s, unless an app.yaml health_check applies. Classic compute stops the old process before it starts the new one, and a crashed process is never restarted. The template already declares health_check on /health (#695), and AppKit binds /health only after migrations finish. So on no-downtime compute with health checks enabled, the old instance does keep serving. A classic single-instance app and a scale-to-zero app go down until a fixed redeploy. I'll describe each case and replace "the start window" with the 300 s health-check limit. #677.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated
the code that stops using the column (expand, then contract). `appkit db migrate` warns when it writes a drop, a
rename, a type change or `SET NOT NULL`.
- **A migration fits in the start window.** A migration that runs longer than the platform's start timeout can never
land. Long backfills run as a separate job against the deploy branch, not as a migration.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review (from reasoning, not reproduced): Only the app's service principal owns objects on the deploy branch, and developers have no owner rights (line 358). A backfill job running as its own identity would get permission denied on the SP-owned tables, and no grant mechanism is described. It also conflicts with Goal 2, "only the deployed app changes the branch it deploys to". With migrations ruled out for long work in the same paragraph, a long backfill has no supported path.

@ditadi ditadi Nov 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly. On a live Lakebase branch, databricks_superuser inherits pg_read_all_data, pg_write_all_data and pg_signal_backend. On a Lakebase fork, a role with that membership can SELECT, INSERT, UPDATE, DELETE and TRUNCATE the tables the service principal owns, and create tables in its schema. It cannot ALTER, DROP or index those tables. So a backfill job works when its role on the deploy branch has the DATABRICKS_SUPERUSER membership. You're right that this contradicts Goal 2 as written. I'll narrow Goal 2 to the schema (only the deployed app changes the deploy branch's schema) and describe a long backfill as an expand migration, then a batched DML job, then a contract migration. #677.

@ditadi

ditadi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

This is an AI-generated review (Claude Code). Treat each comment as a lead to check, not a verdict.

This PR only adds the design doc, so each inline comment marks a place where the doc contradicts itself, design 002 on main, or the code at the top of the two stacks: stack/database-stable/06-serve-operations for operations and stack/database-dev-flow/08-db-check for migrations. Comments marked "checked" are backed by reading that code; the four marked "from reasoning" come from how Postgres and the platform behave and weren't reproduced. Nothing was run: no CLI, no deploy.

Two comments point at code bugs, not just doc drift:

  • init.ts installs drizzle-kit only for the migrate and introspect starts, but clean is now the default for a new fork, so a new app's first appkit db migrate fails.
  • --schema none on a new fork skips the rebuild but still clears APPKIT_DB_NEW_BRANCH, so the copied production rows stay on the branch.

Smaller items without inline comments:

  • databricks apps run-local with exported M2M credentials migrates as an identity other than the app.
  • The journal-ownership rule as written would refuse the service principal's own journal in public.
  • The production registry claims ignore the Jiti source fallback.
  • The v0.87/v0.88 references are stale; main is at v0.91.
  • The "preserves existing beta startup behavior" heading is wrong: an upgrade without schemaName fails the startup check.
  • The operation sections repeat design 002 and have already drifted from it; linking to design 002 would avoid that.
  • The listNotes example has no limit, and getNote lets any caller read any note.
  • The pg_cancel_backend runbook doesn't name a role with the rights it needs.
  • The branch-limit message lists only your own branches, so a new developer has nothing to delete.

Thanks for the review. Replies to the items in the summary:

  • "databricks apps run-local with exported M2M credentials migrates as an identity other than the app." Confirmed: run-local forwards the CLI's own M2M credentials and takes the flag from app.yaml. The L307 rule covers it, since a local PGUSER is the developer, not the app's client id.
  • "The journal-ownership rule as written would refuse the service principal's own journal in public." The code was right and the doc was stale. The migrator trusts a table owned by a role it has, or by the schema owner (apply.ts:424), so the service principal's journal in public passes. Doc fix in docs: design 001, database development flow #677. The security section will also say that any DATABRICKS_SUPERUSER role on the deploy branch can write journal rows (measured on a model of the roles).
  • "The production registry claims ignore the Jiti source fallback." Confirmed (load-routes.ts:60-62). It goes away with the link to design 002.
  • "The v0.87/v0.88 references are stale." Agreed. I'll drop them; main has moved past 0.91 since.
  • "The 'preserves existing beta startup behavior' heading is wrong." Right, an upgrade without schemaName fails the startup check. The section will be named for what it is: the upgrade path for apps without schemaName.
  • "The operation sections repeat design 002 and have already drifted from it." Agreed. They become a summary and a link.
  • "The listNotes example has no limit, and getNote lets any caller read any note." Agreed. Any example that stays uses .limit() and a per-note policy, as in design 002.
  • "The pg_cancel_backend runbook doesn't name a role with the rights it needs." Measured on Lakebase: databricks_superuser inherits pg_signal_backend, so a role with the DATABRICKS_SUPERUSER membership on the deploy branch can cancel or end the migrating session, because the service principal is not a superuser. Someone with CAN USE on the project can create that role. The runbook will name it, with a compute restart (CAN MANAGE) as the fallback.
  • "The branch-limit message lists only your own branches, so a new developer has nothing to delete." The code already handles it: when none of the project's branches are yours, it says to ask a project owner (init.ts:247). I'll fix the doc.

ditadi and others added 3 commits November 15, 2026 20:08
Replace the operation sections with a summary that links to design 002,
and describe the behavior the stack has after this fix round: init
decides from the branch instead of the APPKIT_DB_NEW_BRANCH marker,
forks the default target's branch and installs drizzle-kit for every
start except none; migrate on start runs only as the app's own service
principal and refuses dev- or session-shaped branches; a deploy start
refuses edited migrations and diverged histories; the failure runbook
repairs data with a DATABRICKS_SUPERUSER role instead of editing the
migration; and the deploy section states what Apps does with health
checks, classic and no-downtime compute, crashes and scale to zero.

Also drop the release numbers, rename the upgrade section, correct the
journal trust rule and the ownership recovery, name the role for a stuck
start, and record the answered open questions.

Signed-off-by: ditadi <victordperd@gmail.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…ll ownership inventory in design 001

Migrate on start now requires PGUSER to be set and equal to
DATABRICKS_CLIENT_ID, so the doc states it as an identity check rather
than deployment detection, and names the gap before init writes PGUSER.

The fork-copy rule adds a branch the same init run created, the tag
written only into a journal the migrator trusts with a warning when it
cannot be, migrate, introspect and seed refusing an untouched copy, and
--schema clean after keeping copied rows. The remaining windows are
listed as they stand.

The ownership recovery takes the hand-off's full inventory across
databases before delete-role, and says the delete drops the role's
memberships and grants. Clean lists what CASCADE drops before it asks;
init's rebuild reports it after.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
…foreign key refusal

Introspect keeps the old schema.ts's .private() and writes every other
column and foreign key as the catalog has it.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi
ditadi requested a review from fjakobs November 17, 2026 00:26

@MarioCadenas MarioCadenas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One gap inline. Reviewed at fde70db.

Comment thread designs/001_DatabaseDevelopmentFlow.md Outdated
checked under the schema's advisory lock. A branch ahead of the checkout would otherwise record DDL it never ran.

Introspect keeps the `.private()` an existing `schema.ts` declares, because the server entity client omits private
columns from its reads by default. Every other column, and every foreign key in the catalog, is written as the catalog

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overstates what introspect writes. render.ts:505-520 skips generated columns, unsupported types and non-primary identity columns, and fkProblem (render.ts:655-696) skips composite, cross-schema, self-referential and cyclic foreign keys, which render.test.ts:185-200 reports as skipped. Someone reading this could treat the generated schema.ts and baseline as a faithful copy of the catalog when they aren't.

Could this name the supported subset and say the rest is skipped with a warning?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, it overstated it. 5816bfd rewrites the paragraph to name the subset introspect writes: tables with a single-column primary key and a name AppKit does not reserve, columns of supported types, identity only on the primary key, and fk() only for a single column that references the primary key or a single-column unique key of another table in the same schema. It also says generated columns, unsupported types, non-primary identity columns, unsupported defaults, and composite, cross-schema, self-referencing or cycle-closing foreign keys are left out, each with an // appkit: skipped … comment and a warning, so schema.ts and the baseline are not a full copy of the branch.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up in 4e53b2d: a foreign key on a column introspect cannot write goes with that column, so the paragraph now says the column stays only when it is otherwise supported, and that the column's skip comment covers both.

ditadi and others added 2 commits November 20, 2026 09:52
Introspect leaves out what the builder cannot express, such as generated
columns, unsupported types and defaults, identity columns other than the
primary key, and composite, cross-schema, self-referencing or cycle-closing
foreign keys. The design said every column and foreign key was written as the
catalog has it.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
A foreign key on a column introspect cannot write goes with that column,
and the column's skip comment covers both.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi
ditadi requested a review from MarioCadenas November 20, 2026 13:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants