Skip to content

perf(bigquery): run statements through jobs.query - #4352

Merged
datlechin merged 4 commits into
mainfrom
perf/bigquery-jobs-query
Nov 19, 2026
Merged

datlechin merged 4 commits into
mainfrom
perf/bigquery-jobs-query

Conversation

@datlechin

@datlechin datlechin commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Summary

  • P29 BigQuery: a statement runs through jobs.query with JOB_CREATION_OPTIONAL, a requestId and a 2 s server wait. One that finishes in that wait comes back in one request, first page included. Before: jobs.insert, a 500 ms sleep, jobs.get and jobs.getQueryResults, so at least 3 requests and 500 ms per statement.
  • P29: a statement still running after the wait is polled through its job as before: query timeout, cancel at the deadline, first page from getQueryResults.
  • P29: Stop before the request body has left the client cancels the request, as on main.
  • P29: Stop after that releases the caller on the answer or after 1 s. An answer that shows a running job sends a job cancel. The request is cancelled 10 s after Stop; both deadlines count from Stop.
  • P29: a 401 refresh or 429 retry never starts after Stop, and reuses the requestId.
  • P29: the answer body is capped at 32 MiB while it streams (BigQuery caps an answer at 10 MB).
  • P29: errors jobs.query returns as HTTP 400, 403, 404 and 409 read as failed statements, as job errors did, so messages, the partition-filter hint and the parameter cache eviction are unchanged. Affected rows, bytes processed and billed, cache hit and server time come from the answer.
  • P29: a BigQuery token wait ends when its caller is cancelled; a shared token refresh runs on for other callers.
  • Each lifecycle is a table in code: BigQueryStopLifecycle (stoppable request), BoundedHTTPRead, GoogleAccessTokenWait.
  • Changelog: apple/changelog.d/4352.changed.md. Docs: docs(databases): describe BigQuery jobs.query runs and Stop docs#252.

Test plan

  • TableProGoogleCloudTests: BigQueryStopLifecycleTests checks every stage and event against the lifecycle table; BigQueryStoppableRequestTests has one case per transition on a ManualClock; plus BoundedHTTPReadTests, GoogleAccessTokenWaitTests, BigQueryRequestLoopTests, BigQueryStatementRequestTests (request body). They name new types, so they do not compile on main.
  • package TableProCore 'GoogleCloud|Spanner|Bigtable|Firebase': 230 passed. ios IAMPasswordMintTests: 34 passed.
  • verify.sh on the last code commit: generate, build, plugins, lint (13 files) PASS.
  • Not run against a real BigQuery project: none here. The tests cover the requests built, the answers read and every Stop transition.
  • Probe: URLSession reports the whole request body sent before the answer, over HTTP/1.1 (loopback) and h2 (bigquery.googleapis.com).
  • Round trips from the code, per statement that finishes within 2 s: 3 requests plus a 500 ms sleep before, 1 request after. At 200 ms RTT, about 1.1 s to 0.2 s plus server time.

@datlechin
datlechin enabled auto-merge (squash) November 19, 2026 13:17
@datlechin
datlechin merged commit 30aa585 into main Nov 19, 2026
12 checks passed
@datlechin
datlechin deleted the perf/bigquery-jobs-query branch November 19, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant