Repository navigation
[AIR] CNXT-2687: print the source image size and warn about the 30 GB registry limit before pushing - #7868
[AIR] CNXT-2687: print the source image size and warn about the 30 GB registry limit before pushing#7868vinchenzo-db wants to merge 6 commits into
Conversation
`databricks air images push` now inspects the local image size after the platform check, prints it, and fails before tagging or uploading when it exceeds AI Runtime's 20 GB maximum, with the server's message plus advice to start from a Databricks base image and trim layers (CNXT-2687). Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
Docker reports the uncompressed size while the server measures compressed layers, so an oversized local image may still be accepted: print the size, warn with the limit and advice, and let the push proceed. Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/air/image-push: branch ✅ | main (a4388d7) ❌ | latest (v1.27.0) ❌main (a4388d7): latest (v1.27.0): same output as main (a4388d7) |
There was a problem hiding this comment.
Automated review at 5e75898. Each finding was verified against the code.
Not ready yet: 1 blocking issue(s), tagged [blocking].
This PR adds an image-size info line and an oversized-image warning to databricks air images push. The size check is well-tested and the warning path (oversized image still pushes) works correctly. However, the parse-failure path hard-aborts the push for what is a purely advisory feature, introducing a new failure mode that the base code doesn't have. A second, minor concern is that the size inspect spawns a redundant subprocess on the same image.
Fold {{.Size}} into the platform inspect so no second docker process is
spawned, and warn instead of failing when the size cannot be parsed, since
the size check never gates the push.
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at a5f8163. Each finding was verified against the code.
No blocking issues found.
This PR adds image-size reporting and a warning to databricks air images push, reading the size from the existing docker image inspect call and printing it before pushing. When the uncompressed size exceeds 20 GB, an advisory warning is shown but the push proceeds. The implementation is clean: edge cases (missing | separator, unparseable size) degrade gracefully into warnings, the format string uses values that cannot contain the delimiter, and tests cover at-limit, over-limit, and unreadable-size scenarios. No concrete defects were found.
✅ Integration test reportCommit: 47a3ded
Top 4 slowest tests (at least 2 minutes):
|
maggiewang-db
left a comment
There was a problem hiding this comment.
We have changed the limit: AI Runtime no longer has an image size limit. The only limit now is Databricks Artifact Registry (DAR), which caps images at 30 GB. The 20 GB check and warning in this PR are out of date, so I'm requesting changes to block merging until that's resolved.
AI Runtime no longer limits image size; Databricks Artifact Registry caps images at 30 GB. Update the threshold, messages, tests, golden output and changelog fragment accordingly. Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
|
Thanks, updated to the 30 GB DAR limit and reworded; AI Runtime's own limit is gone from the PR. |
There was a problem hiding this comment.
Automated review at cd98a4f. Each finding was verified against the code.
No blocking issues found. 1 non-blocking comment(s) below.
The PR adds a pre-push size check to databricks air images push, printing the uncompressed image size and warning when it exceeds 30 GB. The logic is clean: the inspect format is extended without a second Docker process, parsing failures are advisory, and the push always proceeds. Tests cover at-limit, over-limit, and unparsable cases plus an acceptance test for the oversized path. One minor user-facing text issue noted below.
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at 47a3ded. Each finding was verified against the code.
No blocking issues found.
This PR adds image-size reporting and a 30 GB advisory warning to databricks air images push, piggy-backing on the existing docker image inspect call. The new code parses the size, prints it, and warns (without blocking) when the uncompressed size exceeds the registry's compressed-layer limit. The implementation is clean: the size/platform parsing handles missing and unparsable values gracefully, tests cover the boundary, over-limit, and unreadable cases, and the acceptance test gains an oversized mode with matching golden output. I found no correctness defects.
Changes
databricks air images pushnow reads{{.Size}}from the existingdocker image inspectcall (combined format string, no extra Docker process) and prints the size (Source image X is 7.1 GB uncompressed; Databricks Artifact Registry caps images at 30 GB compressed.); if the size cannot be read, it warns with the raw inspect value instead. When the local image exceeds 30 GB it also prints a warning that Databricks Artifact Registry may reject the push, with one line of advice: start from a Databricks base image (databricksruntime/air) and trim layers such as model weights and build caches. The size is advisory: an oversized or unparsable size only warns, and the push proceeds. The upload itself is unchanged.Why
Databricks Artifact Registry rejects images over 30 GB, but the CLI gave no size information before the upload, so a multi-gigabyte push from a laptop could run for a long time before failing (Jira CNXT-2687, found in the CUJ PE64 runthrough). Docker already knows the local image size, so printing it before the push costs nothing extra and lets the user stop early. AI Runtime itself no longer limits image size, so the PR only refers to the registry's limit.
Supervisor's call: warn, not fail, because the local size is uncompressed while the registry measures compressed layers, so a hard local failure would block images the registry would accept. The size uses decimal gigabytes, matching Docker's
docker imagesoutput.Tests
TestRunImagePushWarnsAboutOversizedImage(table: at limit, over limit, unreadable size) checks that an image at exactly 30 GB prints no warning, an oversized one prints the warning and is still pushed, and an unparsable size prints a warning and is still pushed.TestRunImagePushTagsAndPushesLocalImageasserts the combined inspect format and the printed size line.acceptance/air/image-pushgained anoversizedfake-docker mode and an "oversized image warns but is pushed" step; golden output regenerated.go test ./cmd/air -run 'TestRunImagePush|TestImageExistsLocally',go test ./acceptance -run TestAccept/air/image-push,task lint-q,task wsandtask check-changelogpass locally. Two unrelatedcmd/airtests (TestGetRunMLflowRetryJSON,TestLogsCommandRepeatsRunLinkAfterStreaming) fail the same way on cleanmainin this sandbox.This pull request and its description were written by Isaac.