Repository navigation
Complete Volume file operations through the kernel backend #1057
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
aakash-saravanan-db
wants to merge
45
commits into
main
Choose a base branch
from
aakash-saravanan_data/feat/volume-restart-20261113
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
45 commits
Select commit
Hold shift + click to select a range
0ff5094
Complete Volume file operations through the native kernel
aakash-saravanan-db 856e533
Consume Volume instructions after failed completion
aakash-saravanan-db 53d8288
Fetch completed async results without a redundant status poll
aakash-saravanan-db 42d3106
Clarify asynchronous Volume completion
aakash-saravanan-db 5d1e6ab
Pin the native source for Volume completion
aakash-saravanan-db e8c9803
Document completed async result retrieval
aakash-saravanan-db 68380a3
Select the Windows-compatible Volume kernel
aakash-saravanan-db d55977d
Keep async result reuse in the kernel and tolerate missing cancellers
aakash-saravanan-db 3253170
Pin native Volume storage retry and terminal response fixes
aakash-saravanan-db 2c8d0f7
Preserve Volume error provenance and stream teardown
aakash-saravanan-db 63b22d2
Avoid cancelling completed Volume transfers during close
aakash-saravanan-db 2031485
Format Volume cancellation cleanup comprehension
aakash-saravanan-db 5027e63
Select native Volume inactivity timeout correction
aakash-saravanan-db 060b64b
build(kernel): select corrected Volume completion
aakash-saravanan-db a689d26
build: select native Volume cancellation and error handling
aakash-saravanan-db 5d728ca
build: use native root authorization fix
aakash-saravanan-db b93ade0
build: update kernel source for replayable uploads
aakash-saravanan-db 3c83e9c
build: update kernel for local stream close
aakash-saravanan-db b6b20dc
fix(volume): select replayable file uploads and tighten validation
aakash-saravanan-db 94f0a87
fix(kernel): use permission-preserving Volume downloads
aakash-saravanan-db 8a0a433
build(kernel): select sanitized storage redirects
aakash-saravanan-db 5965f81
Select native Volume lifetime and replay implementation
aakash-saravanan-db f14d16e
Pin kernel with statement lifecycle cleanup
aakash-saravanan-db 094c214
Document Volume handles and verify async owner cleanup
aakash-saravanan-db 39a6dc7
build: select kernel with async result cleanup
aakash-saravanan-db 18c1a5a
build: select kernel with Volume instruction error telemetry
aakash-saravanan-db 9c8d3e9
build: pin kernel with complete Volume instruction handling
aakash-saravanan-db d7d606d
build: use corrected kernel volume completion
aakash-saravanan-db 827971a
build: select kernel with origin-safe Volume redirects
aakash-saravanan-db bef1659
build: select kernel with normalized upload lengths
aakash-saravanan-db 3694957
build: pin kernel completion telemetry correction
aakash-saravanan-db 974bf84
build: select kernel with restricted-directory GET support
aakash-saravanan-db cb95a26
Keep distinct Volume ownership test cases
aakash-saravanan-db bc2f439
build: pin kernel with bounded symlink resolution
aakash-saravanan-db 55e7f74
Document kernel connection-close cancellation
aakash-saravanan-db f90c814
refactor(volume): streamline completion ownership
aakash-saravanan-db ab50ea0
build: select updated Volume kernel
aakash-saravanan-db b5a9035
fix(volume): preserve Python errors and transfer telemetry
aakash-saravanan-db b9a751b
fixup! fix(volume): preserve Python errors and transfer telemetry
aakash-saravanan-db 0586423
fixup! fix(volume): preserve Python errors and transfer telemetry
aakash-saravanan-db 8caaaf5
fixup! fix(volume): preserve Python errors and transfer telemetry
aakash-saravanan-db b05d562
fixup! fix(volume): preserve Python errors and transfer telemetry
aakash-saravanan-db 9caea50
fixup! fix(volume): preserve Python errors and transfer telemetry
aakash-saravanan-db 71f2ee8
fixup! fix(volume): preserve Python errors and transfer telemetry
aakash-saravanan-db 6ad4f89
fixup! fix(volume): preserve Python errors and transfer telemetry
aakash-saravanan-db File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| 0b5d0edfed32c0745d5abc2468fc3070e2eb2a17 | ||
| 254245b3b4fcc770f543b02e299b114666ac26a2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟠 High —
pyproject.toml:73still allowsdatabricks-sql-kernel ^1.4.0, but the code now calls three kernel APIs that don't exist yet:ExecutedStatement.is_volume_operation(read here on every sync execute),kernel_handle.is_volume_operation(read inKernelResultSet.__init__for every result, including metadataResultStreams), andfetch_result()in place ofawait_result()for async results.If the resolver installs any wheel in the allowed range that lacks them, this isn't limited to Volume ops. Every
execute(), every metadata call and everyget_async_execution_result()raisesAttributeError, so ordinarySELECT 1breaks onuse_kernel=True. The PR description says the floor "has not been selected". This should block merge: bump the floor in the same change. Until then, readingis_volume_operationwithgetattr(..., False)would at least keep non-volume queries working.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The optional-canceller defect is corrected at
ab1efce1: registration uses.get, the Statement remains retained, and cleanup skips absent cancellers. All 276 affected tests pass; final native live checks preserve lazy BinaryIO and one-read empty/nonempty streams.The new Volume marker/completion API dependency is a valid merge prerequisite. The source build pins compatible
22f96fac; a compatible published native version/floor and installed-package proof remain explicitly open. This PR is not merge-ready, and no old-native fallback or invented release version was added.The fetch_result subclaim is separate: released 1.4.0 exposes it, but the substitution is removed from this PR. Merged native #451 provides terminal-response reuse with the original await_result boundary. Actual public-cursor owner and reader/refetch tests show identical await/fetch requests, rows and metadata on the selected native; redundant #1056 is superseded without merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for confirming the Volume-marker dependency is a merge prerequisite, and for dropping the
fetch_result()substitution. That part of the finding is resolved. The core issue is still there, though.pyproject.toml:73still saysdatabricks-sql-kernel ^1.4.0. Line 641 still readsexecuted.is_volume_operationwith no guard.KernelResultSet.__init__still readskernel_handle.is_volume_operationthe same way. So if a published wheel in that range lacks the marker, ordinary sync executes and metadata calls fail withAttributeErrorbefore the code ever checks for a Volume op. A source build pinned to22f96facdoesn't limit what the resolver installs. Leaving this open until the floor is raised to the first published wheel that ships the marker, or until both reads are guarded so non-volume queries keep working.