Repository navigation
[AIR] CNXT-2688: fail on scheme-less /Volumes/ paths in fs commands - #7862
vinchenzo-db wants to merge 10 commits into
Conversation
A path such as /Volumes/main/default/vol/data.csv without the dbfs: scheme was treated as a local path, so `databricks fs cp` and `fs mkdirs` silently acted on the laptop. Fail with a hint to use dbfs:/Volumes/... instead, and show a UC Volume example in `fs cp --help`. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at 907356b. Each finding was verified against the code.
Not ready yet: 1 blocking issue(s), tagged [blocking].
This PR adds a guard in filerForPath that rejects scheme-less /Volumes/ paths with a helpful error pointing to dbfs:/Volumes/…, and updates the cp --help text. The main issue is that both the help text (cp.go:217) and the error message (helpers.go:42) say /Volumes/… "is a local path" while the code now refuses to treat it as one—the documentation and the implementation disagree. The new guard also omits /Skills/, which has the same silent-wrong-filesystem risk, and the testing rule asks for an acceptance test rather than only a unit test for this user-visible error.
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/cmd/fs/uc-path-without-scheme: branch ✅ | main (a4388d7) ❌ | latest (v1.27.0) ❌main (a4388d7): latest (v1.27.0): same output as main (a4388d7) |
There was a problem hiding this comment.
Automated review at 73153e3. Each finding was verified against the code.
Not ready yet: 1 blocking issue(s), tagged [blocking].
This PR adds a guard in filerForPath that rejects scheme-less /Volumes/ and /Skills/ paths with a helpful hint to add dbfs:. It also updates fs cp --help and adds an acceptance test plus a unit test. The guard catches a real UX papercut, but it also blocks legitimate local /Volumes/ paths on macOS (the OS mount point) with no workaround because the file: scheme is rejected too. The new acceptance test skips Windows entirely instead of using the documented MSYS_NO_PATHCONV workaround.
✅ Integration test reportCommit: 1bb23ae
Top 3 slowest tests (at least 2 minutes):
|
On macOS /Volumes/ is the mount point for local drives, so rejecting every scheme-less /Volumes/ path would block real local paths. Keep them local and print a warning on stderr instead. Run the acceptance test on Windows with MSYS_NO_PATHCONV. Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at 6fd3053. Each finding was verified against the code.
No blocking issues found. 2 non-blocking comment(s) below.
This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and updates cp --help with a UC Volume example and a note that scheme-less paths are local. The core logic is clean and the acceptance test pins the warning, but the unit test doesn't verify the new warning and the help-text clarification only covers cp.
…bcommands Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at a57c267. Each finding was verified against the code.
No blocking issues found. 1 non-blocking comment(s) below.
This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and enriches the --help text of every subcommand to clarify that a scheme-less path is always local. The warning logic and tests look correct. The main concern is that the new help-text sentence is copy-pasted verbatim into five subcommands instead of living once in the parent fs command.
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at 58cdf96. Each finding was verified against the code.
No blocking issues found. 1 non-blocking comment(s) below.
This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and improves the --help text to clarify that scheme-less paths are always local. The documentation updates are clear and well-placed. The warning's behavior (keeping the path local) is correct, but the suggestion text can mislead macOS users whose /Volumes/ paths are legitimate local mounts.
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at 9977623. Each finding was verified against the code.
No blocking issues found. 1 non-blocking comment(s) below.
This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and updates help text to clarify that scheme-less paths are always local. The implementation is clean, well-tested, and the trade-off (warn, don't reject) is reasonable. One side-effect worth addressing: the warning also fires during shell tab-completion, producing noise in interactive sessions.
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at baadab1. Each finding was verified against the code.
No blocking issues found. 2 non-blocking comment(s) below.
This PR adds a stderr warning when databricks fs commands receive a scheme-less /Volumes/ or /Skills/ path, and improves --help text to clarify that such paths are treated as local. The help-text changes are clean. Two minor issues with the warning: it fires during tab completion (polluting the terminal) and prints twice when fs cp gets two /Volumes/ arguments.
Move the scheme-less /Volumes/ and /Skills/ warning out of filerForPath, which shell completion also calls, into a helper each command runs once with all of its path arguments. Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at 3bae952. Each finding was verified against the code.
No blocking issues found. 1 non-blocking comment(s) below.
This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and updates help text to clarify that schemeless paths are always local. The implementation is correct and well-tested (acceptance test + unit test). One minor maintenance concern: the UC prefix list is duplicated within the same file.
Co-authored-by: Isaac <no-reply@databricks.com>
|
/review |
There was a problem hiding this comment.
Automated review at 1bb23ae. Each finding was verified against the code.
No blocking issues found. 1 non-blocking comment(s) below.
This PR adds a stderr warning to databricks fs subcommands when a /Volumes/ or /Skills/ path lacks the dbfs: scheme, and updates --help text to clarify that scheme-less paths are local. The change is well-scoped, tested at both the unit and acceptance level, and the doc-comment and code-comment quality is good. One inconsistency with the existing warning mechanism is worth addressing.
Changes
databricks fscommands now warn on stderr, once per command and not during shell completion, when a path argument starts with/Volumes/or/Skills/but has nodbfs:scheme, and keep treating it as a local path:databricks fs cp --helpnow shows a UC Volume example (dbfs:/Volumes/my_catalog/my_schema/my_volume/data.csv) and says that a scheme-less path is local even if it starts with/Volumes/.No behavior change otherwise: scheme-less paths stay local paths, and
/Volumes/...is not auto-routed to the Files API.Why
Fixes CNXT-2688 (AIR CUJ PE64 papercut).
databricks fs cp file /Volumes/main/<schema>/<vol>/andfs mkdirs /Volumes/...were treated as local paths: they failed with a local "permission denied" on the laptop, or silently succeeded in the wrong place where/Volumesis writable. Onlydbfs:/Volumes/...reached Unity Catalog, andfs --helpdid not say so.Supervisor's call: warn, don't reject, because
/Volumesis the macOS mount point for local drives, so a scheme-less/Volumes/...path can be a legitimate local path. The warning removes the silent part of the papercut without breaking those paths or adding a new scheme.Tests
acceptance/cmd/fs/uc-path-without-schemepins the warning and the local behavior for/Volumes/and/Skills/paths, one warning for acpwith two such arguments, and no warning during completion. It runs on Windows withMSYS_NO_PATHCONV=1, asacceptance/cmd/api/*do.TestWarnSchemelessUCPathsincmd/fs/helpers_test.gocovers the warning and the paths that must not warn.go test ./cmd/fs/,go test ./acceptance -run TestAccept/cmd/fs/andtask lint-qpass locally.This pull request and its description were written by Isaac.