Repository navigation
Push a locally renamed project given by its cloud id - #751
Conversation
When a cloud command gets a project id with --push, the local copy was only looked up under the cloud project's name, so a renamed local directory was skipped and the old cloud files ran. Fall back to finding the local project by its cloud id, as pulling already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Requesting changes. The fix works when the command runs from the workspace root, as the description shows. Run from any other folder in the workspace, it adds a new way for the command to fail, and that includes a project that was never renamed.
Finding: the fallback searches from the workspace root, but the push resolves names against the current directory (inline comment)
try_get_project_path_by_cloud_id walks from the CLI root (the folder holding lean.json). get_local_project_path and PushManager.push_projects / _push_project both work relative to Path.cwd(). The CLI finds lean.json by searching upward from the current directory, so cloud commands can run from any subfolder, and then the fallback returns a path the push can't handle:
- From inside the project directory:
cd MyProject && lean cloud backtest <cloud id> --push. The name-based guessMyProject/MyProjectdoesn't exist, so the walk returnsMyProject, which is the current directory itself. The push names it'.'and tries to rename the cloud project to.. I ran this end to end with a project that was never renamed:On master, the same command compiles and backtests.[1/1] Pushing '.' Renaming project in cloud from 'RvPush751' to '.' Cannot push '.': Project name contains unsupported special character (.) Error: Cannot push '38351885' to the cloud, aborting so the cloud does not run outdated files - From a sibling subfolder (
ws/sub, with the project atws/MyProject):push_projectscallspath.relative_to(Path.cwd())outside itstry, so the command stops withError: '.../ws/MyProject' is not in the subpath of '.../ws/sub'. I reproduced this end to end, and with the realPushManagerin a unit test. On master it runs.
Suggestion: a smaller version of the same fallback that keeps master's behavior in these cases is to accept the path it finds only when it lies strictly under the current directory (Path.cwd() in local_path.parents), which is all the rest of the push can name. Please add a test that runs the resolver with the real PushManager from a directory other than the workspace root. The new test mocks push_projects, so it can't catch this.
What I checked
- CI is green on every platform and Python version.
- The new test fails on master and passes on this branch.
tests/components/cloudandtests/commands/cloudpass (170). - The name-based lookup still runs first, so the existing paths from the workspace root are unchanged. When a project has a name-based local match, the full-workspace walk doesn't run. Otherwise it runs once per command, the same walk
PullManageralready does for each project it pulls. - The other callers of
get_cloud_projectpasspush=Falseand are unaffected.
Other repositories: none need an update.
Risk: I set risk:high. The change adds a new failure, and an attempted cloud rename, to cloud backtest, cloud optimize and cloud live deploy with --push when they run outside the workspace root. Once that's fixed, its evidence would be a test and an end-to-end run from inside a project directory and from a subfolder, alongside the existing run from the root.
The push names projects relative to the current directory, so a project found by cloud id elsewhere in the workspace, or the current directory itself, could not be pushed and made the command fail where master ran. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, confirmed and fixed in b923968. A project the cloud-id fallback finds is now used only when it lies strictly under the current directory ( New tests use the real
End to end, from inside the project directory and from a sibling subfolder, the command compiles and backtests without a push, as on master. A project renamed under the current folder or at the root is pushed, renamed in the cloud to match, and backtested. The description has the details. |
There was a problem hiding this comment.
Approving. b923968 resolves my earlier finding. The fallback's result is now used only when it lies strictly under the current directory, which is all the push can name. Everywhere else the command behaves as on master.
What I checked on this commit
- CI is green on the head commit on every platform and Python version.
tests/components/cloudandtests/commands/cloudpass (173). Swapping in each earlier version ofcloud_project_manager.pyconfirms the claims: with master's version, the two renamed-project tests fail. With the first commit's version, bothdoes_not_push_project_outside_the_working_directorycases fail. The new tests use the realPushManagerfrom a directory other than the workspace root, as I asked.- I tested it end to end against a cloud API with
lean cloud backtest <cloud id> --pushon a project I created and pushed:- From inside the project directory, never renamed, the command compiled and backtested without a push, as on master. It no longer tries to rename the project to
.. - From a sibling subfolder, it compiled and backtested, as on master. It no longer raises the
is not in the subpatherror. - With the directory moved to
sub/Renamed...and edited, run fromsub, the project was pushed, renamed in the cloud to match, and compiled.
- From inside the project directory, never renamed, the command compiled and backtested without a push, as on master. It no longer tries to rename the project to
- The rest still holds from my first review. The name-based lookup runs first, so paths that work today are unchanged. The workspace walk runs at most once per command, only when the name-based path is missing. It is the same walk
PullManageralready does for each project it pulls. The other callers ofget_cloud_projectpasspush=False.
Other repositories: none need an update.
Risk: I set risk:low. This is a small bug fix that is easy to revert. It only adds a fallback for a case where master silently ran outdated cloud files, and every path that works on master is unchanged. Unit tests fail without the fix, CI is green and the end-to-end runs pass.
There was a problem hiding this comment.
Requesting changes at b923968. The earlier current-directory/sibling push failures are fixed, and the intended renamed-project cases work. There is one remaining blocker: the newly invoked workspace-wide lookup has side effects and failures before the current-directory guard runs.
P1 — Make fallback discovery non-destructive and bounded
try_get_project_path_by_cloud_id constructs Storage for every directory's config.json (project_manager.py:113–124). Storage.__init__ deletes the file when parsing fails (storage.py:64–74). Thus this new call can delete unrelated application configuration while a cloud command completes successfully.
I verified both outcomes against master:
- With an unrelated
Other App/config.jsoncontaining a JSON comment and the renamed project underSub/Renamed...,lean cloud backtest <id> --pushon this commit pushed, compiled and backtested successfully, but deleted the unrelated file. The same command with master's resolver preserved it. A cloud-only project with no local counterpart also triggers deletion in a focused test: no push is required. - With an unrelated searchable but non-listable directory (
0111), master's resolver returns the cloud project; this commit raisesPermissionErrorfromdirectory.iterdir(). The helper does not catch enumeration failures.
The descendant check cannot protect against either outcome because it runs after discovery. Please make discovery read-only, constrain it to eligible project locations before traversing, and handle inaccessible candidates without breaking cloud-only commands. Add regression coverage for unrelated configuration preservation and enumeration failures. Catching the lookup exception alone would not prevent deletion.
The existing PullManager uses this helper, and UI project discovery also opens candidates through Storage; both have the same configuration-deletion bug on master. I added independently reproduced cases to #374. Please keep safe lookup logic shared with PullManager rather than duplicating another scanner. Those pre-existing pull/UI behaviors are not themselves regressions introduced here.
Verification
- CI on the reviewed head: 54 successful checks, 6 skipped.
python -m pytest -q tests/components/cloud tests/commands/cloud: 173 passed.- With master's resolver substituted, both new renamed-project tests fail because no push occurs; both outside-working-directory cases still pass.
- Full local suite with the source checkout on
PYTHONPATH: 2867 passed, 81 skipped. Without that import-path correction, the completed run was 2866 passed, 81 skipped, 1 failed: the reported child-process version mismatch (lean 1.0.233versuslean 0.0.0-dev), also reproduced with master's resolver. An initial attempt was stopped at its 120-second limit. python static_analysis.py: BUILD PASSED, with five existing unused-import warnings.- Tested end to end: commands from inside the original project and from a sibling directory compiled and backtested without pushing; commands for renamed projects from the root and a containing subfolder pushed, compiled and backtested. All exited 0. The comparison above also confirmed the unrelated-file deletion end to end.
Other repositories: none need an update.
Risk: risk:high. The fallback newly exposes unrelated local files to deletion and adds filesystem-dependent command failures. Green CI and successful renamed-project runs do not cover those regressions.
There was a problem hiding this comment.
Requesting changes on 3d0339d. The read-only configuration lookup fixes the deletion I reported, and an unlistable directory no longer raises. There is one remaining part of the same discovery-isolation finding: an inaccessible candidate can still prevent discovery of readable sibling projects.
P1 — Do not abandon sibling discovery when one candidate cannot be inspected
The new try surrounds the entire directories.extend(...) generator. If one child's is_dir() raises OSError, the generator stops permanently; the catch skips every later sibling, not just the inaccessible entry. A normal directory symlink whose target is below an inaccessible ancestor triggers this on Python 3.11/Linux.
I reproduced this with native directory enumeration, without mocking or sorting it: an unrelated inaccessible symlink is encountered before a readable Renamed/config.json containing the requested cloud ID. The helper returns False, and the real cloud resolver returns the cloud project with zero pushes. Removing only the unrelated symlink makes the same resolver find and push the project.
Tested end to end as a non-root user: with that link present, lean cloud backtest <id> --push exits 0 and completes a backtest using the original cloud source, leaving the edited local source unpushed. Removing only the link makes the same command upload the edit and complete its backtest. This is an unresolved case of #720, not a claim that master successfully pushes this renamed project.
Please keep discovery read-only and shared with PullManager, but isolate failures to the candidate that failed rather than abandoning its parent directory. Add a regression proving that an inaccessible entry does not prevent a later readable renamed project from being found and pushed. The existing permission test covers failure to list a queued directory; it does not cover failure to inspect one child while enumerating its parent.
Verification
- CI on this head: 54 successful checks, 6 skipped.
- Affected cloud/project/pull tests: 291 passed, 1 skipped. The four newly added regression cases fail with the previous helper substituted.
- Full suite with the checkout on
PYTHONPATH: 2871 passed, 81 skipped. python static_analysis.py: BUILD PASSED, with five unused-import warnings.- Real-filesystem, non-root verification also confirms that the ordinary unlistable-directory case is skipped and an unrelated commented configuration is preserved byte for byte.
- Five other end-to-end cases pass: inside the original project and from a sibling directory without pushing; renamed projects from the root and a containing subfolder with the edited files pushed; and a cloud-only project without pushing. The unrelated commented configuration stayed byte-identical throughout.
Other repositories: none need an update.
Risk: I set risk:high. The fallback can still miss a valid local project and silently run stale cloud files with --push; the green tests do not cover this candidate-inspection failure. The earlier configuration-deletion failure itself is fixed.
Checking whether an entry is a folder can fail, such as for a link into a folder without access, and that stopped the search of every entry after it, so a renamed project there was missed and not pushed. Skip only the entry that fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, confirmed and fixed in 8072df6. The helper now lists a folder first, then checks each entry in its own New regression test: a link into a locked folder, listed before a folder holding the renamed project, run as a non-root user. The project is found and pushed, while the previous commit made no push. I also ran the helper on a real filesystem as a non-root user with that layout ( Full suite: 2872 passed, 81 skipped. Static analysis is unchanged. |
Python 3.14's Path.is_dir returns False instead of raising PermissionError, so the test's precondition failed there. The search must still find the renamed project either way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The CI failure on 8072df6 was in my new test, and I've fixed it in 62eb52c. The test first checked that |
There was a problem hiding this comment.
Approving. 8072df6 and 62eb52c resolve the second reviewer's finding on 3d0339d. try_get_project_path_by_cloud_id now lists a folder first and checks each entry in its own try. An entry whose is_dir() raises is skipped by itself, and the search still covers the entries after it. Nothing else in the code changed since my approval of 3d0339d. 62eb52c only drops a test assertion that doesn't hold on Python 3.14.
What I checked on this commit
- CI is green on the head commit: Build and Regression Testing on every platform and Python version. Only the release-only jobs were skipped.
tests/components/cloud,tests/commands/cloudandtests/components/util/test_project_manager.pypass (259) on Python 3.11. With 3d0339d'sproject_manager.pyswapped in, the newfinds_renamed_project_after_an_entry_it_cannot_inspecttest fails.- I also ran the helper on a real filesystem as a non-root user. Two links into a folder without access were listed before
Sub/Renamed. With 3d0339d's helper, the lookup returned nothing. With this commit's helper, it returnedSub/Renamed. An unknown id still returned nothing. - The behavior is unchanged in every other case. A folder that can't be listed is still skipped (
continuenow instead ofpass, with the same result, since nothing follows in the loop). The config read is still read-only. - On Python 3.14,
Path.is_dir()returnsFalseinstead of raising, so the newtrynever fires there and the result is the same. The test asserts only the push, which holds on every version. python static_analysis.py: same five unused-import warnings as on master.
Other repositories: none need an update.
Risk: I set risk:low. This is a small bug fix that is easy to revert. Every path that works on master is unchanged. The helper changes since master remove a side effect (file deletion) and two failures (an unlistable folder, and an entry that can't be inspected hiding the entries after it), and add none. Unit tests fail without each fix, CI is green, and the author's end-to-end runs cover the renamed, cloud-only and outside-the-working-directory cases. The second reviewer's risk:high was set on 3d0339d, not on this commit.
There was a problem hiding this comment.
Approving 62eb52c. The remaining discovery-isolation finding is fixed: each entry's is_dir() now has its own exception handler, so an inaccessible entry no longer hides readable siblings. Configuration discovery remains read-only and shared with PullManager. Both of my earlier findings are resolved.
The Python 3.14 test adjustment is correct: Path.is_dir() now returns False for operating-system errors instead of raising. The test still requires the renamed project to be found and pushed; it exercises the exception-handling regression on Python 3.11.
Verification
- Head CI: 54 successful checks, 6 skipped.
- Affected cloud/project/pull tests: 292 passed, 1 skipped.
- Substituting only
3d0339d's discovery method makes the new inaccessible-entry regression fail because no project update occurs. The current method passes. - Full suite with the source checkout on
PYTHONPATH: 2872 passed, 81 skipped. python static_analysis.py: BUILD PASSED, with the same five unused-import warnings.- Real-filesystem verification as a non-root user: with an inaccessible link enumerated before
Sub/Renamed…, the previous helper returns no project; the current helper finds the renamed project. - Tested end to end with
lean cloud backtest <id> --pushand that inaccessible entry present: the previous helper completes without uploading the local edit; this head uploads the edit, renames the cloud project, compiles and completes the backtest. The unrelated commented configuration stays byte-identical.
The current-directory guard and name-based lookup are unchanged since the previous review. The per-entry change adds no API calls, and the other callers using push=False remain unaffected.
Other repositories: none need an update.
Risk: risk:low. I set this label because this is a small, reversible bug fix, the previously reported regressions are covered, CI is green, and the final failure case now passes end to end.
Closes #720
What changed
lean cloud backtest/optimize/live <cloud id> --pushlooked for the local copy only under the cloud project's current name (get_local_project_path). When the local directory had been renamed, that path didn't exist, so nothing was pushed and the command ran the older cloud files.CloudProjectManager.get_cloud_projectnow falls back to the existingProjectManager.try_get_project_path_by_cloud_idwhen the name-based path doesn't exist, which is the same fallbackPullManageralready uses for renamed directories. The push names projects relative to the current directory, so the fallback uses the project it finds only when that project is strictly under the current directory. Anywhere else, including the current directory itself or a sibling folder, the command behaves as on master: it returns the cloud project without pushing. The name-based lookup still runs first, so the paths that work today are unchanged.try_get_project_path_by_cloud_idused to open everyconfig.jsonin the workspace throughStorage, which deletes a file it can't parse, such as another application's config with comments. It also stopped withPermissionErroron a folder it couldn't list. It now reads each config read-only withjson.loads: a file it can't read or parse is left untouched and doesn't match. It skips a folder it can't list, and an entry it can't inspect (such as a link into a folder without access) is skipped on its own, so the search still covers its siblings. It is still one helper shared withPullManager, solean cloud pullalso stops deleting unrelated configs while it searches for a renamed project. TheStoragedeletion on the other paths described in Standardize Handling of Conflicting Project Names to Local Platform Behavior #374 is unchanged.Testing
test_get_cloud_project_pushes_project_given_by_id_after_its_directory_was_renamed: fails on master (no push) and passes here.test_get_cloud_project_pushes_project_given_by_id_renamed_under_a_workspace_subfolder: uses the realPushManagerand runs from a workspace subfolder that holds the renamed project. It fails on master and passes here.test_get_cloud_project_given_by_id_does_not_push_project_outside_the_working_directory[Python Project|Sub]: uses the realPushManagerand runs from inside the project directory and from a sibling subfolder. It expects no push and no cloud rename. Both fail on this PR's first commit and pass here.test_get_cloud_project_given_by_id_preserves_unrelated_config_it_cannot_parse[True|False]: a renamed project, and a cloud-only project with no local copy, alongsideOther App/config.jsoncontaining a comment. The file must stay byte for byte. Both fail on the previous commit (the file is deleted) and pass here.test_get_cloud_project_given_by_id_skips_directories_it_cannot_list: a folder with mode0111, run as a non-root user. It fails on the previous commit withPermissionErrorand passes here.test_get_cloud_project_given_by_id_finds_renamed_project_after_an_entry_it_cannot_inspect: a link into a folder without access, listed before a folder holding the renamed project, run as a non-root user. The project must be found and pushed. Before 8072df6 it fails on Python up to 3.13, with no push. On 3.14,Path.is_dir()returnsFalseinstead of raising, so the bug can't occur there and the test passes either way. I also ran the helper on a real filesystem as a non-root user with that layout: the previous commit returned nothing, and this one returns the renamed project.test_try_get_project_path_by_cloud_id_returns_project_and_preserves_configs_it_cannot_parse: a direct test of the shared helper. It finds a nested renamed project, returns nothing for an unknown id, and leaves the unparseable config alone. It fails on the previous commit and passes here.python -m pytest -s -rs, with the checkout onPYTHONPATH: 2872 passed, 81 skipped. Without that, the local environment's installedlean1.0.233 makestests/ui/test_app.py::test_command_process_runs_the_cli_as_a_child_processfail, on master too.python static_analysis.py: same output as on master.lean cloud backtest <cloud id> --push:Sub/Renamed…and edited, run fromSub: pushed, renamed in the cloud to match, compiled and backtested. An unrelatedOther App/config.jsonwith a comment kept the same md5.