Repository navigation
Complete quoted matches in one Tab and close the quote on a menu choice - #1831
Open
tleonhardt wants to merge 9 commits into
Open
tleonhardt wants to merge 9 commits into
tleonhardt wants to merge 9 commits into
Conversation
Fixes the two problems tracked in #1821, which affect any match that needs quotes, such as a file name with a space. A second Tab was needed after the opening quote was inserted. Cmd2Completer.get_completions() inserted the quote into the buffer and returned no matches, expecting prompt-toolkit to start a new completion. It never did: prompt-toolkit discards a completion whose input changed while it ran and only restarts when text was added to the end of the input, and the quote goes in front of the search text. get_completions() now leaves the buffer alone during the completion which found that a quote is needed. Once prompt-toolkit has finished that completion, the quote is inserted on the event loop and completion is started again with the quote in place, so the common prefix of the matches can be inserted. The matches are kept for that second completion, so cmd2's completion logic runs once and a completion table is printed once. Nothing is inserted if the input changed in the meantime. The buffer used to be edited from the completion thread. Choosing a match from the menu left the quote unclosed, so the command failed with "Invalid syntax: No closing quotation". Each match offered in a menu now closes an open quote. A directory doesn't, so the rest of the path can still be completed. path_complete() marks directories with a private CompletionItem._allow_finalization field, because a directory is offered without a trailing separator when one follows the cursor. tests/test_completion_keys.py presses the completion key at a prompt which reads from a pipe, so the key bindings, cmd2's completion, and prompt-toolkit's handling of the results are tested together. Co-authored-by: Kevin Van Brunt <kmvanbrunt@gmail.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1831 +/- ##
========================================
Coverage 99.75% 99.76%
========================================
Files 23 23
Lines 6644 6750 +106
========================================
+ Hits 6628 6734 +106
Misses 16 16
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
Don't add a closing quote in front of one which is already there. A completion only replaces text before the cursor, so with the cursor in front of the closing quote of `edit "sp"`, choosing a match from the menu gave `edit "sp ace.txt""`. A single match had the same problem before this branch, and it is fixed the same way. Let a finished match close its quote when delimiter_complete() offers it alongside a match which has more segments. delimiter_complete() set allow_finalization=False for the whole result if any match was unfinished, which kept every match in the menu from closing the quote. It now marks each unfinished match instead, as path_complete() does for directories, and only returns allow_finalization=False for a single unfinished match. Cache the matches for the restarted completion only once it is certain to start. They were cached before checking whether the input had changed, so when the user edited the line and the restart was abandoned, a later request for the same quoted text could reuse them.
Don't add a closing quote when text after the cursor already closes the quote. The previous fix only looked at the character right after the cursor, so with the cursor after `sp` in `edit "sp.bak"`, choosing a match from the menu gave `edit "sp ace.txt".bak"` and split the argument. A closing quote is now only added when the line needs one: an odd number of the quote character after the cursor means the first of them closes this quote and the rest are in pairs. Don't insert the quote for a request which was cancelled. A cleared complete_state was taken to mean that prompt-toolkit had finished the request, but cancelling a slow completion clears it while the request is still running. The quote was then inserted, prompt-toolkit ignored the completion started after it, and the cached matches were left for the next request. get_completions() now checks before it returns whether its request is still the active one, and does nothing if it was cancelled or the line was edited. The matches are no longer cached on the completer. They are passed to the restarted completion in its CompleteEvent, so they are dropped with that request if prompt-toolkit doesn't run it and no other request can be given them.
Start completion again after inserting an opening quote only for the request which comes from the completion key. prompt-toolkit doesn't tell a completer what a request will do with its matches, so starting again as a request for the common prefix changed what every other request does: Ctrl-P in Vi insert mode no longer selected the last match, and bindings which ask for completions directly got none. cmd2 now binds the default completion key itself, in place of prompt-toolkit's binding for Tab, and marks its request with a CompleteEvent of its own. Every other request gets the opening quote as part of each match and goes on to do what it was started for. Don't start again for a request which was cancelled just before prompt-toolkit finished it. The request is now only treated as finished if prompt-toolkit was seen to reach its end with the request's state intact, which a cancelled or outdated request never is. A completion which runs in a thread waits for the event loop to be watching the request before it returns, so prompt-toolkit can't finish it first. Always close a quote which cmd2 opened. Nothing after the cursor can close it, but a quote character in a later argument was taken for its closing quote, so completing `my` in `shell cat my 'a"b'` left the quote open. For a quote which the user typed, quote characters inside an argument quoted with the other character are no longer counted. Add a trailing space after a match which closes its quote in the menu. Without it, pressing Tab after accepting the match completed a new argument right against the closing quote. Don't remove the event handler while prompt-toolkit is calling the event's handlers, which made it skip the next one, and remove it when the prompt ends while a completion is still running. Document that a completer whose matches can be unfinished, like directories, should return allow_finalization=False.
Replace the count of quote characters after the cursor, which misread apostrophes and quotes in other arguments in both directions. A quote is now taken to be closed already in these cases: - The word under the cursor ends with the quote character, as in `edit "sp|.bak"`, or `edit sp|"` where the opening quote is missing. - The user typed the opening quote, so the argument can have whitespace in it, and the rest of the line has a quote character at the end of an unquoted word, as in `edit "sp| 'copy"`. The rest of the line is read with the parser's own quoting rules, so the quotes of another quoted argument and an apostrophe in the middle of a word don't count. A quote which cmd2 adds goes around what was one unquoted word, so nothing past that word closes it. `edit my| 5" nail` and `shell cat my| 'a"b'` both get their closing quote. Don't let a buffer which completes while typing start a completion of its own when the opening quote is inserted. prompt-toolkit ran that one and ignored the completion which cmd2 started for the matches it had found. Leave Tab to prompt-toolkit when the session shows completions like readline, which has a binding of its own for it. Don't finalize the only match when it is marked as one the user will keep completing. A completer which filters the results of path_complete() or delimiter_complete() down to one directory-like match got a trailing space after it. Say in the CHANGELOG that Completions.allow_finalization now applies when there are several matches.
Treat the redirection characters and the terminators as the end of a word when deciding whether a quote is already closed, as the parser does. With the cursor after `sp` in `shell cat "sp.bak">out`, the closing quote was not seen because `.bak">out` has no whitespace in it, and choosing a match added a second quote at the cursor. The text after the cursor is now split by a function which follows the parser's quoting rules and knows about punctuation, in place of shlex. A quote right after a redirection character starts a quoted argument, so the quotes of `>"out file"` are not taken for a closing quote. Put the opening quote in the buffer right away when a key binding asks for completions itself, as the binding which lists completions like readline does. That binding reads the buffer again once it has the completions, so it can insert the text the matches share with the quote in place. It used to get each match with the quote in front, which changes text before the cursor, so it only listed the matches and left the line as it was. This is only done from the event loop, when prompt-toolkit is not running a request and the buffer holds the line being completed.
… case Keep the opening quote when a key binding asks for completions itself and the word follows a shortcut. The quote goes after the shortcut, but the completions still replaced the shortcut and the quote, so in a readline-style session `@my` became `@my file.txt" `. Don't insert the quote into a prompt which has already ended. If the line was submitted in the instant between prompt-toolkit finishing the request and cmd2 inserting the quote, the last redraw showed a quote which the returned line didn't have. Bind Tab to cmd2's own request whatever the completion key is set to, with a custom key bound alongside it. With a custom completekey, Tab was still prompt-toolkit's binding, so it only opened the menu for several matches which need a quote instead of inserting the quote and the text they share. The custom key itself still only opens the menu on its first press.
Take a quote character anywhere in an unquoted word for a closing quote, not only one at the end of the word. With the cursor after `sp` in `shell cat "sp backup".txt`, the quote which closes the quoted part of the file name was missed and a second one was added at the cursor. A single quote with a letter on each side is an apostrophe and is still left alone, as in `don't`. Only put the opening quote in the buffer for a request made directly by a key binding when the session lists completions like readline. The Meta-* binding of Emacs mode asks the same way but inserts every match after the cursor, so it got the quote in the buffer and then matches without one: `shell cat "spsp ace.txt" spare.txt" `. It now gets each match with its own quotes. The session's style is read before cmd2's completion logic runs, since that sets the session to a menu style. Tell the buffer's listeners when the opening quote is inserted, whether or not the buffer completes while typing, so auto-suggest is asked about the new text. Only the buffer's own completion while typing is held off for that insertion, which is what took the place of the completion cmd2 starts.
Replace the check which read the rest of the line to decide whether a quote was already closed. Whether a quote character further along is this argument's closing quote, part of another argument, an apostrophe or an inch mark can't be told, and each revision of the check got some line wrong which main gets right. The latest was `shell cat "my| --title="x"`, which lost its closing quote. A closing quote is now only added where it is certain to be needed, and everywhere else the line is treated as it was before this branch: - The only match always closes its quote, as on main, unless the next character is already that quote. That exception fixes `edit "my|"` becoming `edit "my file.txt""`. - A match chosen from several closes its quote at the end of the line, or when cmd2 opened the quote around a word which ended at the cursor. Otherwise nothing is added, as on main. This removes the argument splitter, the apostrophe rule and their table of cases. In 17 lines taken from the reviews, main gets 12 right, the previous check 16 and this one 15, and this one is never worse than main. The two it gives up are a quote the user typed which is not closed anywhere, completed from the menu in the middle of a line, such as `edit "sp| other`. They are left unclosed as they are on main.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1821.
Fixes the two problems that issue tracks. Both affect any completion which needs quotes, such as a file name with a space, with any command which completes paths or choices.
Changes
One Tab is enough when a match needs an opening quote
Previously the first Tab only inserted the quote and a second Tab was needed to complete the name:
Cmd2Completer.get_completions()inserted the quote into the buffer and returned no matches, expecting prompt-toolkit to start a new completion. It never did. prompt-toolkit discards a completion whose input changed while it ran, and it only starts another one when text was added to the end of the input. The quote goes in front of the search text.Now:
get_completions()leaves the buffer alone during the completion which finds that a quote is needed. When prompt-toolkit has finished that completion, the quote is inserted on the event loop and completion is started again with the quote in place, so the common prefix of the matches can still be inserted.CompleteEvent, soCmd.complete()runs once for one Tab and a completion table is printed once.cmd2starts. The buffer's other listeners are told about the quote as usual, so auto-suggest is asked about the new text.Only the completion key's request is started again
prompt-toolkit doesn't tell a completer what a request will do with its matches. Starting again always asks for the common prefix to be inserted, which is what Tab does, so it is only right for a request which comes from Tab.
cmd2now binds Tab itself, in place of prompt-toolkit's binding for it, whatevercompletekeyis set to. It does what that binding does, and marks its request with aCompleteEventof its own so the completer can recognize it. A customcompletekeyis bound alongside Tab, as before.CompleteStyle.READLINE_LIKE), which has a Tab binding of its own.completekey.A match chosen from the menu closes its quote
Previously the closing quote was only added for a single match, so choosing one from the menu and running the command failed:
Now each match offered in a menu at the end of the line closes an open quote and adds a trailing space, like a single match does. A match which the user will keep completing does neither, so the rest of it can still be completed:
path_complete()marks directories with a new private field,CompletionItem._allow_finalization. A trailing separator can't be used to recognize them, because a directory is offered without one when a separator already follows the cursor. The field is not part of an item's equality or hash.delimiter_complete()marks each match which has more segments in the same way. A finished match offered alongside one, such assome filenext tosome dir/, closes its quote.In the middle of a line, a closing quote is only added where it is certain to be needed
A completion only replaces text before the cursor, so the argument may carry on after it. Whether a quote character further along closes this argument, or belongs to another argument, or is an apostrophe or an inch mark, can't be told. An earlier version of this PR tried to work that out, and each revision got some line wrong which
maingets right (see "Fixes made during review"). So nothing depends on it now:main, unless the next character is already that quote.cmd2opened the quote around a word which ended at the cursor. Anywhere else nothing is added, as onmain.In 17 lines taken from the reviews,
maingets 12 right and this gets 15, and it is never worse thanmain. The two it gives up are the last line above andedit "sp| "spare.txt": a quote which the user typed and which is not closed anywhere, completed from the menu in the middle of a line.Documentation and tests
Completions.allow_finalizationnow applies when there are several matches.tests/test_completion_keys.pypresses keys at a prompt which reads from a pipe, so the key bindings,cmd2's completion logic, and prompt-toolkit's handling of the results are tested together. It covers one Tab for a single match (with and withoutcomplete_in_thread), the shared prefix, shell arguments,read_input(), a customcompletekey, Ctrl-P in Vi insert mode, shortcuts, directories,delimiter_complete(), completing in the middle of a line in front of a closing quote, another quoted argument, an apostrophe, an inch mark or a redirection, a readline-style session, the history search, and a prompt which ends while a completion is running. These tests are skipped on Windows, like the other tests which use a pipe for input.tests/test_pt_utils.pyrun prompt-toolkit's real completion code. They cover a request which prompt-toolkit is still waiting on, one which is cancelled or outdated whilecmd2is completing, one which is cancelled just before prompt-toolkit finishes it, requests which don't come from the completion key, a buffer which completes while typing, auto-suggest, the reuse of the matches, and other listeners on the buffer's event.Fixes made during review
These are in the second to ninth commits.
edit "sp|"becameedit "sp ace.txt"". Later revisions tried to work out from the rest of the line whether the quote was already closed: counting quote characters, skipping the ones inside other quoted arguments, reading the line with the parser's rules, treating redirections as the end of a word, and telling apostrophes from quotes. Each one fixed the lines reported against it and got another wrong, among themedit "sp.bak",shell cat my 'a"b',edit "sp 'copy",echo 'my don't,edit my 5" nail,shell cat "sp.bak">out,shell cat "sp backup".txtandshell cat "my --title="x". The ninth commit replaces all of that with the two rules above, which get every one of those lines right.delimiter_complete()kept every match open. It returnedallow_finalization=Falsefor the whole result when any match had more segments, sopick "some filestayed open becausesome dir/was also offered.edit "sp ace.txt"followed by Tab gaveedit "sp ace.txt"-h. The trailing space prevents that.complete_while_typing=True, inserting the quote started an automatic completion which prompt-toolkit ran in place of the onecmd2started, so Tab leftedit "my.cmd2's binding took precedence over prompt-toolkit's binding for that style.edit loas it was. It now getsedit "long namein one Tab.delimiter_complete()no longer returnsallow_finalization=Falsefor several matches, so a completer which filtered its results down to one unfinished match got a trailing space after it.shell cat "spsp ace.txt" spare.txt". Putting the quote in the buffer is now limited to readline-style sessions.complete_while_typingon, the insertion didn't notify the buffer's listeners, so no new suggestion was asked for until the next key press.@mybecame@my file.txt".completekeyonly opened the menu. Tab was still prompt-toolkit's binding there, so for several matches which need a quote it inserted neither the quote nor the text they share.Behavior changes worth a look
cmd2binds Tab. It replaces prompt-toolkit's binding for Tab and behaves the same way, including only being active while text can be inserted. This is the case whatevercompletekeyis set to. An application which adds its own binding for Tab to the session's key bindings is affected by the order in which the two are added.cmd2added around a word that ended at the cursor. A match which needs no quote is unchanged.Completions.allow_finalization=Falsenow also keeps the matches in a menu from closing a quote. It was ignored for multiple matches before.delimiter_complete()returnsallow_finalization=Truefor several matches even when some have more segments. It used to returnFalse, which had no effect for several matches before this PR.path_complete()ordelimiter_complete()marked as unfinished is not finalized even when it is the only match. Before, onlyCompletions.allow_finalizationdecided that.completekeyis unchanged: its first press opens the menu and inserts nothing, whether or not the match needs a quote. Tab behaves the same with or without one.on_completions_changedwhen it has all the results of a request, then clearing the completion state and notifying again when there were none. If a future prompt-toolkit release changed that, Tab on a name which needs a quote would stop completing. The new unit tests run that code path, so CI would catch it.Known limits
edit "sp| otherstays unclosed. A single match inside a quoted word whose closing quote comes later still gets a second one, soedit "my|.bak"becomesedit "my file.txt".bak". Both are whatmaindoes.CompleteStyle.READLINE_LIKEonly lists matches that way on the first Tab.Cmd.complete()sets the session's style to one of the menu styles on every completion, as it does onmain. That is not changed here.cmd2can't mark individual matches as unfinished, because the field is private. The docs and CHANGELOG now say that such a completer should returnallow_finalization=Falsewhen any of its matches is unfinished, which keeps every match in the menu open, as before this PR. Making the field public would be an API addition.Testing
make check,make test(3188 passed, 3 skipped on macOS), andmake docs-testpass locally.edit my, that a menu choice closes its quote and the command then runs, that a directory stays open and completion continues inside it, that the only match closes its quote in front of--title="x"and5" nail, that a menu choice in the middle of a line closes the quote inedit sp| otherand adds nothing inshell cat "sp| backup".txtandedit "sp| other, and that a customcompletekeyopens the menu on its first press while Tab still inserts the quote and the shared text.complete_while_typingwere only exercised by the tests.