Skip to content

fix(web): synchronize iOS keyboard edits (#830) - #839

Open
HadrienKerlero wants to merge 10 commits into
Ark0N:masterfrom
HadrienKerlero:fix/830-ios-keyboard
Open

HadrienKerlero wants to merge 10 commits into
Ark0N:masterfrom
HadrienKerlero:fix/830-ios-keyboard

Conversation

@HadrienKerlero

Copy link
Copy Markdown

Fixes #830

Keep xterm's helper textarea in sync with native iOS editing instead of supplementing xterm's first-character keypress output.

Changes

  • Give iOS WebKit one native-input owner per main/split/grid terminal. Other platforms retain the existing keyCode-229 recovery.
  • Reconcile suggestion insertion, word/phrase replacement, keyless accelerated deletion, dictation and composition as one ordered edit transaction; send individual DEL operations before replacement text.
  • Preserve ordinary spaces with white-space: pre. Keep dictated line breaks editable in the native field but project them to spaces, as requested by the reporter; only explicit Enter submits.
  • Show the native trackpad caret with a visual marker, without injecting arrows into the receiving CLI. Local-echo geometry comes from the overlay's actual layout.
  • Capture session destinations and reserve existing delayed Enter/paste operations in a per-session FIFO. Reset native ownership at toolbar/programmatic-input boundaries, including background insertion into a live unfocused pane.
  • Remove complete Unicode code points from local echo without changing its existing UTF-16 offset bookkeeping.

Clean-code review

Device acceptance

The reporter tested the standalone probe and then the integrated isolated beta on their iPhone and confirmed the remaining keyboard features work. Copy/paste feature work was explicitly excluded; existing Codeman clipboard features are preserved.

The browser replays preserve the logged iOS event order, multi-character key values, first-letter charCode, and composed: true. They are regression coverage, not a claim that Chromium emulates the iOS keyboard.

Delivery boundary

The FIFO guarantees session destination and enqueue ordering through Codeman's existing reliable transport. It does not introduce new ACK/completion guarantees or redesign reconnect, wake-buffering, or named-key transport behavior.

Verification on current upstream master

  • npm test: 757 test files passed, 17,284 tests passed; 1 file / 13 tests skipped by the repository's existing conditions.
  • Browser input/IME/split/voice suites: 5 files, 55 tests passed, including 23 iOS integration cases.
  • Overlay package regression suites: 3 files, 165 tests passed.
  • npm run build, typecheck, lint, format:check, check:frontend-syntax, check:frontend-dead, check:browser-excludes, check:public-assets, and check:lockfile: passed.
  • Codeman's pre-commit formatting hook passed. The branch is based on current upstream master (e99b42c3), not the reporter's modified installation.

@Ark0N

Ark0N commented Nov 19, 2026

Copy link
Copy Markdown
Owner

Thank you for this, and for the device testing with the reporter. The PR gives each terminal pane on iOS a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit. It also adds a per-session FIFO that keeps delayed Enter and paste writes with the session they were typed for. The gate passes here (757 files, 17,284 tests), as do the new iOS browser suite and the overlay package. A few things need to change before merge:

  1. Slash-command keys now submit the unsent draft (src/web/public/keyboard-accessory.js:1350). sendCommand() passes flushEcho: true, so with local echo on (every phone, Android too), a draft hel followed by a /compact tap goes out as the prompt hel/compact and the command never runs. On master the command went to an empty composer and the draft stayed in the overlay. Please drop flushEcho there (keep the plain _prepareInput so the iOS owner still settles) and change the command case in test/terminal-input-fifo.test.ts:671 to expect only /help then \r. The flush before Esc, Opt+Enter, Tab and the arrows matches the typed path, so that part can stay.

  2. Some out-of-gate tests still stub the old send path. test/shift-enter-keypress.browser.test.ts fails 2 of 5: pressMain() (line 111) stubs _sendInputAsync and uses a probe session that is not in app.sessions, and _enqueueInput now drops that session's input. The nightly browser suite is green on master, so this would turn it red. test/mobile/keyboard.test.ts fails 7 more cases than master for the same reason (local-echo typing and the six one-shot Ctrl shell-bar cases). With the stubs retargeted to _enqueueReliableInput and the probe session added to app.sessions, shift-enter passes 5/5 and the mobile file is back to master's own failures. Please commit those harness updates, as you already did for terminal-keycode229-recovery.browser.test.ts.

  3. Please keep the fix(terminal): Shift+Enter no longer submits after inserting a newline #520 guard in the gate. test/shift-enter-keypress-swallowed.test.ts was the only npm test check that the Shift/Ctrl+Enter gate swallows keypress as well as keydown. Restore it with a pattern that accepts your iOS clause: match the ev.key === 'Enter' && (ev.shiftKey || ev.ctrlKey) prefix, and keep the check that the condition never tests ev.type. The browser test header (lines 17-18) still points at it.

  4. The overlay lost its prompt fallback (packages/xterm-zerolag-input/src/zerolag-input-addon.ts:643). else if (!this._lastPromptPos) became else, so the overlay now hides whenever the finder returns null instead of painting at the last known prompt. The OpenCode finder returns null whenever the cursor row has no ┃, so on every touch device, unsent OpenCode text blinks out during redraws. Please keep the fallback (the anchor can still come from that layout), or explain why it has to go and confirm OpenCode typing on an Android device.

  5. The new xterm privates need pinning. terminal-ios-native-input.js requires _inputEvent, _handleTextAreaBlur, _finalizeComposition and _compositionView. If an xterm upgrade renames one, create() returns null, iOS gets neither the owner nor the recovery, and iPhone keyboard: tapping a suggestion only sends its first letter, and hold-delete never goes word by word #830 comes back with no signal. Add them to the shipped-bundle name list and the version message in test/xterm-private-api.test.ts (around line 72).

  6. Docs: src/web/CLAUDE.md:81 still says the composer delivers through _sendInputAsync. The per-session FIFO belongs in docs/architecture-invariants.md under "Input delivery and WS resilience" (gaps order enqueueing, not ACKs; ids missing from app.sessions are dropped; delete clears the FIFO). docs/ios-native-input.md:20 points at IOS_DIAG.md, which is not in this repository, so either link the sequence in iPhone keyboard: tapping a suggestion only sends its first letter, and hold-delete never goes word by word #830 or drop the sentence.

Push the fixes to this branch and the PR goes back into review.

HadrienKerlero added a commit to HadrienKerlero/Codeman that referenced this pull request Nov 20, 2026
Keep the unsent draft out of slash-command keys; settle the iOS owner without flushing local echo.
Restore the overlay last-prompt fallback and the Ark0N#520 Shift/Ctrl+Enter gate guard.
Pin the xterm privates the iOS owner needs; retarget browser/mobile harnesses to the session FIFO edge.
Document the session FIFO invariants and link Ark0N#830 for the device sequences.
@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for the thorough review. All six points are addressed in 89d6a9e on this branch.

  1. Slash commands keep the draft. sendCommand() now calls the plain _prepareInput(undefined, 'accessory-command'), so the iOS owner still settles but local echo is not flushed. The command case left the shared producer table in test/terminal-input-fifo.test.ts and became its own test: with echo on, the wire carries only /help then \r, and hel stays in the overlay. With echo off, hel had already been typed into the PTY, as on master. The flush before Esc, Opt+Enter, Tab and the arrows is unchanged.
  2. Out-of-gate harnesses. test/shift-enter-keypress.browser.test.ts: pressMain() stubs _enqueueReliableInput, registers the probe session in app.sessions and removes it afterwards, and clears only that id from the _pendingInput map instead of overwriting the map with ''. Result: 5/5. test/mobile/keyboard.test.ts: all ten send stubs now target _enqueueReliableInput (the probe sessions there were already registered). Result: 5 failures, a subset of master's own 6 measured on e99b42c3 in the same run. The extra codex write-through failure passes now.
  3. fix(terminal): Shift+Enter no longer submits after inserting a newline #520 guard restored in the gate as test/shift-enter-keypress-swallowed.test.ts. It matches the if (ev.key === 'Enter' && (ev.shiftKey || ev.ctrlKey)…) { gate in both terminal-ui.js and terminal-tile.js, accepts the trailing iOS clause, and fails if that gate's condition tests ev.type. The browser test header points at it again.
  4. Overlay prompt fallback restored (else if (!this._lastPromptPos)). The anchor still comes from the renderer layout, so while the finder briefly returns null, the overlay and the iOS caret both stay at the last known prompt. The anchor test that expected null now asserts that the overlay stays visible at the same anchor through a prompt-less redraw.
  5. xterm privates pinned. _inputEvent, _handleTextAreaBlur, _finalizeComposition and _compositionView are in the shipped-bundle name list of test/xterm-private-api.test.ts. They are also named in its version message and failure message, along with what breaks if they go missing.
  6. Docs. src/web/CLAUDE.md now describes Compose delivery through the session FIFO (_enqueueInput / _sendBracketedPaste()). docs/architecture-invariants.md has a Session FIFO paragraph under "Input delivery and WS resilience": gaps order enqueueing, not ACKs; ids missing from app.sessions are dropped; _cleanupSessionData() clears the FIFO and its gap timer. docs/ios-native-input.md now links the logged sequences in iPhone keyboard: tapping a suggestion only sends its first letter, and hold-delete never goes word by word #830 instead of IOS_DIAG.md.

Verification on this branch:

  • npm test: 758 files passed, 17,286 tests passed (1 file / 13 tests skipped by existing conditions).
  • Browser suites (ios-native-input, terminal-keycode229-recovery, shift-enter-keypress, split-pane-terminal, voice-input): 5 files, 52 tests passed.
  • test/mobile/keyboard.test.ts: 48 passed, 5 failed; the 5 are among master's own 6 failures (e99b42c3, same machine, same run).
  • Overlay package zerolag-input-addon.test.ts: 94 passed.
  • npm run build, typecheck, lint, format:check, check:frontend-syntax, check:frontend-dead, check:browser-excludes, check:public-assets, check:lockfile: passed.

@Ark0N

Ark0N commented Nov 20, 2026

Copy link
Copy Markdown
Owner

Thanks for the quick turnaround on the first review. This PR gives each iOS terminal pane a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit, and it moves browser input onto a per-session FIFO.

All six points from the first round are resolved. I re-ran everything here. The gate passes (758 files, 17,286 tests) and the static checks are green. The overlay package passes (301 tests), and so do the iOS, Shift+Enter, keyCode-229, split, voice and IME preview browser suites (64 tests). test/mobile/keyboard.test.ts now fails only 5 cases, all of them among master's own 6.

One regression needs a fix before merge:

  1. Typing on iOS no longer scrolls back to the prompt (src/web/public/terminal-ios-native-input.js:316). Text the native owner reconciles never goes through xterm's triggerDataEvent(data, true), so xterm's scrollOnUserInput doesn't run for it. I measured it in the served app with your replay fixture. With the terminal scrolled to the top, typing h on master moves the viewport to the bottom (viewportY 0 to 272). On this branch the h is sent and the viewport stays at 0. With local echo on, the overlay holds the text but stays hidden because the prompt row is off screen, so the user types blind. Android is unaffected, since the keyCode-229 path still goes through xterm.

    This hits every iOS session where a touch drag scrolls xterm's own scrollback (shell, OMP, OpenCode and the other CLIs without mouse tracking), in the main pane and in tiles. One line in reconcile(), just before sendTransaction(ops), covers both hosts:

    if (terminal.options && terminal.options.scrollOnUserInput !== false && typeof terminal.scrollToBottom === 'function') terminal.scrollToBottom();

    With that line, the same probe returns the viewport to the bottom with local echo on and off. Please add a case to test/ios-native-input.browser.test.ts that scrolls up, types a character and asserts buffer.active.viewportY === buffer.active.baseY.

Two smaller things:

  1. _updateLocalEchoState() (src/web/public/terminal-ui.js:4425) adds a new session.mode !== 'shell' && session.mode !== 'codex' test. The root CLAUDE.md asks for no new CLI-id branches outside stock.ts, and the same function already makes this decision when it computes policy. Please compute policy first and use const shouldEnable = policy === 'buffer';, so the decision is made in one place.
  2. Docs: nothing points at docs/ios-native-input.md yet, so please link it from the iOS paragraph in src/web/CLAUDE.md:35. The split-pane paragraph at src/web/CLAUDE.md:13 still says Pane B wires the keyCode-229 controller, but on iOS it now gets the native owner, so that line needs an update too.

Push the fixes to this branch and the PR goes back into review.

@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for catching the scroll regression and for measuring it in the served app. All three points are addressed in cb09e25 on this branch.

  1. Typing on iOS scrolls back to the prompt again. reconcile() in terminal-ios-native-input.js now calls terminal.scrollToBottom() just before sendTransaction(ops), when scrollOnUserInput is not false. It only runs when there is something to send, and a throw is recorded instead of blocking the send. The main pane and tiles each pass their own xterm, so the one line covers both. New case in test/ios-native-input.browser.test.ts, run with local echo off and on: write 200 lines, scrollToTop(), type h through the replay fixture, then assert buffer.active.viewportY === buffer.active.baseY. With the fix stashed, both cases fail (expected 0 to be 172); with it, they pass.
  2. One echo decision. _updateLocalEchoState() computes policy first and uses const shouldEnable = policy === 'buffer';, so the mode check lives in one place. The later policy block now only assigns _localEchoPolicy and clears predictions. Behavior is unchanged: shell and codex are off, other modes buffer when echo is on.
  3. Docs. The iOS paragraph in src/web/CLAUDE.md links docs/ios-native-input.md. The split-pane paragraph now says Pane B gets the iOS native-input owner on iOS WebKit touch, and the keyCode-229 controller otherwise.

Verification on this branch:

  • npm test: 758 files passed, 17,286 tests passed (1 file / 13 tests skipped by existing conditions).
  • Browser suites (ios-native-input, terminal-keycode229-recovery, shift-enter-keypress, split-pane-terminal, voice-input, mobile-ime-preview): 6 files, 62 tests passed, including the 2 new scroll cases.
  • npm run build, typecheck, lint, format:check, check:frontend-syntax, check:frontend-dead, check:browser-excludes, check:public-assets, check:lockfile: passed.

@Ark0N

Ark0N commented Nov 20, 2026

Copy link
Copy Markdown
Owner

Thanks for the third round, Hadrien, and for turning the scroll fix around so quickly. This PR gives each iOS terminal pane a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit. It also moves every browser input producer onto a per-session FIFO.

All three points from the last round are in. Typing scrolls back to the prompt, and the new browser case covers it with echo on and off. The overlay switch now comes from policy, and the docs link docs/ios-native-input.md.

I re-ran everything on cb09e25. The typecheck, lint, format, frontend syntax, dead-code, browser-excludes, public-assets and lockfile checks are green. npm test passes (758 files, 17,286 tests), and so does the overlay package (301 tests). The iOS, keyCode-229, Shift+Enter, split, voice, IME preview and tile scroll browser suites pass (65 tests). test/mobile/keyboard.test.ts fails the same 5 cases as last round.

Two things need to change before merge.

  1. The branch conflicts with master. feat(mobile): Send on Enter for the pinned CJK input field #820 (Send on Enter for the CJK field) landed after your base and changed the same input boundaries, so src/web/public/input-cjk.js, terminal-ui.js, app.js and src/web/CLAUDE.md conflict. GitHub runs no CI while they do. When you rebase:

    • CjkInput's new _submit() and sendPrompt path need your captured target (_targetForSend()).
    • Master's _sendCjkPrompt() in terminal-ui.js still sends to this.activeSessionId with a raw setTimeout Enter. It should take that target and reserve the text, the 120 ms gap and Enter on the session FIFO, the way _sendComposedPrompt() does.
    • _cleanupPreviousSession() should keep master's keepCjkDraft / stashDraft handling, and clearTerminalInput() should keep CjkInput.clear({ discard: true }).
    • Two spots merge without a conflict marker and still break feat(mobile): Send on Enter for the pinned CJK input field #820. The gate catches neither:
      • src/web/public/session-ui.js:2075: your submitFromToolbar() early return sits above master's cjk-input-visible branch. The phone Enter button then never calls CjkInput.submit(); it sends a bare Enter and the field's text stays unsent. Master's guard in test/cjk-hold-per-device-setting.test.ts only checks the source text, so it still passes. Keep the CJK branch first, or move it into submitFromToolbar().
      • src/web/public/app.js:10067: the CjkInput.clear() you added in the same-session reload block wipes a held Send on Enter draft, which master keeps on that path with keepCjkDraft: reloadingSameSession. Drop that line after the rebase.
  2. Voice compose Send now submits the unsent draft together with the dictation (src/web/public/voice-input.js:1083). _sendToTarget() calls _prepareExternalInput(pane, 'voice-send', { flushEcho: true }), and in compose mode with local echo on, that flushes the typed draft ahead of the reviewed text. I checked this in the served app: with hel in the overlay, Send put hel and then Please fix the bug\r on the wire, so the prompt went out as helPlease fix the bug. On master the dictation went out alone and hel stayed in the overlay. It's the same case as /compact in the first round. Compose Send is the only path where flushEcho changes anything here: direct mode puts active-session text into the overlay instead, and with echo off there is nothing to flush. Please drop flushEcho at that line and keep the plain _prepareExternalInput so the iOS owner still settles. Then add a compose-mode case to test/voice-input-target.test.ts asserting that only the dictated text and \r are sent and the draft stays in the overlay.

Two small ones while you're in there:

  1. docs/architecture-invariants.md:1084 (split-pane section) still says Pane B carries its own keyCode-229 controller. Please give it the iOS wording you added to src/web/CLAUDE.md:13.
  2. test/image-paste-trap.test.ts:209 still stubs _flushLocalEchoPending, which this PR removed. Please drop the stub.

Push the rebase and the fixes to this branch and the PR goes back into review.

Keep one native-input owner per iOS terminal and reconcile keyboard edits without duplicate producers.
Preserve session-bound input ordering, local-echo Unicode deletion, and visual caret geometry.
Cover device event sequences and lifecycle handoffs; remove obsolete helpers and source-text-only regression tests.

Fixes Ark0N#830
Keep the unsent draft out of slash-command keys; settle the iOS owner without flushing local echo.
Restore the overlay last-prompt fallback and the Ark0N#520 Shift/Ctrl+Enter gate guard.
Pin the xterm privates the iOS owner needs; retarget browser/mobile harnesses to the session FIFO edge.
Document the session FIFO invariants and link Ark0N#830 for the device sequences.
The native owner bypasses xterm triggerDataEvent, so scrollOnUserInput never ran; scroll before sending.
Derive the local-echo overlay switch from the echo policy instead of a second mode check.
Link docs/ios-native-input.md and describe Pane B input ownership on iOS.
…draft (Ark0N#830)

Held-mode Ctrl, Backspace and restored drafts send to the session their text was typed for; a Ctrl-only field releases it.
Voice compose Send settles the iOS owner without flushing the unsent local-echo draft.
Cover Send on Enter across focus moves, the toolbar Enter in the CJK field and the same-session reload.
Describe Pane B iOS input ownership; drop a stub for a removed helper.
@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for the careful third round, especially the two silent merges with #820. The branch is now rebased onto current master (5548c17c, with #820) and force-pushed. The conflicts are resolved in the first commit, and this round's fixes are in ddb5a64.

1. Rebase onto #820

  • CJK sends use the captured target. CjkInput.init takes captureTarget, send(text, sessionId) and sendPrompt(text, sessionId). _submit() hands the target from _targetForSend() to send for live text and bare Enter, and through _sendHeld() to sendPrompt for a held answer. _resetToPhantom() clears _heldBase and the held target.
  • _sendCjkPrompt(text, targetSessionId) uses the captured session (falling back to the focused pane) and refuses when that session is no longer in app.sessions. A single character goes on that session's FIFO as [text, gap 120, '\r' with useMux]. A lone digit while a permission or question dialog waits stays text only, and an armed Ctrl is still spent. Longer text keeps going through _sendComposedPrompt(), which reserves the paste, the 120 ms gap and Enter on the same FIFO. Master's raw setTimeout Enter is gone.
  • _cleanupPreviousSession() keeps master's keepCjkDraft / stashDraft handling, now ahead of _settleMainInput('session'), and the duplicate CJK clear is gone. clearTerminalInput() runs _prepareExternalInput(pane, 'clear') and then CjkInput.clear({ discard: true }).
  • session-ui.js sendEnterKey(): the cjk-input-visible branch now comes first, then submitFromToolbar(). The source-text guard in test/cjk-hold-per-device-setting.test.ts is now a behavioral test: it runs the real sendEnterKey with the field shown and hidden, and asserts that CjkInput.submit() or submitFromToolbar() runs exactly once.
  • app.js same-session reload: the extra CjkInput.clear() is dropped, so keepCjkDraft: reloadingSameSession keeps a held draft again. The same-session guard in that test file now also fails if anything in the reload block calls CjkInput.clear.
  • A restored draft keeps its session too. restoreDraft(sessionId) now binds the held target to that session. Before, a draft stashed on a switch and restored on the way back had no target, so after a focus move to Pane B, Enter without an edit sent A's draft to B. A second case covers restore → focus move → Enter, and fails without the binding.
  • Held-mode Ctrl and Backspace keep the target. feat(mobile): Send on Enter for the pinned CJK input field #820's one-shot Ctrl in a held field (_holdSettle()) and the Backspace that passes through on an empty field (_holdInput()) now send to the captured session too. When Ctrl takes the only character, the target is released, so the next answer captures the pane it is typed in. Two cases cover a Ctrl key across a focus move and a fresh answer after a Ctrl-only field; each fails with its fix reverted.
  • New end-to-end case in test/terminal-input-fifo.test.ts: real CjkInput in Send on Enter mode, a held y typed for session A, focus moved to Pane B, Enter. The wire carries y and, 120 ms later, \r, both for A. If _submit() uses the focused pane instead of the captured target, the case fails (B instead of A).
  • test/mobile-shell-keyboard.test.ts: master's Send on Enter tests now observe _enqueueReliableInput behind the FIFO. They also cover the captured session across a tab switch and refusal for a session that has closed.

2. Voice compose Send keeps the draft

_sendToTarget() calls the plain _prepareExternalInput(pane, 'voice-send'), so the iOS owner still settles, but the typed draft is no longer flushed. New compose-mode case in test/voice-input-target.test.ts: with hel in the overlay, Send puts only Please fix the bug\r on the wire, and hel stays in the overlay. test/terminal-input-fifo.test.ts has a matching case with echo on and off. With flushEcho put back, both fail.

3. Docs

The split-pane section of docs/architecture-invariants.md now says Pane B gets the iOS native-input owner on iOS WebKit touch, and the keyCode-229 controller otherwise.

4. Stale stub

test/image-paste-trap.test.ts no longer stubs _flushLocalEchoPending.

Verification on this branch:

  • npm test: 765 files passed, 17,484 tests passed (1 file / 13 tests skipped by existing conditions).
  • Browser suites (ios-native-input, terminal-keycode229-recovery, shift-enter-keypress, split-pane-terminal, voice-input, mobile-ime-preview): 6 files, 62 tests passed.
  • Overlay package: 301 tests passed.
  • npm run build, typecheck, lint, format:check, check:frontend-syntax, check:frontend-dead, check:browser-excludes, check:public-assets, check:lockfile: passed.

@Ark0N

Ark0N commented Nov 20, 2026

Copy link
Copy Markdown
Owner

Thanks for the rebase and the fourth round, Hadrien, and for catching the restored-draft target yourself. This PR gives each iOS terminal pane a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit, and it moves every browser input producer onto a per-session FIFO.

Everything from the last round is in. The CJK sends carry the captured target, _sendCjkPrompt() reserves the text, gap and Enter on that session's FIFO, the toolbar Enter calls CjkInput.submit() first, the same-session reload keeps the held draft, and voice compose Send leaves the overlay draft alone.

I re-ran everything on ddb5a64. The typecheck, lint, format, frontend syntax, dead-code, browser-excludes, public-assets and lockfile checks are green. npm test passes (765 files, 17,484 tests), and so does the overlay package (301 tests). The iOS, keyCode-229, Shift+Enter, split, voice, IME preview, capture-geometry and tile scroll browser suites pass too (73 tests). test/mobile/keyboard.test.ts fails 5 cases, all of them among master's own 6.

Two things need to change before merge.

  1. The phone Enter button no longer scrolls back to the prompt (src/web/public/terminal-ui.js:857). On master, sendEnterKey() sent \r through triggerDataEvent('\r', true), and that true is what runs xterm's scrollOnUserInput. submitFromToolbar() puts the \r on the FIFO directly, so nothing scrolls. I measured it in the served app (Chromium, so Android too). With 200 lines written and the terminal scrolled to the top, app.sendEnterKey() on master ends at viewportY === baseY with local echo on and off. On this branch the \r is sent and the viewport stays at 0 in both modes. This is the same kind of bug as the round-2 typing case, and it affects every session where a touch drag scrolls xterm's own scrollback. Adding these two lines right after the guard in submitFromToolbar() fixes it; with them, the same probe passes in both modes:

    const t = pane.terminal;
    if (t.options?.scrollOnUserInput !== false && typeof t.scrollToBottom === 'function') t.scrollToBottom();

    Please add a browser case that scrolls to the top, clicks the toolbar Enter button and asserts buffer.active.viewportY === buffer.active.baseY.

  2. test/codex-predictive-echo.test.ts still records through _sendInputAsync (lines 330 and 477). Typed codex input now goes from _sealPendingInput to _enqueueInput to _enqueueReliableInput and never calls _sendInputAsync, so the byte-identity case fails at line 499 (withPredictor.length is 0). I reproduced its recorder in the served app with a codex-mode session. On master it records ab, DEL, the arrow, c and the paste; on this branch it records nothing, while the wire carries the same bytes. The file is outside the gate and the nightly, and it is a release-checklist item. Please retarget both recorders to _enqueueReliableInput, as you did for test/mobile/keyboard.test.ts.

A few small ones while you're in there:

  1. app.sendInput() (src/web/public/terminal-ui.js:7206) has no production caller left (check:frontend-dead --production lists it as test-only). You can remove it or keep it, but either way please update the @dependency headers of image-input.js:10, voice-input.js:28 and keyboard-accessory.js:36, which still say they use it.
  2. Docs: docs/reliable-input-delivery.md:78 says _sendInputAsync() splits an oversized paste; _enqueueReliableInput() does that now. docs/architecture-invariants.md:1288 (Mobile prompt composer) still describes delivery through _sendInputAsync with an Enter 120 ms later, where it is now the FIFO's frame, 120 ms gap and Enter. The _sendInputAsync JSDoc at src/web/public/app.js:4202 ("Never blocks the keystroke flush") is no longer true, since a write can now wait behind a gap.
  3. test/image-paste-trap.test.ts:194 still sets app._pendingInput = '', the old string field, which is a Map now. Please drop it.

Push the fixes to this branch and the PR goes back into review.

…Ark0N#830)

submitFromToolbar() puts the CR on the FIFO, bypassing xterm scrollOnUserInput; scroll the focused pane before sending.
Remove app.sendInput(), which has no production caller; update @dependency headers and test stubs.
Retarget the codex predictive-echo recorders to _enqueueReliableInput.
Correct the docs and the _sendInputAsync JSDoc for the session FIFO; drop stale _pendingInput stubs.
@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for the fourth round, and for measuring the toolbar Enter in the served app. All five points are addressed in 05fc2f9 on this branch.

  1. The toolbar Enter scrolls back to the prompt. submitFromToolbar() now calls pane.terminal.scrollToBottom() right after its guard, unless scrollOnUserInput is false. It uses the focused pane's own terminal, so it covers the main pane and tiles, with local echo on or off. New case in test/ios-native-input.browser.test.ts, run with echo off and on: write 200 lines, scrollToTop(), click the real app.sendEnterKey() toolbar button, wait for the \r on the wire, then assert buffer.active.viewportY === buffer.active.baseY. With the two lines removed, both cases fail.
  2. test/codex-predictive-echo.test.ts: both recorders (the Pasting content doesn't append to existing prompt #219 debug trace and the byte-identity trace) now wrap _enqueueReliableInput. I couldn't run the file here, because it needs the real codex binary. Instead I checked its recorder in the served app with a codex-mode session: typing ab, Backspace, ArrowLeft, c, then a paste records a, b, DEL, ESC[D, c and the paste at _enqueueReliableInput, so the byte-identity comparison has its bytes again.
  3. app.sendInput() is removed, since nothing in production calls it. The @dependency headers now name what each file actually uses: image-input.js (_sendInputAsync, _prepareExternalInput), voice-input.js (_sendInputAsync, _enqueueInput, _prepareExternalInput) and keyboard-accessory.js (_enqueueInput, _sendInputAsync, _prepareExternalInput). The upload comment in image-input.js describes the captured-session path instead of the removed method. The test stubs and the expect(sendInput).not.toHaveBeenCalled() lines that only pinned the removed method are gone. The tab-switch upload test still asserts that _sendInputAsync delivers to the session the upload started in. The two notes warning against driving browser tests with app.sendInput() (root CLAUDE.md, docs/pi-integration-plan.md) now name app._sendInputAsync(), which bypasses the overlay the same way.
  4. Docs. docs/reliable-input-delivery.md says _enqueueReliableInput() splits an oversized paste. The Mobile prompt composer section in docs/architecture-invariants.md describes the frame, the 120 ms gap and Enter enqueued on the session FIFO through _sendBracketedPaste(). The _sendInputAsync JSDoc now says it seals the batched keystrokes, enqueues on the session's FIFO, and can wait behind an earlier gap.
  5. _pendingInput: the string stub in test/image-paste-trap.test.ts is gone. So are the string _pendingInput fields in the clearTerminalInput tests of test/path-picker-ui.test.ts, along with the assertion that pinned one of them.

Verification on this branch:

  • npm test: 764 files, 17,483 tests passed (1 file / 13 tests skipped by existing conditions). The one failure in that run was test/multiuser-auth.test.ts (per-IP pre-gate, attempt 49 got 429 instead of 401), which this PR doesn't touch. It passed 19/19 in three isolated reruns.
  • Browser suites (ios-native-input, terminal-keycode229-recovery, shift-enter-keypress, split-pane-terminal, voice-input, mobile-ime-preview): 6 files, 64 tests passed, including the 2 new toolbar Enter scroll cases.
  • npm run build, typecheck, lint, format:check, check:frontend-syntax, check:frontend-dead, check:browser-excludes, check:public-assets, check:lockfile: passed. check:frontend-dead --production no longer lists sendInput.

@Ark0N

Ark0N commented Nov 21, 2026

Copy link
Copy Markdown
Owner

Thanks for this round, Hadrien, and for turning the toolbar Enter scroll around so quickly. This PR gives each iOS terminal pane a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit. It also moves every browser input producer onto a per-session FIFO.

Everything from the last round is in. Toolbar Enter scrolls back to the prompt, and the new browser case covers it with echo on and off. The codex predictive-echo recorders wrap _enqueueReliableInput, app.sendInput() is gone with its @dependency headers updated, and the docs and the stale _pendingInput stubs are fixed.

I re-ran everything on 05fc2f9. The typecheck, lint, format, frontend syntax, dead-code, browser-excludes, public-assets and lockfile checks are green. npm test passes (765 files, 17,484 tests), and so does the overlay package (301 tests). The iOS, keyCode-229, Shift+Enter, split, voice, IME preview, tile scroll and capture-geometry browser suites pass too (75 tests).

One thing needs to change before merge:

  1. Deleting a multi-code-point emoji also deletes the characters in front of it (src/web/public/terminal-ios-native-input.js:18 and :306). editBetween() counts what the field lost in code points, and reconcile() sends one DEL per code point, but WebKit removes a whole grapheme cluster on one Backspace. A red heart (U+2764 U+FE0F), a skin-tone emoji or a flag therefore puts 2 DELs on the wire, and a ZWJ family puts 5. Claude Code and Codex both delete one grapheme per DEL (Claude's Cursor.backspace() walks grapheme boundaries, and Codex's composer documents delete_backward as "Delete one grapheme to the left"), so the extra DELs erase 1 to 4 characters of the draft. On master, xterm sent exactly one DEL per Backspace.

    I measured it in the served app with the iOS owner and echo off, in a codex-mode session. With ok plus the emoji in the field, one real page.keyboard.press('Backspace') leaves the field at ok and puts 2, 2, 2 and 5 DELs on the wire for the heart, skin tone, flag and family, and 1 for a plain thumbs up (U+1F44D).

    Any path where the deletion reaches the PTY is affected: Codex on iPhone (echo off by policy), Claude with echo off or after an arrow key hands the line back to PTY echo, and grid tiles and Pane B on an iPad. With local echo on, pending overlay text comes out right, since removeChar() also takes one code point per DEL.

    Please count deletions in grapheme clusters (Intl.Segmenter with granularity: 'grapheme', falling back to the code-point split where it is missing), and make the overlay's removeChar() remove one grapheme per DEL (packages/xterm-zerolag-input/src/zerolag-input-addon.ts:23, lastCodePointWidth) so local echo stays in step. A shell then behaves as it does on master. Please also add U+2764 U+FE0F and ZWJ cases to test/terminal-ios-native-input.test.ts and the overlay package tests; today only the single-code-point thumbs up is covered.

Two small ones while you're in there:

  1. Toolbar Enter on a codex session no longer clears the predictive echo (src/web/public/terminal-ui.js:830). On master the \r went through triggerDataEvent, so _predictHookOnData('\r') cleared the predictions. The echo-off branch of _submitMainInput() now puts the \r on the FIFO directly; calling this._predictHookOnData('\r') in that branch restores it.
  2. When CodemanIosNativeInput.isSupported() is true but create() returns null, iOS gets neither the owner nor the keyCode-229 recovery it had on master (src/web/public/terminal-ui.js:751, src/web/public/terminal-tile.js:654). Falling back to the recovery when create() returns null keeps master's behaviour.

Push the fixes to this branch and the PR goes back into review.

…llback (Ark0N#830)

Count native edits and overlay deletions in grapheme clusters (Intl.Segmenter, code-point fallback), so one Backspace on a multi-code-point emoji sends one DEL.
Toolbar Enter with echo off tells the codex predictor about the CR, clearing its predictions as typed Enter does.
Fall back to the keyCode-229 recovery when the iOS owner refuses to start, in the main pane and tiles.
@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for the fifth round, and for measuring the grapheme deletions per emoji on the wire. All three points are addressed in 71c7d01 on this branch.

  1. One DEL per grapheme cluster. editBetween() in terminal-ios-native-input.js now counts in grapheme clusters, using Intl.Segmenter with granularity: 'grapheme' and falling back to the code-point split when it is missing. So reconcile() sends one DEL for a red heart, a skin-tone emoji, a flag or a ZWJ family, as xterm did on master. The overlay's removeChar() now removes one grapheme cluster per DEL (lastGraphemeWidth, same segmenter and fallback), for both pending and flushed text, so local echo stays in step.
    • test/terminal-ios-native-input.test.ts: editBetween cases for U+2764 U+FE0F, a skin tone, a flag and a ZWJ family, both a trailing deletion and a mid-line replacement. A replay case types ok plus each emoji and presses one Backspace, in direct and local-echo mode. It asserts the field is ok, exactly one DEL went on the wire, and the modeled destination still holds ok. The harness now matches WebKit and the CLIs: its Backspace removes a whole grapheme cluster from the field, and its destination deletes one grapheme per DEL. Both used to work per code point. With code-point counting put back, the new cases fail.
    • packages/xterm-zerolag-input/test/zerolag-input-addon.test.ts: one removeChar() removes the whole heart, skin tone, flag and family, from pending text and from flushed text (UTF-16 offset included).
  2. Toolbar Enter clears the codex predictions again. The echo-off branch of _submitMainInput() now calls this._predictHookOnData('\r') before putting the \r on the FIFO. Typed Enter only reaches _submitMainInput() with local echo on, so the hook still runs once per Enter. A new case in test/terminal-input-fifo.test.ts runs the real hook in predict mode and fails without the call.
  3. Recovery fallback. When isSupported() is true but create() returns null, _initTerminalInputOwner() (main pane) and _createInputOwner() (tiles and Pane B) now create the keyCode-229 recovery, as on master, and attach no caret marker. New cases in test/terminal-input-fifo.test.ts and test/terminal-tile-input.test.ts fail without the fallback.

Verification on this branch:

  • npm test: 765 files, 17,489 tests passed (1 file / 13 tests skipped by existing conditions).
  • Overlay package: 309 tests passed.
  • Browser suites (ios-native-input, terminal-keycode229-recovery, shift-enter-keypress, split-pane-terminal, voice-input, mobile-ime-preview): 6 files, 64 tests passed.
  • npm run build, typecheck, lint, format:check, check:frontend-syntax, check:frontend-dead, check:browser-excludes, check:public-assets, check:lockfile: passed.

@Ark0N

Ark0N commented Nov 21, 2026

Copy link
Copy Markdown
Owner

Thanks for the sixth round, Hadrien, and for turning the grapheme fix around so quickly. This PR gives each iOS terminal pane a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit, and it moves every browser input producer onto a per-session FIFO.

All three points from the last round are in: one Backspace on a multi-code-point emoji is one DEL on iOS, toolbar Enter clears the codex predictions again, and when create() refuses, the main pane and tiles fall back to the keyCode-229 recovery.

I re-ran everything on 71c7d01. The typecheck, lint, format, frontend syntax, dead-code, browser-excludes, public-assets and lockfile checks are green. npm test passes (765 files, 17,489 tests), as does the overlay package (309 tests). The iOS, keyCode-229, Shift+Enter, voice, split, tile scroll and IME preview browser suites pass (67 tests). test/mobile/keyboard.test.ts fails 5 cases, all of them among master's own 6 on the same machine.

One thing needs to change before merge:

  1. Android local echo now deletes the characters in front of a multi-code-point emoji (src/web/public/terminal-keycode229-recovery.js:69). The overlay's removeChar() now removes one grapheme per DEL on every platform, but the Android recovery's editBetween still counts code points. A soft-keyboard Backspace that arrives as keyCode 229 goes through that diff. A red heart (U+2764 U+FE0F), a skin tone or a flag puts 2 DELs on the wire, a ZWJ family puts 7, and the overlay drops one grapheme for each. I measured it in the served app with an Android user agent and local echo on: ok plus the emoji in the overlay, then one keyCode-229 Backspace. The heart and the skin tone leave o. The family empties the overlay and sends the 4 DELs left over to the PTY. A single-code-point emoji (U+1F600) stays ok. Since Enter sends the overlay's pending text, the prompt loses those characters. On master the overlay removed one UTF-16 unit per DEL, so the heart came out right there.

    Please count grapheme clusters in that editBetween too, with the same Intl.Segmenter and code-point fallback as terminal-ios-native-input.js, and update its comment (lines 64-68). Then add the heart, skin tone, flag and ZWJ family next to the single-code-point emoji case in test/terminal-keycode229-recovery.test.ts:727 (one DEL each), plus one case that feeds the recovery's DELs into the overlay and asserts that pendingText stays ok. This also stops Android with echo off from over-deleting in Claude Code and Codex, which delete one grapheme per DEL.

One small one while you're in there:

  1. xterm-zerolag-input is published on its own, and this PR changes its API: getInputEndAnchor(), onLayout() and the InputEndAnchor type are new, and removeChar() now removes a grapheme cluster. Please add an "xterm-zerolag-input": patch line to .changeset/830-ios-keyboard.md (the overlay change in perf(zerolag): rebuild only the changed overlay line, one paint layer per line #713 listed both packages), and add the two methods and the grapheme unit to the API table in packages/xterm-zerolag-input/README.md:206.

Push the fixes to this branch and the PR goes back into review.

…overlay change (Ark0N#830)

The keyCode-229 recovery diff counts grapheme clusters, matching the overlay and the iOS owner, so one Backspace on a multi-code-point emoji is one DEL.
List xterm-zerolag-input in the changeset; document getInputEndAnchor(), onLayout() and the grapheme unit of removeChar().
@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for the sixth round, and for catching the Android path with the per-emoji measurements. Both points are addressed in COMMIT on this branch.

  1. Android local echo keeps the draft when an emoji is deleted. editBetween() in terminal-keycode229-recovery.js now counts grapheme clusters, with the same Intl.Segmenter (granularity: 'grapheme') and code-point fallback as terminal-ios-native-input.js. Its comment says so. A keyCode-229 Backspace on a red heart, skin tone, flag or ZWJ family is now one DEL, so the overlay drops exactly that emoji, and with echo off Claude Code and Codex delete one grapheme as they should. New cases in test/terminal-keycode229-recovery.test.ts, next to the single-code-point emoji case:
    • one DEL each for U+2764 U+FE0F, a skin tone, a flag and a ZWJ family;
    • for each, the recovery's output fed into the real ZerolagInputAddon (imported from packages/xterm-zerolag-input/src): one removeChar() per DEL, and pendingText stays ok.
      With code-point counting put back, all 8 cases fail.
  2. xterm-zerolag-input release notes and docs. .changeset/830-ios-keyboard.md now lists 'xterm-zerolag-input': patch, and changeset status bumps both packages. In packages/xterm-zerolag-input/README.md:
    • The removeChar() row says it removes the last grapheme cluster.
    • The Backspace handling section explains the unit: one grapheme per call, so a 'flushed' return needs exactly one \x7f in Claude Code and Codex. It also warns that a line editor deleting by code point (bash readline, for a skin tone or a flag) removes less. Grapheme boundaries come from Intl.Segmenter, or one code point where it is missing.
    • getInputEndAnchor() (the InputEndAnchor { col, row }, or null while hidden) and onLayout(callback) (returns an unsubscribe function) are in the Rendering table.

Verification on this branch:

@Ark0N

Ark0N commented Nov 21, 2026

Copy link
Copy Markdown
Owner

Thanks for the seventh round, Hadrien, and for turning the Android grapheme fix around so quickly. This PR gives each iOS terminal pane a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit. It also moves every browser input producer onto a per-session FIFO.

Both points from the last round are in. The keyCode-229 recovery counts grapheme clusters, and its heart, skin tone, flag and family cases go through the real overlay. The changeset and the package README now cover xterm-zerolag-input.

I re-ran everything on e4ac772. Typecheck, lint, format, frontend syntax, dead code, browser excludes, public assets and lockfile are green. npm test passes (764 files). The one failure in the full run was a spreadsheet-preview timeout under load, in a file this PR does not touch, and that file passes on its own. The overlay package passes (309 tests), and so do the iOS, keyCode-229, Shift+Enter, split, voice and IME preview browser suites (64 tests). test/mobile/keyboard.test.ts fails 5 cases, all of them among master's own 6 on this machine.

The grapheme switch left one small problem, and there is one nit:

  1. Typing a combining mark onto a letter now sends a DEL plus the whole cluster (src/web/public/terminal-ios-native-input.js:36, src/web/public/terminal-keycode229-recovery.js:86). Both editBetween() functions find the common prefix in graphemes, so when a keystroke extends the last cluster, the prefix stops before it. Typing Thai กิ่ one key at a time puts ก, DEL กิ, DEL กิ่ on the wire, and Hindi क्ष puts क, DEL क्, DEL क्ष. Master sent three plain appends. Claude Code, Codex and the overlay still end up with the right text, and so does bash, because readline deletes a base letter together with its zero-width marks (I fed both byte sequences into read -e through a PTY). A line editor that deletes one code point per DEL, such as zsh with COMBINING_CHARS off (its default), turns the second DEL into a doubled letter: กกิ่ and कक्ष. Those bytes reach the PTY in every echo-off pane on iPhone and Android (shell sessions, codex, grid tiles and Pane B). The fix is one line at the top of each function, before the grapheme split:

    if (next.startsWith(previous)) return { deleted: 0, inserted: next.slice(previous.length) };

    With it, a pure append sends exactly what master sent, while deletions and replacements keep the grapheme count, so the emoji cases do not change. With that line in both files, the six input test files (352 tests) still pass. To pin it, add a case next to the emoji ones in test/terminal-ios-native-input.test.ts and test/terminal-keycode229-recovery.test.ts that types กิ่ and क्ष one field state at a time and asserts plain appends.

  2. Nit: ov.setFlushed(detected.length, detected) in the Tab-completion fallback (src/web/public/terminal-ui.js:1104) repeats what detectBufferText() already stored, and it costs one extra render before rerender(). The line can go.

Neither item needs another round from you: both can be applied on top at merge time. If you would rather make the changes yourself, push them to this branch.

Both editBetween() functions return a pure append as-is before the grapheme split, so a mark typed onto the last letter is not a DEL plus the retyped cluster (zsh would double the base letter).
Drop a redundant setFlushed() in the Tab-completion fallback; detectBufferText() already stores it.
@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for the seventh round, and for checking the Thai and Hindi sequences against read -e and zsh. I applied both items myself, in COMMIT on this branch.

  1. A combining mark typed onto a letter is a plain append again. Both editBetween() functions (terminal-ios-native-input.js and terminal-keycode229-recovery.js) now return { deleted: 0, inserted: next.slice(previous.length) } when next starts with previous, before the grapheme split. So typing กิ่ or क्ष one key at a time sends three plain appends, as on master, and zsh with COMBINING_CHARS off no longer doubles the base letter. Deletions and replacements still count grapheme clusters, so the emoji cases are unchanged.
    • test/terminal-ios-native-input.test.ts: editBetween checks for ก → กิ and क् → क्ष, plus a replay case that types each word one field state at a time and asserts the wire is exactly its code points, in direct and local-echo mode.
    • test/terminal-keycode229-recovery.test.ts: the same two words typed one field state at a time through the recovery, asserting plain appends, plus an editBetween check.
      With the line removed from both files, all of these fail (7 cases).
  2. Tab-completion fallback: the redundant ov.setFlushed(detected.length, detected) is gone; detectBufferText() already stores it, and rerender() paints once.

Verification on this branch:

@Ark0N

Ark0N commented Nov 21, 2026

Copy link
Copy Markdown
Owner

Thanks for the eighth round, Hadrien, and for applying the combining-mark append and the Tab-completion nit yourself. This PR gives each iOS terminal pane a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit. It also moves every browser input producer onto a per-session FIFO.

Both items from the last round are in. A mark typed onto a letter is a plain append again in both editBetween() functions, with the Thai and Hindi cases pinned, and the redundant setFlushed() is gone.

I re-ran everything on 414dd51. Typecheck, lint, format, frontend syntax, dead code, browser excludes, public assets and lockfile are green. npm test passes (765 files, 17,501 tests), the overlay package passes (309 tests), and the iOS, keyCode-229, Shift+Enter, split, voice and IME preview browser suites pass too (64 tests).

There is one small regression, plus a few nits:

  1. Toolbar Enter leaves an armed one-shot Ctrl armed (src/web/public/terminal-ui.js:865). On master the toolbar Enter went through triggerDataEvent('\r', true), so handleTerminalData called consumeCtrl('\r') and the modifier was spent. submitFromToolbar() now puts the CR on the FIFO directly, and nothing disarms it. I checked this with the terminal-input-fifo harness: after toggleCtrl() and submitFromToolbar() the bar is still armed, and the next typed c goes out as \x03. In a shell, a d would end the shell. Calling KeyboardAccessoryBar.clearCtrl?.() right after the guard in submitFromToolbar() restores master's behaviour, and a case next to the toolbar submit test in test/terminal-input-fifo.test.ts would pin it.
  2. Nit: lastGraphemeWidth() (packages/xterm-zerolag-input/src/zerolag-input-addon.ts:40) segments the whole string on every Backspace. That costs about 6 ms at 10,000 characters and 34 ms at 60,000 on a desktop CPU, where the old slice(0, -1) took constant time. Segmenting text.slice(-64) gives the same last cluster.
  3. Nit: the accessory Shift+Tab now flushes the overlay draft (sendKey() passes flushEcho: true, src/web/public/keyboard-accessory.js:1233), but a typed Shift+Tab does not, because \x1b[Z is not a composer nav key. It is harmless in Claude Code, so this one is optional.
  4. Nit: the comment in test/xterm-private-api.test.ts:72-74 still says iOS gets neither the owner nor the recovery when create() refuses. Since 71c7d01 it falls back to the recovery.
  5. Nit: src/web/public/terminal-ios-native-input.js:38 says "as on master", which won't mean anything after the merge.
  6. Nit: scripts/prepare-test-vendor.mjs:20 and the root CLAUDE.md Testing block (line 359) still name only pretest:mobile. Also, docs/architecture-invariants.md:148 says the FIFO drains into _enqueueReliableInput, but a key op goes out as a direct POST /send-key (src/web/public/app.js:4164).

None of these needs another round from you. They can all be applied on top at merge time. If you would rather make the changes yourself, push them to this branch.

)

Toolbar Enter spends one-shot Ctrl; Backspace finds the last grapheme with Segments.containing(); accessory Shift+Tab keeps the draft like a typed one.
Pass the captured session to Compose uploads, voice compose and held CJK drafts; clear Ctrl on deletion-only native edits; path picker clears predictions.
Clear xterm dead-key state on native commits; restore a borrowed clipboard field before classifying a key; overlay removeChar spans the pending/flushed boundary.
Place the trackpad caret from the overlay layout (getInputAnchor, committedText) or validated xterm cells, hiding when unproven; no end-minus-width arithmetic.
Grapheme-aware replay Backspace; docs, README rows, changeset and comment updates.
@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for the eighth round. I applied all six items myself, and then ran a full review of the whole PR against current master before pushing. It found ten more problems, which this commit (cda5d96) also fixes. The details follow.

Round-eight items

  1. Toolbar Enter spends an armed one-shot Ctrl again. submitFromToolbar() calls KeyboardAccessoryBar.clearCtrl?.() right after its guard, as consumeCtrl('\r') did on master. A new case in test/terminal-input-fifo.test.ts (echo on and off) arms Ctrl, presses toolbar Enter, then types c: c and \r go out, not \x03.
  2. lastGraphemeWidth() no longer walks the whole draft. It uses segment(text).containing(text.length - 1). I didn't use a fixed slice(-64): a cluster has no length bound, and regional-indicator pairing depends on where the run starts. In Node 22 it takes 0.09 ms at 60,000 characters, against 41 ms before. New overlay tests cover an 81-unit cluster (a plus 80 combining accents; a 64-unit tail scan fails it), an odd run of flags, and a 60,000-character draft.
  3. Shift+Tab: the two now match in the other direction. The accessory Shift+Tab leaves the unsent draft in the overlay, as a typed Shift+Tab does (sendKey('\x1b[Z', { keepDraft: true }); the iOS owner still settles). It cycles a mode and never edits the composer.
  4. The test/xterm-private-api.test.ts comment now says a refused create() falls back to the recovery.
  5. terminal-ios-native-input.js now says "as xterm sends it" instead of "as on master".
  6. scripts/prepare-test-vendor.mjs and the root CLAUDE.md Testing block name pretest:browser too. The Session FIFO paragraph in docs/architecture-invariants.md now says a key op is a direct POST /api/sessions/:id/send-key outside the exactly-once layer.

Found by a full review of the PR

  • Compose upload target. In the split view, Compose for Pane B uploaded attachments into Pane A's session (_uploadAndInsertImages(images, { insert: false }) had no sessionId), while the path went into B's prompt. It now passes the captured session.
  • One-shot Ctrl on an iOS Backspace. A deletion-only native transaction left Ctrl armed, so the next c became Ctrl+C. Master spent it on Backspace. Main pane and tiles now clear it.
  • Voice compose destination. After "New" for another session, the reused compose overlay still sent to the first session. The destination now lives on the overlay and updates with each dictation.
  • Path picker and codex predictions. Picking a path no longer goes through insertTerminalText(), which used to clear predictions; it now clears them itself.
  • CJK held drafts and the captured target. stashDraft() and clear() filed a held Send on Enter draft under the active or owner session, not the session it was typed for. In split view, Enter could later send B's answer to A. A captured target now takes precedence.
  • iPad dead keys. With a hardware keyboard, Option+E then E left xterm's _unprocessedDeadKey set (our _inputEvent patch never cleared it), so the next Enter or arrow was swallowed. The patch now clears it when native input commits text. _unprocessedDeadKey is added to the private-name guard.
  • Clipboard snapshot before classifying a key. After a right-click menu over a selection, keyFilter saw the borrowed clipboard text, so a Backspace on an empty line was lost. It now restores the owned field before classifying.
  • Overlay removeChar() across the pending/flushed boundary. A combining mark or skin-tone modifier typed after its base had been flushed left the base behind. It now segments flushed + pending text together and returns 'flushed' when the cluster reaches transmitted text.
  • Replay fixture. The browser fixture's Backspace removed one code point; it now removes a grapheme cluster, as WebKit does. New browser cases cover the heart, skin tone, flag and ZWJ family, in both echo modes.
  • Exact trackpad caret. The marker subtracted the suffix's cell width from the line end. A wide character that doesn't fit the last column leaves an empty cell, so the marker landed one cell off for each such wrap, and working backwards from the end can't resolve that. The marker now uses an authoritative position:
    • Local echo: the overlay records the cells it actually paints during its existing line split, with no second layout. New public getInputAnchor(offset) (UTF-16, forward wrap affinity; null when hidden, out of range, inside a surrogate pair, or on a tail-sliced row) and committedText. The host maps the native field as a suffix of committedText, or hides.
    • Echo off, tiles, Pane B: the marker reads xterm's parsed cells back from the cursor across soft wraps only, skipping validated wide-wrap padding, and shows only on an exact match of the native field. It hides on an ambiguous repetition, inside a multi-code-point cell, or after a sent edit until xterm parses its next write, and never guesses. That wait is a heuristic, not proof of delivery: any parsed write clears it, so if unrelated output arrives before the echo and identical text already sits before the cursor, the cells can still match the wrong text. Today's marker makes that assumption with no check at all. A copy that borrows and restores the field doesn't re-arm the wait.
    • test/ios-native-input.browser.test.ts asserts explicit cells (a你b from column 78 at 80 columns: (78, 0), (0, 1), (2, 1)) instead of the old end-minus-width oracle. The fixture sizes the terminal box so the app's own fit yields 80 columns.
    • README (two rows) and changeset updated.

Every behavioral fix above has a test that fails with the fix reverted. The caret design went through a design review first; it is the narrower version (instrument the existing layout, no new layout engine).

Verification on this branch:

  • npm test: 765 files, 17,524 tests passed (1 file / 13 tests skipped by existing conditions).
  • Browser suites (ios-native-input, terminal-keycode229-recovery, shift-enter-keypress, split-pane-terminal, voice-input, mobile-ime-preview): 6 files, 74 tests passed. The iOS suite also passed three times in a row on its own, and the iOS and split-pane suites passed twice more after the last fix.
  • Overlay package: 325 tests passed, tsc clean.
  • npm run build, typecheck, lint, format:check, check:frontend-syntax, check:frontend-dead, check:browser-excludes, check:public-assets, check:lockfile: passed. changeset status bumps both packages.

@Ark0N

Ark0N commented Nov 21, 2026

Copy link
Copy Markdown
Owner

Thanks for the ninth round, Hadrien, and for reviewing the whole PR yourself before pushing. This PR gives each iOS terminal pane a native-input owner, so suggestions, phrase replacement, dictation and held delete reach the PTY as one ordered edit. It also moves every browser input producer onto a per-session FIFO.

All six round-eight items are in. Toolbar Enter spends an armed one-shot Ctrl, lastGraphemeWidth() uses containing() instead of walking the draft, the accessory Shift+Tab keeps the draft the way a typed one does, and the comment, prepare-test-vendor and FIFO doc nits are fixed. The ten extra fixes look right too, including the Compose upload target, Ctrl on a deletion-only edit, the dead-key flag, the clipboard restore before classifyKey(), the pending/flushed boundary in removeChar(), and the caret, which now comes from the overlay layout or validated xterm cells instead of end-minus-width arithmetic.

I re-ran everything on cda5d96. Typecheck, lint, format, frontend syntax, dead code, browser excludes, public assets and lockfile are green. npm test passes (765 files, 17,525 tests), and so does the overlay package (325 tests, tsc clean). The iOS, keyCode-229, Shift+Enter, split, voice, IME preview, tile scroll and capture-geometry browser suites pass (85 tests) against a freshly built overlay bundle. test/mobile/keyboard.test.ts fails 5 cases, all among master's own 6 on the same machine.

One nit, small enough to fix at merge time: docs/ios-native-input.md:7 still gives caret() as { visible, text, offset }. It now also returns generation, and the marker's wait for parsed output is keyed on that field.

This is ready to merge. Thanks for staying with it through nine rounds, and for the careful device testing with the reporter.

caret() also returns generation; the parsed-buffer marker waits for a parsed write only when it changes.
Say the wait is a heuristic, not a delivery acknowledgement.
@HadrienKerlero

Copy link
Copy Markdown
Author

Thanks for the ninth round, and for staying with this through nine reviews. I fixed the doc nit on the branch so it doesn't have to wait for merge time:

  • docs/ios-native-input.md now gives caret() as { visible, text, offset, generation }, and says generation advances only when an edit is reconciled (sent) or the line is reset. A clipboard snapshot borrowing the field, and its restoration, leave it unchanged.
  • The parsed-buffer paragraph says the marker waits for xterm's next parsed write when generation changes. It also no longer claims that wait prevents mistaking old output for a delivered edit: any parsed write clears it, so it's a heuristic, not a delivery acknowledgement, matching what I wrote in the previous comment.

Docs only (git diff touches only that file); no code or test changes since cda5d96.

@Ark0N

Ark0N commented Nov 21, 2026

Copy link
Copy Markdown
Owner

Thanks for this fix, Hadrien, and for sticking with it through nine rounds of review. The PR gives every iOS terminal pane a native-input owner, so keyboard suggestions, phrase replacement, held delete and dictation reach the PTY as one ordered edit. It also moves every browser input producer onto a per-session FIFO, so delayed Enters and pastes stay with the session they were typed for.

The last commit updates docs/ios-native-input.md to describe caret() with its generation field, which was the only open item. I re-ran everything on 2d818e3. Typecheck, lint, format, frontend syntax, dead code, browser excludes, public assets and lockfile are green. npm test passes (765 files, 17,525 tests), and so does the overlay package (325 tests). The iOS, keyCode-229, Shift+Enter, split, voice, IME preview, tile scroll and capture-geometry browser suites pass (85 tests). test/mobile/keyboard.test.ts fails 5 cases, all of them among master's own 6 on the same machine.

Nothing needs to change at merge time, and it can go in as it stands. Thanks also to the reporter for the device testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

iPhone keyboard: tapping a suggestion only sends its first letter, and hold-delete never goes word by word

2 participants