Skip to content

Add -MaxOutputChars, keeping the start and the end of long output - #75

Merged
HalRiveria-Bot merged 7 commits into
mainfrom
feat/output-cap
Nov 19, 2026
Merged

HalRiveria-Bot merged 7 commits into
mainfrom
feat/output-cap

Conversation

@HalRiveria-Bot

Copy link
Copy Markdown
Collaborator

What was wrong

A program's output is kept in memory until it's handed over. A program that prints gigabytes used that much, and the only workaround (nulling Stdout/Stderr in -OnComplete) frees output after the fact, not while one program is printing.

What changed

  • -MaxOutputChars on Start-PkProcess, Invoke-PkProcessWithTimeout, Invoke-PkProcessBatch and Invoke-PkProcessDispatch (its default starter). Default 0, no cap; a negative value is refused before anything starts. In the batch and dispatcher, a command's or job's own MaxOutputChars key overrides it (0 opts out; a bad value is that command's start error), as TimeoutSeconds does.
  • Each stream is capped on its own. Past the cap, the first half and the last half are kept, never split inside a surrogate pair. The rest is still read, so the program never stalls on a full pipe.
  • Results get StdoutDroppedChars and StderrDroppedChars: how many characters were left out, 0 with no cap. They're always present, at the end of Complete-PkProcess's, Invoke-PkProcessWithTimeout's and each batch result's fields.
  • The cap's tail grows with the output, so a large cap costs nothing until output reaches it. [int]::MaxValue is fine. A ring buffer filled with block copies once it's full.
  • A snapshot at a stream timeout no longer ends in the first half of a surrogate pair whose second half hasn't been read yet. This was possible before with UTF-16 output; the class comment already claimed it couldn't happen.
  • C#: PkStreamReadTask gains Run(reader, maxChars) and DroppedChars. TypesVersion 10.
  • README: the results list, the batch and dispatcher paragraphs, the single-process table, and Limits.

How it was tested

  • New harness section BE, 15 assertions:
    • The head/tail split and the dropped count.
    • Output exactly at the cap.
    • [int]::MaxValue with short output.
    • The Process.Start fallback path.
    • stderr on its own.
    • Surrogate pairs at both cuts (cmd /u).
    • 20,000 lines under a 1,000-character cap: at most 1,000 characters held while it ran, and the count adds up to the uncapped length.
    • A 20,000-character cap matching the uncapped text's first and last 10,000 exactly.
    • A stream timeout.
    • The batch's parameter, key override, opt-out and bad key.
    • The dispatcher's default starter with a string key.
    • -1 refused by all four.
  • The two result-shape assertions are updated for the new fields.
  • On the old code, 12 FAILED, 579 passed, on both runtimes: the first 10 BE assertions plus the 2 shape tests. Three were added after that run ([int]::MaxValue, the fallback, the ring wrap), all using the new parameter.
  • On this branch, the full harness passed on 5.1 and then 7: 594 assertions each.
  • Not covered by a test: the snapshot surrogate fix needs a read to split a pair at the moment of a stream timeout, which can't be timed from the harness.

Depends on nothing open.

🤖 Generated with Claude Code

HalRiveria and others added 7 commits November 19, 2026 01:23
…patch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the shared default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	ParallelKit.Harness.ps1
#	ParallelKit.ps1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…shot in half a pair

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HalRiveria-Bot
HalRiveria-Bot merged commit 296c7b2 into main Nov 19, 2026
2 checks passed
@HalRiveria-Bot
HalRiveria-Bot deleted the feat/output-cap branch November 19, 2026 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants