Skip to content

Reader cleanup: drop output nobody can get, report a failed read, close the reader - #70

Merged
HalRiveria-Bot merged 2 commits into
mainfrom
fix/reader-cleanup
Nov 18, 2026
Merged

HalRiveria-Bot merged 2 commits into
mainfrom
fix/reader-cleanup

Conversation

@HalRiveria-Bot

Copy link
Copy Markdown
Collaborator

What was wrong

Three problems in how a program's output is read after it exits:

  • Output kept piling up after a stream timeout. Once Complete-PkProcess handed over the text a held stream had so far, the read went on appending everything the leftover program wrote, though nobody could get it. A helper writing 60 MB after a 500 ms stream timeout took managed memory from 9 MB to 127 MB on 5.1 (17 to 134 MB on 7), and it stayed there until the helper let go. 3.2 did the same.
  • A failed read could be reported as only a stream timeout. Task.WaitAll with a timeout throws only once every task has finished. So when one stream's read failed while the other was still held open, the result had StreamTimedOut set and StreamError $null, and the failed stream came back $null with no reason. When it was reported, StreamError was PowerShell's wrapper text ("Exception calling "WaitAll" ... One or more errors occurred"), not the read's own reason.
  • A finished program's pipe handles waited for the garbage collector. The reader was never closed, so its pipe handle stayed open until a GC.

What changed

  • PkStreamReadTask: the first snapshot taken before a stream ends becomes that stream's text from then on. The read still drains to the end, so the program holding the pipe never blocks on a full pipe, but it drops what it reads. The task's Result is that snapshot, or the whole text if none was taken. The reader is closed when its stream ends, on both start paths.
  • Complete-PkProcess takes StreamError from the faulted read task's own exception, whether or not the wait timed out.
  • A second Complete-PkProcess on the same job now returns the same text as the first, not what arrived since. The help and README say so.
  • Test-only reads TestHeldLength and TestReaderClosed on PkStreamReadTask.
  • [PkState]::TypesVersion goes to 9.
  • README: the StreamTimedOut paragraph covers the dropped text, the repeat call, and StreamError alongside StreamTimedOut.

How it was tested

New harness section BA, 8 assertions:

  • a ping -t keeps writing after the result: the buffer stays at the result's length, a second Complete-PkProcess returns the same text, and once the ping is killed (by parent PID) the read ends with the handed-over text as its result and its reader closed;
  • both readers are closed after a normal program, on both start paths;
  • a faulted read next to a held stream sets StreamError alongside StreamTimedOut;
  • a faulted read next to a finished stream gives the read's own reason.

On the old code (main), all 8 fail on both runtimes. Full harness on Windows PowerShell 5.1:

[FAIL] a second Complete-PkProcess on the same job returns the same text (296 characters, first 149)
[FAIL] a program that keeps writing after the result: the buffer stays at the result's 149 characters (at the result missing, 3 s later missing, still reading True)
[FAIL] once the writer is killed the read ends, and its result is the text handed over, not what came after (True, 296 characters)
[FAIL] a held stream's reader is closed when the stream ends
[FAIL] created in the job: both readers are closed once a program's output ends (stdout missing, stderr missing)
[FAIL] Process.Start fallback: both readers are closed once a program's output ends (stdout missing, stderr missing)
[FAIL] a read that failed while the other stream timed out is reported in StreamError (StreamTimedOut True, StreamError '')
[FAIL] a failed read next to one that ended gives the read's own reason (StreamError 'Exception calling "WaitAll" with "2" argument(s): "One or more errors occurred."')
8 FAILED, 552 passed

The three reader-closed checks fail there because the old build has no TestReaderClosed; the handle counts below show the effect directly.

With this change, all 8 pass on both runtimes.

Handles above the start after 300 quick programs, before a GC (each comes back to the start after a GC):

main this PR
Windows PowerShell 5.1 +1343 +1149
PowerShell 7 +74 +60

The difference is the pipe handles. Most of what's left on 5.1 is the handles of the ended reader threads (#69), which .NET Framework releases when the Thread objects are collected; in a 2,000-program batch the collections that happen anyway kept it lower than 3.3 did.

Managed memory 8 s after a 500 ms stream timeout, while a helper writes 60 MB to the held stream:

main this PR
Windows PowerShell 5.1 9 -> 127 MB 9 -> 11 MB
PowerShell 7 17 -> 134 MB 17 -> 18 MB

Full harness: Windows PowerShell 5.1 ALL PASSED (560 assertions), PowerShell 7 ALL PASSED (560 assertions).

🤖 Generated with Claude Code

HalRiveria and others added 2 commits November 18, 2026 15:40
… close the reader

- The first snapshot of a stream that hasn't ended is its text from then on.
  The read still drains to the end, so the program holding the pipe never
  blocks on it, but drops what it reads instead of building up text nobody
  can get.
- Complete-PkProcess sets StreamError from a read that failed even when the
  other stream timed out: Task.WaitAll with a timeout only throws once every
  task has finished, so the failure went unreported.
- The reader is closed when its stream ends, so a finished program's pipe
  handle goes then rather than at the next garbage collection.

TypesVersion goes to 9: the C# changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StreamError now comes from the faulted read task in every case, rather than
PowerShell's wrapper around Task.WaitAll ("One or more errors occurred").
The help and README say that a second Complete-PkProcess on the same job
returns the same text, as does the read task's Result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HalRiveria-Bot
HalRiveria-Bot merged commit d66a781 into main Nov 18, 2026
2 checks passed
@HalRiveria-Bot
HalRiveria-Bot deleted the fix/reader-cleanup branch November 18, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants