Repository navigation
Reader cleanup: drop output nobody can get, report a failed read, close the reader - #70
Merged
Merged
Conversation
… close the reader - The first snapshot of a stream that hasn't ended is its text from then on. The read still drains to the end, so the program holding the pipe never blocks on it, but drops what it reads instead of building up text nobody can get. - Complete-PkProcess sets StreamError from a read that failed even when the other stream timed out: Task.WaitAll with a timeout only throws once every task has finished, so the failure went unreported. - The reader is closed when its stream ends, so a finished program's pipe handle goes then rather than at the next garbage collection. TypesVersion goes to 9: the C# changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StreamError now comes from the faulted read task in every case, rather than
PowerShell's wrapper around Task.WaitAll ("One or more errors occurred").
The help and README say that a second Complete-PkProcess on the same job
returns the same text, as does the read task's Result.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
Three problems in how a program's output is read after it exits:
Complete-PkProcesshanded over the text a held stream had so far, the read went on appending everything the leftover program wrote, though nobody could get it. A helper writing 60 MB after a 500 ms stream timeout took managed memory from 9 MB to 127 MB on 5.1 (17 to 134 MB on 7), and it stayed there until the helper let go. 3.2 did the same.Task.WaitAllwith a timeout throws only once every task has finished. So when one stream's read failed while the other was still held open, the result hadStreamTimedOutset andStreamError$null, and the failed stream came back$nullwith no reason. When it was reported,StreamErrorwas PowerShell's wrapper text ("Exception calling "WaitAll" ... One or more errors occurred"), not the read's own reason.What changed
PkStreamReadTask: the first snapshot taken before a stream ends becomes that stream's text from then on. The read still drains to the end, so the program holding the pipe never blocks on a full pipe, but it drops what it reads. The task'sResultis that snapshot, or the whole text if none was taken. The reader is closed when its stream ends, on both start paths.Complete-PkProcesstakesStreamErrorfrom the faulted read task's own exception, whether or not the wait timed out.Complete-PkProcesson the same job now returns the same text as the first, not what arrived since. The help and README say so.TestHeldLengthandTestReaderClosedonPkStreamReadTask.[PkState]::TypesVersiongoes to 9.StreamTimedOutparagraph covers the dropped text, the repeat call, andStreamErroralongsideStreamTimedOut.How it was tested
New harness section BA, 8 assertions:
ping -tkeeps writing after the result: the buffer stays at the result's length, a secondComplete-PkProcessreturns the same text, and once the ping is killed (by parent PID) the read ends with the handed-over text as its result and its reader closed;StreamErroralongsideStreamTimedOut;On the old code (main), all 8 fail on both runtimes. Full harness on Windows PowerShell 5.1:
The three reader-closed checks fail there because the old build has no
TestReaderClosed; the handle counts below show the effect directly.With this change, all 8 pass on both runtimes.
Handles above the start after 300 quick programs, before a GC (each comes back to the start after a GC):
The difference is the pipe handles. Most of what's left on 5.1 is the handles of the ended reader threads (#69), which .NET Framework releases when the
Threadobjects are collected; in a 2,000-program batch the collections that happen anyway kept it lower than 3.3 did.Managed memory 8 s after a 500 ms stream timeout, while a helper writes 60 MB to the held stream:
Full harness: Windows PowerShell 5.1
ALL PASSED (560 assertions), PowerShell 7ALL PASSED (560 assertions).🤖 Generated with Claude Code