Skip to content

Wait for a job's output alongside the other jobs, not on the dispatch loop's thread - #71

Merged
HalRiveria-Bot merged 2 commits into
mainfrom
fix/output-wait-in-loop
Nov 18, 2026
Merged

HalRiveria-Bot merged 2 commits into
mainfrom
fix/output-wait-in-loop

Conversation

@HalRiveria-Bot

Copy link
Copy Markdown
Collaborator

What was wrong

The batch's -JobFinalize calls Complete-PkProcess, whose wait for the output (up to 15 s) ran on the dispatch loop's own thread. While one program's output was held open by something it started, no other job was polled, started, killed at its time limit or stopped by an abort. Two programs at -MaxConcurrent 2, one leaving a helper holding its output for 6 s, the other with a 2 s time limit: the limited one was killed at 6.2 s. A dispatcher whose -JobFinalize calls Complete-PkProcess did the same.

What changed

  • The dispatcher's default -JobPollCompleted, which the batch uses, counts a job as done once its program has exited and both reads have ended, or 15 s after it saw the exit (Complete-PkProcess's default stream timeout). The job keeps its slot meanwhile, so the slot count still means jobs in flight, and the loop carries on with the other jobs.
  • Complete-PkProcess waits only what's left of that wait, timed by the poll's new OutputWait key. With the default it has nothing left to wait for.
  • After an abort's wait, the final pass sets OutputWaitOver on the jobs still running. A job whose program has finished then takes the output that has arrived (StreamTimedOut), with at most a 250 ms wait, instead of being force-stopped and reported as aborted. A program still running is force-stopped as before.
  • A retried job has both keys cleared. Both are added to the documented list of job keys the dispatcher owns.
  • Help and README: the default poll, what a -StreamTimeoutMs in -JobFinalize still does, and the batch's behavior while a program's output is held open.

The time-limit taskkill (Stop-PkTimedOutJob) still runs on the loop; it didn't fall out of this change.

How it was tested

New harness section BB, on the batch and the dispatcher's default poll:

  • one program waits about 6 s for its output while four 1 s programs go through the other slot;
  • a 2 s time limit is enforced while another job waits for its output (batch, and dispatcher with Complete-PkProcess in -JobFinalize);
  • an abort while a job waits for its output: the run ends in time, and that job keeps what arrived;
  • output held past the stream timeout is given up on once, after 15 s (a guard for the new give-up path; it passes on the old code too).

The helpers' pings carry -w 4321, and the section kills only those.

On the old code (main), the four new behaviors fail on both runtimes. Full harness on Windows PowerShell 5.1:

[FAIL] batch: other programs keep running through the free slot while one waits for its output (last of 4 done at 8.2 s; the held one whole: True)
[FAIL] batch: a program's 2 s time limit is enforced while another waits for its output (killed at 6.2 s)
[FAIL] batch: an abort while a job waits for its output ends the run in time, and that job keeps what arrived (8.2 s, Aborted False, StreamTimedOut False, Stdout 'held', force-stopped 0)
[FAIL] dispatcher: with the default poll, a 2 s time limit is enforced while another job waits for its output (killed at 6.2 s; the held one whole: True)
4 FAILED, 561 passed

With this change, on both runtimes:

[PASS] batch: other programs keep running through the free slot while one waits for its output (last of 4 done at 4.2 s; the held one whole: True)
[PASS] batch: a program's 2 s time limit is enforced while another waits for its output (killed at 2.1 s)
[PASS] batch: an abort while a job waits for its output ends the run in time, and that job keeps what arrived (2.4 s, Aborted False, StreamTimedOut True, Stdout 'held', force-stopped 0)
[PASS] batch: output held past the stream timeout is given up on once, after 15 s (15.1 s)
[PASS] dispatcher: with the default poll, a 2 s time limit is enforced while another job waits for its output (killed at 2.1 s; the held one whole: True)

Full harness: Windows PowerShell 5.1 ALL PASSED (565 assertions), PowerShell 7 ALL PASSED (565 assertions).

🤖 Generated with Claude Code

HalRiveria and others added 2 commits November 18, 2026 16:04
…lize

Complete-PkProcess's wait for the output (up to 15 s) ran inside the
batch's -JobFinalize, on the dispatch loop's own thread, so while one
program's output was held open no other job was polled, started, killed
at its time limit or stopped by an abort.

The dispatcher's default -JobPollCompleted, which the batch uses, now
counts a job as done once its program has exited and both reads have
ended, or 15 s after it saw the exit. The job keeps its slot meanwhile.
Complete-PkProcess waits only what's left of that (OutputWait), and after
an abort's wait the final pass sets OutputWaitOver, so a job whose program
had finished takes the output that arrived instead of being force-stopped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After an abort's wait, a program that exited just before the final pass
could still have its last output in flight. Complete-PkProcess now waits
up to 250 ms then, instead of not at all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HalRiveria-Bot
HalRiveria-Bot merged commit d746955 into main Nov 18, 2026
2 checks passed
@HalRiveria-Bot
HalRiveria-Bot deleted the fix/output-wait-in-loop branch November 18, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants