Skip to content

feat: add round-robin connection balancing - #45

Open
stepanenkoxx wants to merge 2 commits into
mainfrom
add-round-robin-balancing
Open

stepanenkoxx wants to merge 2 commits into
mainfrom
add-round-robin-balancing

Conversation

@stepanenkoxx

Copy link
Copy Markdown

Changes

  • Add opt-in round-robin balancing for replicas and for both connection getters in proxy mode. In proxy mode, the getters share one rotation.
  • Allow a latency tolerance to limit rotation to healthy endpoints close to the fastest one. Validate the tolerance at initialization and keep a copy of the strategy.
  • Use the fastest eligible endpoint by default.

Testing

  • Node.js 18 build, lint, and 27 unit tests passed.
  • The PostgreSQL integration suite requires Docker. The local Docker daemon was unavailable.

Comment thread lib/dispatcher.ts
}

// the window may shrink between health checks, so wrap the stored position into it
const index = this.balancingCursor % eligible.length;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1 [P2] Preserve the round-robin position when eligible hosts change

AI generated

The cursor indexes the filtered list, so the same index can point to different hosts after a health check. With one call after each check and eligible proxies alternating between [A, B, C] and [B, C], the picks are A, C, C, C, … even though B stays healthy throughout. Store the cursor in the original connection order and skip ineligible hosts as it advances.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each change to the eligible set causes at most one repeated or skipped pick, and the rotation evens out again after that. That only adds up when there's about one request per health check (every 5 s by default), and at that traffic balancing doesn't matter.

I could add something like this

// position in `this.connections` of the connection last returned by round-robin
private lastRoundRobinPosition = -1;

private selectRoundRobin(candidates: PDConnection[], latencyToleranceMs?: number) {
    let eligible = candidates;

    if (latencyToleranceMs !== undefined) {
        const minLatency = fastestConnection(candidates).latency;
        eligible = candidates.filter((c) => c.latency <= minLatency + latencyToleranceMs);
    }

    // `eligible` keeps the order of `this.connections`, so continuing after the last returned
    // position does not repeat or skip hosts when hosts join or leave between health checks
    const next =
        eligible.find((c) => this.connections.indexOf(c) > this.lastRoundRobinPosition) ??
        eligible[0];
    this.lastRoundRobinPosition = this.connections.indexOf(next);

    return next;
}

it works, but it adds complexity for no real gain i think
I suggest keeping current approach

Comment thread lib/types.ts Outdated
* is within this value of the lowest one are rotated (0 — only the fastest ones).
* When omitted, all healthy connections are rotated
*/
latencyTolerance?: number;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer latencyToleranceMs so the unit is clear at the call site.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Make the unit clear at the call site.
@stepanenkoxx
stepanenkoxx force-pushed the add-round-robin-balancing branch from 8a68da1 to 8d07dd8 Compare November 19, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants