Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,14 @@ export const {db, CoreBaseModel, helpers} = initDB({
- `suppressStatusLogs`: Boolean value that disables database health-check status logs (useful for developers)
- `beforeTerminate`: Function called before terminating a connection, must return a Promise
- `topologyMode`: Connection topology, either `primary-replica` (the default) or `proxy`
- `balancingStrategy`: How a host is chosen for `db.replica` (and for both `db.primary` and `db.replica` in `proxy` mode), an object with a `kind` field:
- `{kind: 'fastest'}` (the default): always picks the host with the lowest latest health-check latency; among equally fast hosts, the first one in the connection list
- `{kind: 'round-robin', latencyToleranceMs?: number}`: healthy hosts are used in turn, spreading load across them. Without `latencyToleranceMs`, every healthy host takes part regardless of latency, so with hosts in several data centers part of the load goes to distant ones. Set `latencyToleranceMs` to rotate only hosts whose latest health-check latency is within that value of the fastest one, e.g. `10` to skip distant hosts, or `0` to rotate only the hosts sharing the lowest latency
- `knexOptions`: Non-required additional options that will be passed to Knex before initialization
- `onKnexCreated`: Optional callback called synchronously for each Knex instance created by the dispatcher, before database health checks start. Use it to attach instrumentation, event listeners, or plugins that do not require an active database connection. If it throws, initialization fails. The callback receives the Knex instance and returns nothing.
- `onHealthcheck`: Optional synchronous callback called after each database health check with a snapshot of every host's availability, latency, and role. In `primary-replica` mode, unavailable hosts have the `unknown` role; proxy connections do not include a role. It is called even when `suppressStatusLogs` is enabled. Callback errors do not interrupt database routing or subsequent health checks.

When all connection strings point to equivalent proxy or router instances, such as SPQR routers, use `proxy` mode. Healthy endpoints are then eligible for both primary and replica queries, and the endpoint with the lowest latest health-check latency is selected:
When all connection strings point to equivalent proxy or router instances, such as SPQR routers, use `proxy` mode. Healthy endpoints are then eligible for both primary and replica queries, and the endpoint with the lowest latest health-check latency is selected (with the `round-robin` balancing strategy, endpoints are rotated instead):

```typescript
initDB({
Expand Down Expand Up @@ -96,14 +99,14 @@ The `initDB` constructor exports three elements: `db`, `CoreBaseModel`, and `hel

### db

`db`: Instance of the [PGDispatcher](https://github.com/gravity-ui/postgreskit/blob/main/lib/dispatcher.ts) module responsible for connection balancing. Under the hood, this module creates N instances of knex (N is the number of hosts passed in `connectionString`). By default, it polls the database hosts every `healthcheckInterval` milliseconds, requesting if they are primary hosts or replica hosts (using the `SELECT pg_is_in_recovery()` query). In `proxy` topology mode, it uses `SELECT 1` instead and routes both `db.primary` and `db.replica` to the fastest healthy endpoint.
`db`: Instance of the [PGDispatcher](https://github.com/gravity-ui/postgreskit/blob/main/lib/dispatcher.ts) module responsible for connection balancing. Under the hood, this module creates N instances of knex (N is the number of hosts passed in `connectionString`). By default, it polls the database hosts every `healthcheckInterval` milliseconds, requesting if they are primary hosts or replica hosts (using the `SELECT pg_is_in_recovery()` query). In `proxy` topology mode, it uses `SELECT 1` instead and routes both `db.primary` and `db.replica` to the fastest healthy endpoint (or rotates between healthy endpoints with the `round-robin` balancing strategy).

Public `db` methods:

- `ready`: Function that returns a Promise that is resolved after the database connection has been established. This function is useful when you do not want to start query processing until the database is ready: for this, add `await db.ready(); next()` as middleware
- `terminate`: Terminates all connections and returns a Promise which is resolved after `knex.destroy()` executes for each connection
- `primary`: Getter that always points at the knex instance targeting the current primary host. If there is no primary host, when trying to access `db.primary`, you will get an error with the `ERR_DB_READ_ONLY` code
- `replica`: Getter that always points at the knex instance targeting the fastest available replica host. The speed is determined from the latest health-check query response time. If only the primary host is available, `db.replica` returns it, logging a warning about an attempt to read from the `primary` replica. If no host is available, `db.replica` throws an error with the `ERR_DB_UNAVAILABLE` code
- `replica`: Getter that points at the knex instance targeting the fastest available replica host. The speed is determined from the latest health-check query response time. With the `round-robin` balancing strategy, healthy replicas (limited by `latencyToleranceMs`, if set) are returned in turn, so consecutive accesses may return different instances and therefore replicas with different replication lag. If only the primary host is available, `db.replica` returns it, logging a warning about an attempt to read from the `primary` replica. If no host is available, `db.replica` throws an error with the `ERR_DB_UNAVAILABLE` code

Both `db.primary` and `db.replica` point to regular knex instances that can do everything described in the [knex documentation](https://knexjs.org/).

Expand Down
12 changes: 11 additions & 1 deletion lib/core.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,25 @@ import {type Constructor, Model} from 'objection';

import {defaultDispatcherOptions, defaultExLogger, defaultKnexOptions} from './constants';
import {PGDispatcher} from './dispatcher';
import type {BaseModel, ExLogger, PGHealthcheckHandler, TopologyMode} from './types';
import type {
BalancingStrategy,
BaseModel,
ExLogger,
PGHealthcheckHandler,
TopologyMode,
} from './types';

export type {
BalancingStrategy,
FastestBalancingStrategy,
PGConnectionRole,
PGConnectionStatus,
PGHealthcheckHandler,
PGHealthcheckStatus,
PGPrimaryReplicaConnectionStatus,
PGPrimaryReplicaHealthcheckStatus,
PGProxyHealthcheckStatus,
RoundRobinBalancingStrategy,
TopologyMode,
} from './types';

Expand All @@ -24,6 +33,7 @@ export interface CoreDBDispatcherOptions {
suppressHealthcheckStatusLogs?: boolean;
beforeTerminate?: () => Promise<void>;
topologyMode?: TopologyMode;
balancingStrategy?: BalancingStrategy;
}

export type GetModelParams = {cancelOnTimeout?: boolean; useLimitInFirst?: boolean};
Expand Down
74 changes: 68 additions & 6 deletions lib/dispatcher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import knexBuilder from 'knex';
import type {Knex} from 'knex';

import type {
BalancingStrategy,
Dict,
ExLogger,
PDOptions,
Expand Down Expand Up @@ -49,6 +50,40 @@ interface PDCheckupResult {
primary: boolean;
}

// returns the first connection with the lowest latency, so ties go to the earliest one
function fastestConnection(candidates: PDConnection[]) {
let fastest = candidates[0];

for (const candidate of candidates) {
if (candidate.latency < fastest.latency) {
fastest = candidate;
}
}

return fastest;
}

function validateBalancingStrategy(strategy?: BalancingStrategy): BalancingStrategy {
if (strategy?.kind !== 'round-robin') {
return {kind: 'fastest'};
}

const {latencyToleranceMs} = strategy;

if (
latencyToleranceMs !== undefined &&
(!Number.isFinite(latencyToleranceMs) || latencyToleranceMs < 0)
) {
throw new Error(
`balancingStrategy.latencyToleranceMs must be a non-negative finite number, got ${String(
latencyToleranceMs,
)}`,
);
}

return {kind: 'round-robin', latencyToleranceMs};
}

export class PDError extends Error {
code?: string;
}
Expand All @@ -63,6 +98,8 @@ export class PGDispatcher {
private hcTimer?: Timeout | null;
private isInit = false;
private isTerminating = false;
private readonly balancingStrategy: BalancingStrategy;
private balancingCursor = 0;

constructor({
connections = [],
Expand All @@ -76,6 +113,8 @@ export class PGDispatcher {
throw new Error('Empty connections list is not allowed');
}

this.balancingStrategy = validateBalancingStrategy(options.balancingStrategy);

this.connections = connections.map((connectionString) => {
const url = new URL(connectionString);
const knex = knexBuilder({
Expand Down Expand Up @@ -163,7 +202,7 @@ export class PGDispatcher {
this.checkConnectionsAvailability();

if (this.isProxyMode) {
return this.fastestHealthyConnection.knex;
return this.selectConnection(this.healthyConnections).knex;
}

const primaryConnections = this.connections.filter((c) => c.primary);
Expand All @@ -189,14 +228,13 @@ export class PGDispatcher {
this.checkConnectionsAvailability();

if (this.isProxyMode) {
return this.fastestHealthyConnection.knex;
return this.selectConnection(this.healthyConnections).knex;
}

const replicaConnections = this.healthyConnections.filter((c) => !c.primary);

if (replicaConnections.length) {
const fastestReplica = replicaConnections.sort((a, b) => a.latency - b.latency)[0];
return fastestReplica.knex;
return this.selectConnection(replicaConnections).knex;
}
if (this.connections.length > 1) {
this.logger.error({message: 'No alive replica available, using master for read'});
Expand Down Expand Up @@ -389,8 +427,32 @@ export class PGDispatcher {
return this.options.topologyMode === 'proxy';
}

private get fastestHealthyConnection() {
return this.healthyConnections.sort((a, b) => a.latency - b.latency)[0];
// `candidates` must be non-empty.
private selectConnection(candidates: PDConnection[]) {
const strategy = this.balancingStrategy;

if (strategy.kind === 'round-robin') {
return this.selectRoundRobin(candidates, strategy.latencyToleranceMs);
}

return fastestConnection(candidates);
}

// Rotates between all connections or, when `latencyToleranceMs` is set, only those whose
// latest health-check latency is within `latencyToleranceMs` of the fastest one.
private selectRoundRobin(candidates: PDConnection[], latencyToleranceMs?: number) {
let eligible = candidates;

if (latencyToleranceMs !== undefined) {
const minLatency = fastestConnection(candidates).latency;
eligible = candidates.filter((c) => c.latency <= minLatency + latencyToleranceMs);
}

// the window may shrink between health checks, so wrap the stored position into it
const index = this.balancingCursor % eligible.length;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1 [P2] Preserve the round-robin position when eligible hosts change

AI generated

The cursor indexes the filtered list, so the same index can point to different hosts after a health check. With one call after each check and eligible proxies alternating between [A, B, C] and [B, C], the picks are A, C, C, C, … even though B stays healthy throughout. Store the cursor in the original connection order and skip ineligible hosts as it advances.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each change to the eligible set causes at most one repeated or skipped pick, and the rotation evens out again after that. That only adds up when there's about one request per health check (every 5 s by default), and at that traffic balancing doesn't matter.

I could add something like this

// position in `this.connections` of the connection last returned by round-robin
private lastRoundRobinPosition = -1;

private selectRoundRobin(candidates: PDConnection[], latencyToleranceMs?: number) {
    let eligible = candidates;

    if (latencyToleranceMs !== undefined) {
        const minLatency = fastestConnection(candidates).latency;
        eligible = candidates.filter((c) => c.latency <= minLatency + latencyToleranceMs);
    }

    // `eligible` keeps the order of `this.connections`, so continuing after the last returned
    // position does not repeat or skip hosts when hosts join or leave between health checks
    const next =
        eligible.find((c) => this.connections.indexOf(c) > this.lastRoundRobinPosition) ??
        eligible[0];
    this.lastRoundRobinPosition = this.connections.indexOf(next);

    return next;
}

it works, but it adds complexity for no real gain i think
I suggest keeping current approach

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

this.balancingCursor = index + 1;

return eligible[index];
}

private checkConnectionsAvailability() {
Expand Down
25 changes: 25 additions & 0 deletions lib/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,30 @@ import type {PGDispatcher} from './dispatcher';

export type TopologyMode = 'primary-replica' | 'proxy';

/**
* always the connection with the lowest latest health-check latency;
* among equally fast ones, the first in the connections list (default)
*/
export interface FastestBalancingStrategy {
kind: 'fastest';
}

/**
* every healthy connection in turn; with `latencyToleranceMs`, only those whose latest
* health-check latency is within that many ms of the lowest one
*/
export interface RoundRobinBalancingStrategy {
kind: 'round-robin';
/**
* latency window in ms: when set, only connections whose latest health-check latency
* is within this value of the lowest one are rotated (0 — only the fastest ones).
* When omitted, all healthy connections are rotated
*/
latencyToleranceMs?: number;
}

export type BalancingStrategy = FastestBalancingStrategy | RoundRobinBalancingStrategy;

export interface PGConnectionStatus {
readonly host: string;
readonly healthy: boolean;
Expand Down Expand Up @@ -39,6 +63,7 @@ export interface PDOptions {
suppressHealthcheckStatusLogs?: boolean;
beforeTerminate: () => Promise<void>;
topologyMode: TopologyMode;
balancingStrategy?: BalancingStrategy;
}

export type Dict = {[key: string]: unknown};
Expand Down
Loading
Loading