Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .claude/knowledge/journal/2026-11-20.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Learning journal 2026-11-20

### 2026-11-20: Resources, networking and performance — Optimized Arrow row paths must be checked against the legacy converter across all value shapes
**Context:** PR #628 replaced Table.toArray() with direct per-batch vector converters, PR #633 added a fast path for Decimal128, and PR #637 hardened parity tests. Each test compares the new output with the legacy getObjectRows output for scalar, struct, list, map and dictionary-encoded columns. Dictionary decimals turned out to keep the legacy unscaled-string form.
**Rule:** When optimizing Arrow row materialization, assert deep equality with the legacy converter for scalar, nested (struct/list/map), null, empty and dictionary-encoded columns in both object and array row modes, and keep any legacy quirks you find instead of silently fixing them.

### 2026-11-20: Resources, networking and performance — Object-row fast paths need a fallback for duplicate, __proto__ or mismatched column names
**Context:** In PR #628, the direct-vector object-row path builds rows keyed by column name. It falls back to the Table path when names repeat, a name is `__proto__`, or Arrow field names or counts differ from the Thrift schema.
**Rule:** When building plain-object rows keyed by server column names, fall back to the old path (or use a safe key strategy) if names are duplicated, include `__proto__`, or do not line up with the Arrow fields.

### 2026-11-20: Errors, logging and telemetry — Prove the instrumented path ran before asserting a spy was not called
**Context:** PR #633's test checks the decimal fast path by stubbing `byteLength` getters and asserting they are not read for safe integers. It also asserts `vector.get` was called once per value, because otherwise the notCalled checks would prove nothing.
**Rule:** When a test asserts that a spy or getter was not called to show a fast path was taken, also assert that the stubbed inputs were actually used, and check that the slow-path cases do trigger the spy.

### 2026-11-20: Resources, networking and performance — Node Buffers from pools are views; respect byteOffset/length
**Context:** PR #627 dropped a copy by passing `response.buffer()` output straight to Arrow decoding. A test checks that a small pooled Buffer (where `buffer.byteLength` is larger than `length`) still decodes, and also tests subarray bodies.
**Rule:** When passing Node Buffers to downstream decoders without copying, test with pooled and subarray Buffers, and never read `buf.buffer` without applying `byteOffset` and `length`.

### 2026-11-20: Bot workflow and CI — Kernel e2e path filters must include backend-shared modules
**Context:** PR #628 changed lib/result/ArrowResultConverter.ts, which both backends use. It had to add lib/result/ and the shared e2e util to the kernel-e2e workflow's changed-files regex, because otherwise the kernel e2e tests would be skipped.
**Rule:** When changing code shared by the Thrift and kernel backends (e.g. lib/result/ or shared e2e utils), make sure the kernel-e2e workflow's path filter covers those paths so the kernel e2e suite actually runs.
7 changes: 6 additions & 1 deletion .claude/knowledge/lessons.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Lessons

<!-- distilled-through: 2026-11-18 -->
<!-- distilled-through: 2026-11-20 -->

## Configuration and authentication
- L-0001 (3×, #457): Trace option support through DBSQLClient, auth adapters and each transport. Type declarations and class comments do not establish forwarding; shared preparation can precede backend-specific filtering.
Expand Down Expand Up @@ -49,6 +49,7 @@
- L-0040 (1×, #596): Error telemetry may carry class, SQLSTATE and terminal/retryable status only; exclude messages, stacks, URLs and other free-form error text.
- L-0041 (1×, #608): Decode every napi error, including status/hasMore calls. Any call that may fetch must share fetchChunk's best-effort failure cleanup, with rejecting-fake tests per call site.
- L-0042 (1×, #619): Detect user close/cancel with dedicated persistent intent flags read before cleanup, not lifecycle.isClosed. Fetch-failure cleanup must still close cancelled operations.
- L-0071 (1×, #633): Spy notCalled assertions proving a fast path need a positive check that the stubbed inputs were used, plus slow-path cases that do trigger the spy.

## Resources, networking and performance
- L-0043 (1×, #497): Portable Bash with nounset must guard empty-array expansion on older shells and distinguish unset-only defaults from unset-or-empty defaults.
Expand All @@ -68,6 +69,9 @@
- L-0057 (1×, #595): ProxyAgent needs TLS options at its top level. Verify proxy TLS/mTLS with a real CONNECT proxy and generated test certificate.
- L-0058 (1×, #577): Merge Arrow result rows with indexed concatChunks copies. Preserve provider batches and track owned leftover slices with offsets.
- L-0059 (1×, #611): Every auxiliary HTTP request, including OAuth, exchange and CloudFetch, must use the connection provider's getAgent to honor proxy/TLS settings.
- L-0068 (1×, #637 #633 #628): Optimized Arrow row paths must deep-equal the legacy converter for scalar, struct/list/map, null, empty and dictionary columns in object and array modes; preserve legacy quirks (e.g. dictionary decimals).
- L-0069 (1×, #628): Object rows keyed by server column names must fall back to the legacy path for duplicate names, __proto__, or Arrow field names/counts that mismatch the Thrift schema.
- L-0070 (1×, #627): Zero-copy Buffer handoff to decoders must honor byteOffset/length; test pooled and subarray Buffers and never read buf.buffer whole.

## Bot workflow and CI
- L-0060 (1×, #499): In bot workflows with both schedule and workflow_dispatch triggers, keep numeric input knobs string-typed and parse them downstream.
Expand All @@ -78,3 +82,4 @@
- L-0065 (1×, #566): Build after installs that omit prepare so generated lib/version.ts exists before tests/type checks.
- L-0066 (1×, #566 #562): Serialize branch-editing bot flows with one per-PR concurrency group and restore dependency-setup lockfile rewrites before agent edits.
- L-0067 (1×, #621 #620): GitHub Actions expressions allow hyphens in dot-dereferenced names; do not flag or rewrite `${{ inputs.some-name }}` as subtraction.
- L-0072 (1×, #628): Changes to backend-shared code (lib/result/, shared e2e utils) must be covered by the kernel-e2e workflow path filter so the kernel suite actually runs.
Loading