diff --git a/.claude/knowledge/journal/2026-11-20.md b/.claude/knowledge/journal/2026-11-20.md new file mode 100644 index 00000000..87c86283 --- /dev/null +++ b/.claude/knowledge/journal/2026-11-20.md @@ -0,0 +1,21 @@ +# Learning journal 2026-11-20 + +### 2026-11-20: Resources, networking and performance — Optimized Arrow row paths must be checked against the legacy converter across all value shapes +**Context:** PR #628 replaced Table.toArray() with direct per-batch vector converters, PR #633 added a fast path for Decimal128, and PR #637 hardened parity tests. Each test compares the new output with the legacy getObjectRows output for scalar, struct, list, map and dictionary-encoded columns. Dictionary decimals turned out to keep the legacy unscaled-string form. +**Rule:** When optimizing Arrow row materialization, assert deep equality with the legacy converter for scalar, nested (struct/list/map), null, empty and dictionary-encoded columns in both object and array row modes, and keep any legacy quirks you find instead of silently fixing them. + +### 2026-11-20: Resources, networking and performance — Object-row fast paths need a fallback for duplicate, __proto__ or mismatched column names +**Context:** In PR #628, the direct-vector object-row path builds rows keyed by column name. It falls back to the Table path when names repeat, a name is `__proto__`, or Arrow field names or counts differ from the Thrift schema. +**Rule:** When building plain-object rows keyed by server column names, fall back to the old path (or use a safe key strategy) if names are duplicated, include `__proto__`, or do not line up with the Arrow fields. + +### 2026-11-20: Errors, logging and telemetry — Prove the instrumented path ran before asserting a spy was not called +**Context:** PR #633's test checks the decimal fast path by stubbing `byteLength` getters and asserting they are not read for safe integers. It also asserts `vector.get` was called once per value, because otherwise the notCalled checks would prove nothing. +**Rule:** When a test asserts that a spy or getter was not called to show a fast path was taken, also assert that the stubbed inputs were actually used, and check that the slow-path cases do trigger the spy. + +### 2026-11-20: Resources, networking and performance — Node Buffers from pools are views; respect byteOffset/length +**Context:** PR #627 dropped a copy by passing `response.buffer()` output straight to Arrow decoding. A test checks that a small pooled Buffer (where `buffer.byteLength` is larger than `length`) still decodes, and also tests subarray bodies. +**Rule:** When passing Node Buffers to downstream decoders without copying, test with pooled and subarray Buffers, and never read `buf.buffer` without applying `byteOffset` and `length`. + +### 2026-11-20: Bot workflow and CI — Kernel e2e path filters must include backend-shared modules +**Context:** PR #628 changed lib/result/ArrowResultConverter.ts, which both backends use. It had to add lib/result/ and the shared e2e util to the kernel-e2e workflow's changed-files regex, because otherwise the kernel e2e tests would be skipped. +**Rule:** When changing code shared by the Thrift and kernel backends (e.g. lib/result/ or shared e2e utils), make sure the kernel-e2e workflow's path filter covers those paths so the kernel e2e suite actually runs. diff --git a/.claude/knowledge/lessons.md b/.claude/knowledge/lessons.md index 0b3ae3f9..1839bda1 100644 --- a/.claude/knowledge/lessons.md +++ b/.claude/knowledge/lessons.md @@ -1,6 +1,6 @@ # Lessons - + ## Configuration and authentication - L-0001 (3×, #457): Trace option support through DBSQLClient, auth adapters and each transport. Type declarations and class comments do not establish forwarding; shared preparation can precede backend-specific filtering. @@ -49,6 +49,7 @@ - L-0040 (1×, #596): Error telemetry may carry class, SQLSTATE and terminal/retryable status only; exclude messages, stacks, URLs and other free-form error text. - L-0041 (1×, #608): Decode every napi error, including status/hasMore calls. Any call that may fetch must share fetchChunk's best-effort failure cleanup, with rejecting-fake tests per call site. - L-0042 (1×, #619): Detect user close/cancel with dedicated persistent intent flags read before cleanup, not lifecycle.isClosed. Fetch-failure cleanup must still close cancelled operations. +- L-0071 (1×, #633): Spy notCalled assertions proving a fast path need a positive check that the stubbed inputs were used, plus slow-path cases that do trigger the spy. ## Resources, networking and performance - L-0043 (1×, #497): Portable Bash with nounset must guard empty-array expansion on older shells and distinguish unset-only defaults from unset-or-empty defaults. @@ -68,6 +69,9 @@ - L-0057 (1×, #595): ProxyAgent needs TLS options at its top level. Verify proxy TLS/mTLS with a real CONNECT proxy and generated test certificate. - L-0058 (1×, #577): Merge Arrow result rows with indexed concatChunks copies. Preserve provider batches and track owned leftover slices with offsets. - L-0059 (1×, #611): Every auxiliary HTTP request, including OAuth, exchange and CloudFetch, must use the connection provider's getAgent to honor proxy/TLS settings. +- L-0068 (1×, #637 #633 #628): Optimized Arrow row paths must deep-equal the legacy converter for scalar, struct/list/map, null, empty and dictionary columns in object and array modes; preserve legacy quirks (e.g. dictionary decimals). +- L-0069 (1×, #628): Object rows keyed by server column names must fall back to the legacy path for duplicate names, __proto__, or Arrow field names/counts that mismatch the Thrift schema. +- L-0070 (1×, #627): Zero-copy Buffer handoff to decoders must honor byteOffset/length; test pooled and subarray Buffers and never read buf.buffer whole. ## Bot workflow and CI - L-0060 (1×, #499): In bot workflows with both schedule and workflow_dispatch triggers, keep numeric input knobs string-typed and parse them downstream. @@ -78,3 +82,4 @@ - L-0065 (1×, #566): Build after installs that omit prepare so generated lib/version.ts exists before tests/type checks. - L-0066 (1×, #566 #562): Serialize branch-editing bot flows with one per-PR concurrency group and restore dependency-setup lockfile rewrites before agent edits. - L-0067 (1×, #621 #620): GitHub Actions expressions allow hyphens in dot-dereferenced names; do not flag or rewrite `${{ inputs.some-name }}` as subtraction. +- L-0072 (1×, #628): Changes to backend-shared code (lib/result/, shared e2e utils) must be covered by the kernel-e2e workflow path filter so the kernel suite actually runs.