Repository navigation
Conversation
Clay looked up cached text measurements by hash alone, so two strings whose hashes collide shared one measurement: the second was sized and wrapped as the first, in the same frame or a later one. A Clay patch now also compares the text bytes and config on lookup. Each Term keeps copies of cached keys in its own storage, because a frame's wire bytes do not outlive its render. Storage is reclaimed by resetting Clay's cache before a layout that might not fit, and a frame larger than the storage keys on its own bytes, with the cache cleared once it has been presented.
Changeset suggestionThe current changeset no longer matches this PR. This review contains a corrected replacement. Why: Changeset package scope does not match the affected packages confidently. Changeset is not sufficiently consumer-focused. View the proposed changeset---
'@bomb.sh/tty': patch
---
TODO: Write changeset message (inference failed: HTTP 401: {"error":{"message":"User not found.","code":401}})Review this changeset manually If this draft is correct, react with 🚀 and Cooper will commit it to this branch. |
| @@ -0,0 +1,5 @@ | |||
| --- | |||
There was a problem hiding this comment.
Changeset needs revision.
Changeset package scope does not match the affected packages confidently. Changeset is not sufficiently consumer-focused.
View the proposed replacement
---
'@bomb.sh/tty': patch
---
TODO: Write changeset message (inference failed: HTTP 401: {"error":{"message":"User not found.","code":401}})
commit: |
There was a problem hiding this comment.
I had a look at xxhash.com as suggested by the original Clay author in one of the linked issues, and ran a prototype to see if I could get it to work. It did.
Could we use XXH3-128 for the measurement cache instead of retaining copies of the text? We could pin xxHash as a git submodule and include its header through the existing Clay patch mechanism.
The patch would store and compare the full 128-bit fingerprint, along with text length and the relevant font settings. Clay could still derive its bucket index from part of the hash.
This would have two main benefits:
• Significantly lower memory usage: approximately 192 KiB of additional cache storage at maximum capacity, avoiding the roughly 2 MiB text-copy arena.
• The fix stays entirely within Clay: apart from adding the dependency and patch, no changes to tty’s renderer or memory-management code would be needed. There would be no key-copying callback, storage rollover, or borrowed-key lifetime to manage.
In total, it looks like it would add around 3 KiB to the bundled wasm.ts.
The tradeoff is that a 128-bit fingerprint makes accidental collisions possible, but extraordinarily unlikely.... as in like really unlikely, whereas comparing the original bytes guarantees correct collision handling. If that tradeoff is acceptable, then it would be substantially simpler to maintain.
Replaces the retained copies of cached text with a 128-bit fingerprint, as suggested in review. The Clay patch stores the XXH3-128 hash of each measured text and reuses an entry only when the fingerprint, the text length and the text config all match. The bucket index comes from the low 32 bits of the hash. The renderer no longer takes part: the key-copy hook, the per-instance key storage and the borrowed-key path are removed, and the tests that exercised them go with them. The two collision regressions remain. xxHash is pinned as a submodule at v0.8.3 and included from the Clay patch. src/string.h supplies the <string.h> it includes, since the build is freestanding. The measurement cache paragraph is dropped from the renderer spec as a private implementation detail.
|
It worked you as suggested. I updated the patch.
|
What does this PR do?
Fixes incorrect text widths caused by hash collisions in Clay's measurement cache. Two ASCII strings with widths 57 and 91 share a 32-bit cache ID; measuring them in one frame or across renders returns 57 for both, and the longer string loses its tail.
Refs nicbarker/clay#664, which includes a standalone C reproduction against current Clay main.
The Clay patch keys each cache entry on an XXH3-128 fingerprint of the text. An entry is reused only when the fingerprint, the text length and the text config (font id, font size, letter spacing) all match. The bucket index comes from the low 32 bits of the hash. Statically allocated strings are fingerprinted by address, as Clay's existing hash does.
The fix stays inside the Clay patch. The renderer (
src/clayterm.c,src/clayterm.h) is unchanged frommain, and public APIs are unchanged.Dependency
xxhash/(v0.8.3) and included from the Clay patch withXXH_INLINE_ALLandXXH_NO_STREAM.src/string.hsupplies the<string.h>thatxxhash.hincludes, since the build is freestanding.xxhashis added to the fmt/lint excludes, the benchmark workflow path filters andBUILD.md.Trade-off
A 128-bit fingerprint makes an accidental collision extraordinarily unlikely rather than impossible. Because the text length is compared as well, a colliding entry's measured words stay within the bounds of the new string, so the worst case is a wrong width, not an out-of-bounds read. XXH3 is not a cryptographic hash.
Cost against
mainBuilt locally with clang 20.1.7:
mainwasm.tsclayterm_size(80, 24)Tests
Two regressions in
test/width.test.tscover the colliding pair in one frame and across renders. Both fail onmainand pass here.Local validation at
334252c: WASM build, full Deno suite (29 tests, 393 steps),deno task fmt:check, lint and typecheck pass.gen-wcwidth:checkwas not run locally.Toolchain note
Linking with clang 24 fails with
undefined symbol: __multi3: that compiler turns xxHash's portable 64×64 multiply into a 128-bit multiply, which needs a compiler-rt helper the freestanding build does not link. Clang 20 builds clean. No shim is included.Type of change
Checklist
deno task test)deno task fmt,deno task fmt:check)AI-generated code disclosure