Skip to content

eval: round 15 prep (board task, six-arm references, r15 fuzz rules, prereg draft) - #71

Open
ryansolid wants to merge 45 commits into
mainfrom
claude/project-thread-xp2irv
Open

ryansolid wants to merge 45 commits into
mainfrom
claude/project-thread-xp2irv

Conversation

@ryansolid

@ryansolid ryansolid commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

This branch prepares round 15, the six-arm, two-task round meant for publication. Every task now has a reference that grades 0 in every arm, with by-design holds excluded.

What's in it

  • New held-out task tasks/board. A sprint board with:

    • brief (approved), API, fixtures;
    • tests: visible 26, hidden 45, overlap 26;
    • harness/fuzz-board.mjs, the server model plus oracles. Allowances B1 to B3 are documented at the top of the file. It also checks each card's count against the server.
  • Board references:

    arm lines grade
    rx (rx-cols) 429 0; excluded: 1 hidden (H-11), 2 overlap (shared-list holds), 1 overlap (legal)
    react 681 0
    contract 510 0
    xstate 890 0
    foldkit 1293 0
    vanilla 752 0
    • Every other arm grades hidden 45/45, overlap 26/26, fuzz 0/60.
    • rx-cols is one keyed optimistic store per fetch and every write is a guess. It is graded on Solid next eafe671b6: fuzz 0/60, details in rx-cols/GRADE.txt.
    • rx (its own write ledger), rx-guide and rx-store (one flat store) are kept for comparison.
  • .rx changes from writing the reference:

    • keys apply at any depth (ST-4a, taken by default), in the compiler, ref runtime and oracle;
    • guide async.md, "One store over the server";
    • catchRead snapshot in handlers, ARGUMENT_COUNT, props in C() UNBOUND, DV-22 extended.
  • desk9 references for react, xstate, foldkit and vanilla, all graded 0.

  • Rulings now in the harness:

    • fuzz.mjs --r15 drops A2: at rest, a field must show the server's last answer.
    • A4 now also covers the detail, labels, comments and two-views checks for writes that land after their page was left.
    • grade.sh takes TASK= and FUZZ_ROUND=.
  • Solid repros, in results/r15prep/repro/:

    • for solidjs/solid #4856, #4859, #4888, #4889, #4897 and #4902, all fixed;
    • for #4904 (open): a card moved into a column that just emptied renders as undefined.
  • PREREG-15.md, a DRAFT, not frozen.

    • It sets out the arms, builds (released only), grading, 144 runs and the H15a/b/c hypotheses.
    • It also covers the shared-list hold exclusion, the Solid fixes the frozen build must contain, and the caveat that the board guide lines came from writing the reference.

Not in it


🤖 Generated with Claude Code

https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
A write that lands after its page was left is not judged on the next visit
(Ryan 2026-11-14: late writes are out of scope). A4 was only applied to rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…XState references follow it

Ryan 2026-11-14: with two accepted writes to one field, the server's last answer is the
only correct value at rest. --r15 drops allowance A2; FUZZ_ROUND=--r15 sh harness/grade.sh.
All six desk9 references grade 0 under it (20 seeds; React, Exact, XState, Foldkit and
vanilla also 0/60 on seeds 21-80).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…erence, fuzz seed 45)

Also replay.mjs: DW=<n> widens the detail text it prints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…ence (grades 0)

Brief frozen 2026-11-14 (Ryan). 26 visible, 45 hidden, 26 overlap tests; fuzz-board.mjs
with oracles for duplicates, headings, counts, the work-in-progress limit, the detail,
and settled state. TASK=board sh harness/grade.sh grades it. The fuzzer catches two
planted bugs in the reference (limit off by one, an accepted toggle not kept).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…el under a pending guess

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…rite visit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…15 prereg draft

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Failed ❌

View logs ↗
3289544 2026-11-16T11:29:18.797Z View logs ↗
  • Build: Failed ❌

View logs ↗
192e36c 2026-11-16T11:06:38.460Z View logs ↗
  • Build: Failed ❌

View logs ↗
cd47f41 2026-11-16T10:57:29.043Z View logs ↗
  • Build: Failed ❌

View logs ↗
2d357af 2026-11-16T10:37:37.044Z View logs ↗
  • Build: Failed ❌

View logs ↗
670f26e 2026-11-16T07:00:06.468Z View logs ↗
  • Build: Failed ❌

View logs ↗
486af79 2026-11-15T22:42:14.191Z View logs ↗
  • Build: Failed ❌

View logs ↗
8395412 2026-11-15T22:25:13.144Z View logs ↗
  • Build: Failed ❌

View logs ↗
6146c67 2026-11-15T20:19:18.772Z View logs ↗
  • Build: Failed ❌

View logs ↗
a93efdb 2026-11-15T18:19:53.431Z View logs ↗
  • Build: Failed ❌

View logs ↗
f482392 2026-11-15T18:17:35.105Z View logs ↗

View all previews: View all previews ↗

… answered add's placeholder while work is in the air

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…, minimal trail)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…s/solid#4874, rx runtime catchRead, pending probe)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…); guide: one store over the server

- Runtime: `catchRead` with no observer (a component fn called from a handler or
  action) is a snapshot (04 R-8): the committed value, or the alternative on a
  matching failure. A fresh error boundary there had no value while a refresh
  was out and threw NotReadyError (S-catch-snapshot).
- Checker (ref/ and crate): ARGUMENT_COUNT (06 AC-1f) for a call of a declared
  action with the wrong number of arguments; `props` in `component C()` is
  UNBOUND (03 RG-4b). AST gains `noProps`. S-check-1 rejected variants, a
  check.rs unit test, guide/diagnostics.md sections.
- pending? inside a held `loading on(sel)` region reads false: Solid's, under
  05 DV-22 (plain-Solid repro saved). Guide and GUESS_WHILE_LOADING text say
  to test `pending? x` in the handler too (R-12).
- guide/async.md: "One store over the server".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…k shown column), fuzz B3 declined count hold, repro for solidjs/solid#4887

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…on, two Solid repros

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…and by key; record seed)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…(seed 28); repro for solidjs/solid#4897

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…re-ask the row's current answer; task-neutral examples

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…/60 on next 3dd86e8f2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…to another store)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…ST-4a changes from writing it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…cile; DV-19/20/23 fixed (#64)

- ref/SOLID_NEXT -> rc.22 (221b5f9); runtime and eval deps ^2.0.0-rc.22
- landDraft returns a keyed answer from the derive (Solid reconciles by key at any depth, ST-4a);
  unkeyed answers still assigned. Drops the hand-written keyed merge from #63.
- S-guess-reask-refresh, S-latest-reask, S-solid-46: divergence records removed (fixed in rc.22)
- S-solid-44: only DV-22 (pending? under a new question) still recorded, instant 3
- All ref runners green: oracle/compiled 131/131, ssr 124 (7 skip), types 177, crate 367/367

Closes #64

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
@socket-security

socket-security Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatednpm/​@​solidjs/​web@​2.0.0-rc.21 ⏵ 2.0.0-rc.22100 +110083 +196 -1100
Updatednpm/​@​solidjs/​compiler@​2.0.0-rc.21 ⏵ 2.0.0-rc.2299 +1100100 +195 -1100
Updatednpm/​solid-js@​2.0.0-rc.21 ⏵ 2.0.0-rc.229910095 +196100
Updatednpm/​@​solidjs/​signals@​2.0.0-rc.21 ⏵ 2.0.0-rc.22100 +1100100 +196 -1100

View full report

…path guess when the row leaves and returns)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
… fuzz trails (jsdom)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…ter retry; regression from #4908)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…d key; repro for solidjs/solid#4928

A key asked twice (an archive asked on two visits) was answered twice and the last answer went to the app: the
first archived the card, the second found nothing and failed, so the app reverted while the server had archived.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…120)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt
…tem toggled twice in a busy period

From Ryan's ruling on two accepted writes to one field (settled = the server's last answer): the answers can come
back in another order than the clicks, and the screen keeps the last click until its re-ask lands. Board seed 70.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0117BoWLtkg281f1NkGEDjCt

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants