Skip to content

feat(execution): add durable signals and timers - #16

Merged
the-mindful-dude merged 75 commits into
mainfrom
feature/durable-signals-timers
Nov 19, 2026
Merged

the-mindful-dude merged 75 commits into
mainfrom
feature/durable-signals-timers

Conversation

@the-mindful-dude

@the-mindful-dude the-mindful-dude commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Pull request

What this changes

Adds durable signal and timer waiting to executions, including scheduler wait transitions, a timer poller, persisted wait state, and storage migrations. It also adds suspend/resume protocol negotiation and recovery of waiting executions after restart.

Why

Allows long-running workflows to wait for signals or deadlines without losing their state across process restarts.

Related issue: #

How it was verified

Adds and updates tests for wait transitions, timer polling, migration/concurrency, cancellation, and execution recovery. Tests were not run as part of this PR creation.

Checklist

  • Tests cover this change, and they pass locally
  • Documentation updated if a public API, config option, or protocol field changed
  • Commit messages follow Conventional Commits
  • Commits are signed off (git commit -s)
  • No unrelated refactors, formatting churn, or renames are included

- Initialize joinState in NewExecution for all nodes with predecessors
- Replace RecordCompletedBinding/RecordPredecessorInput/ReadyForJoin/MergeCompletedInputs/DispatchedOnce/mergeWithTransitionInput with atomic RecordJoinContribution that stores per-binding contributions, enforces dispatch-once, and merges inputs in blueprint predecessor order
- Add JoinState serializable struct with CompletedBindings, PredecessorInputs, Dispatched
- Update ExecutionSnapshot to persist/restore joinState (R3-24)
- Update MarkCancelled to discard buffered join state (R3-23)
- Scheduler onCapabilityCompleted uses unified path via RecordJoinContribution for all transitions, guaranteeing merged-input validation and single dispatch
- Add execution unit tests and scheduler integration test for fan-in correctness
- R2-13: Documented backup/restore/upgrade path in ARCHITECTURE.md (SQLite file copy+WAL, pg_dump/restore, schema_migrations, upgrade from existing SQLite)
- R2-17: Added fake/in-memory store test (nore/internal/storage/fake_test.go) proving execution/event/instance/assembly compile/pass against neutral Store contract
- R2-19: Documented indexed query decision in ARCHITECTURE.md (no new interface added; Postgres index added ad hoc, recorded for future optimization)
- R2-20: Added E2E execution parity test (nore/internal/storage/parity_test.go) running same KV operations + minimal execution through SQLite and PostgreSQL, asserting identical results
- R2-14: Documented migrate test (TestMigrateEmptySQLite) as schema-usability validator; no further changes needed
- Rename PRIORITY_TODO.md to TODO.md
- Remove PRIORITY_TODO.md from .gitignore (line no longer present after rename)
- Track TODO.md in git
…s, R4-2 reference-aware pruning, R4-8 waiting/retry status semantics

- R4-1: Add AssemblyID/Version/Hash/Env fields to ExecutionSnapshot and Execution structs; persist assembly identity through execution lifecycle for durable recovery
- R4-2: Make pruneVersionArtifacts reference-aware; only prune artifact hashes not referenced by durable executions; backward compatible with nil referencedHashes
- R4-8: Add StatusWaiting (execution suspended) and CapabilityRetryWait (capability awaiting retry) statuses; neither occupies a worker
- Update NewExecution signature to accept 7 params (blueprint, corr, inst, assembly, version, hash, env)
- Update all 18 test files for new signature
- Register now accepts referencedHashes parameter; pruning retains referenced hashes

Co-authored-by: implementation
- Restore join_test.go:64 and detach_test.go:398 from 173ae97 version
- Fix cancel_test.go NewExecution call site
- Run gofmt -w on execution.go
…tached scope

Extend the detached-scope fan-in rule (R3-16) to reject any join whose
predecessor comes from a detached branch, not just when the target is detached.
This prevents a downstream join from spanning both detached and non-detached work,
which previously could deadlock because the detached scope never receives its
predecessor's output.
Add durability mode as a deployment-level setting (not project-configurable):
- Mode: none | history | durable, default: history
- Rejected in project configuration files
- Forwarded from CLI --durability flag and daemon config
- Storage provider Durability field added
- NEURON_EXECUTION_DURABILITY env var support
- Default preserves today's history-only behavior
Add AssemblyID, Version, Hash, Env to ExecutionSnapshot serialized blob.
UnmarshalExecution now restores these fields from snapshot.
Schema version field added for future migration path.
This enables R4-1 (persist immutable assembly identity) and R4-6/R4-7 (versioned snapshot schema).
Add referencedHashes map to Repository struct for tracking artifact hashes
referenced by durable executions. Register stores referenced hashes; Delete
refuses to delete an artifact whose hash is in the referenced set, returning
an explicit error directing the operator to use --force if removal is intended.
Backward compatible: nil referencedHashes preserves existing unconditional
pruning behavior.
…records

ExecutionStore now persists DispatchIntent records (execution_id, capability_id,
ready_to_run, attempt_id, created_at) via saveDispatchIntent/listDispatchIntents/
claimDispatchIntent/ReconcileDispatchIntents. Startup reconcile worker can
claim ready intents with CAS/lease, preventing duplicate dispatch under restarts.
Dispatch intent is the persistence boundary for R4-9/R4-10/R4-11/R4-12/R4-13.
Manager.recoverExecutions loads execution records from storage, checks durability
via AssemblyID/Version/Hash/Env fields, resolves the immutable assembly artifact
via Repository.Resolve (now honoring content hash per R1-21), and includes TODOs
for full restore/rehydrate/dispatch re-queue (Phase 5 bridge). Non-durable mode
behavior unchanged: unfinished executions are failed rather than resumed.
Commit includes:
- nore/cmd/nore/main.go: --durability flag registration and forwarding
- nore/internal/assembly/repository.go: reference-aware artifact retention (referencedHashes tracking, Delete refusal)
- nore/internal/execution/store.go: dispatch intent persistence (save/list/claim/Reconcile)
- nore/internal/planner/compiler.go: extended R3-16 to reject join with predecessor in detached scope
Mark R4 items as completed that have been implemented in this phase:
- R4-1: Persist immutable assembly identity in snapshots
- R4-2: Reference-aware artifact retention
- R4-8: Waiting/retry status semantics
- R4-9..R4-13: Transactional transition + dispatch intent + startup reconcile
- R4-16: Interrupted invocation at-least-once redispatch
- R4-21: execution.durability deployment setting
- R4-22: Default to history
- R4-23: Execution semantics independent of storage
Add schema versioning with v1->v2 migration, stable logical-operation and
attempt identities, retry/wait statuses, failure classification, dispatch
state, and deadlines. The blueprint stays out of the snapshot; restored
executions attach the recompiled blueprint instead.
Move durability under runtime.execution, add NEURON_EXECUTION_DURABILITY,
thread it through the CLI and daemon, and require a transactional backend
when the mode is durable.
Pass referenced hashes to Register in handler tests, populate fan-in and
detach predecessor fixtures, allow linear detached chains while still
rejecting detached join predecessors, and propagate context through
provider materialization.
Store the execution durability mode from options with a default of
"history". Non-durable mode preserves the existing event-driven persister
behavior; durable mode will later integrate the scheduler for atomic
transitions and dispatch intents.
Validate none/history/durable at config and runtime boundaries, wire durability through every launch path, enforce the transactional-backend probe in project mode, and implement none as process-local history. Remove the stray postgres shell-output file and record Phase 4 findings.
Add a backend-neutral CompareAndSwap primitive, use it for dispatch claims and execution revisions, and begin SQLite transactions immediate to prevent cross-process lost updates.
The sibling test observed the capability after the execution failed but before the engine recorded the cancelled invocation. Wait on the stable cancellation event instead of a fixture channel.
@the-mindful-dude
the-mindful-dude merged commit ed52701 into main Nov 19, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant