Skip to content

Finish the LLM run timeline and keep its waterfall inside the panel - #302

Open
iamwavecut wants to merge 3 commits into
mainfrom
feat/admin-llm-run-timeline
Open

iamwavecut wants to merge 3 commits into
mainfrom
feat/admin-llm-run-timeline

Conversation

@iamwavecut

Copy link
Copy Markdown
Owner

Summary

The LLM Dialogs run detail becomes a timeline with a "Where the time went" waterfall and a context rail. The waterfall no longer draws bars past its panel.

  • Timeline. Trigger, memory recall, prompt, model rounds, tool calls, auxiliary calls (vision), chat messages that arrived mid-run, runtime notes, outcome. Steps follow start time. It is built in the browser from the existing /admin/api/llm/dialogs/detail payload.
  • Waterfall. Each round, the tools it called and the auxiliary calls those tools made are drawn from their real start and end and nested under their parent. Queue (trigger message → run start), Prepare (run start → first call) and Finish (last call → run end) rows appear when they are long enough. A time scale and per-actor totals (model, tools, vision) sit above the rows. Every bar is clipped to its track.
  • Rail. Recall map (pl-constellation), persona and chat settings, run facts.
  • Colours. The step palette follows the actor: model (indigo, nested calls muted), tools (orange), people (accent: the trigger and chat messages), memory (violet), runtime (neutral). Light-theme overrides keep the step labels readable. A new --c-bg-raised token gives raised blocks a surface that differs from the card in both themes.
  • Logs tab. Entries from the SSE stream render their message, time and level.

Why the waterfall overflowed

RuntimeLlmRunBuffer::record_tool_result attached a tool result to the run's latest round. For understand_media, that round is the auxiliary vision call the tool itself made. The waterfall started the tool at the end of that call, and nothing clipped the bar. In one production run, three parallel understand_media calls ended 20% past the end of the run.

Backend changes

  • Tool results and sent markers attach to the latest non-auxiliary round.
  • RunToolCall.started_at records the wall-clock start. Parallel understand_media calls are recorded after the whole batch ends, so their start comes from the parallel executor.
  • One-off runs start at round.at - duration_ms. round.at is stamped when the call returns, so these runs showed a wall time of 1 ms.
  • TurnContextArtifact.chat_title fills the run origin's chat title.

Contract

These changes are additive. tool_calls[].started_at appears in the admin detail JSON and the llmRuns GraphQL JSON. chat_title uses serde(default). No routes, IDs or existing fields change.

Verification

  • cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings

  • cargo test -p openplotva-web: guard tests and asset hashes, 21 passed

  • cargo test -p openplotva-app --lib -- runtime_llm_runs dialog_turn: 44 passed, including the new tools_and_sent_markers_skip_nested_auxiliary_rounds and the one-off start assertion

  • Every inline admin script and components.js pass node --check.

  • Live production payloads, rendered by the branch's page without a deploy, checked at 2871, 1440, 768 and 390 px, in the dark and light themes, collapsed and fully expanded:

    • no bar leaves its track;
    • the page never scrolls horizontally;
    • there are no JS errors.

    Cases covered: parallel media tools with vision, vision before the first round, send_message + web_search, a running run, one-off memory_subject_merge/optimize_prompt, and a failed run made from a copy of a real run in the browser.

  • tools/service-smoke.web-ui.spec.js gains real timings and waterfall assertions. The same assertions were checked in the browser against this fixture. The Docker smoke itself was not run.

The run buffer attached a tool result to whatever round came last. For
understand_media that is the auxiliary vision call the tool itself made,
so the admin waterfall started the tool at the end of that call and drew
it past the end of the run. Tool results and sent markers now go to the
latest non-auxiliary round, tool calls carry their wall-clock start
(parallel understand_media calls are recorded only after the batch), and
one-off runs start when their call began rather than when it returned.
The waterfall draws each round, the tools it called and the auxiliary
calls those tools made from their real start and end, clips every bar to
its track, and adds the queue before the run, the preparation before the
first call, a time scale and per-actor totals. Timeline steps follow
start time, tools resolve their names by tool_call_id, and step colours
follow the actor: model, tools, people, memory, runtime. Major steps are
raised tone blocks; tools, auxiliary calls and chat messages are rows on
the spine. The header and the waterfall reflow on narrow screens.
@github-actions

Copy link
Copy Markdown
Warnings
⚠️ Large PR (4079 changed lines). Consider splitting unless this is generated/mechanical.
Messages
📖 Danger checked 12 changed file(s).

Generated by 🚫 dangerJS against e83b2d5

@iamwavecut

iamwavecut commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner Author

Keeping this as one PR. 3888 of the 4079 changed lines are the admin page (web/admin/): one feature, the run timeline. Its waterfall, step renderers and rail render one run together. The backend fix is a separate commit (7f91e2d, tool calls attach to the round that issued them) and can be reviewed on its own. Splitting would give a timeline without its waterfall fix, or a fix for a page that does not exist yet.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant