Skip to content

feat(ai): send tool results and errors on tool spans - #1283

Open
DanielVisca wants to merge 4 commits into
mainfrom
posthog/ai-tool-span-results
Open

DanielVisca wants to merge 4 commits into
mainfrom
posthog/ai-tool-span-results

Conversation

@DanielVisca

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Handover: this is ready for the AI observability team to review and own. Please don't merge without your sign-off.

AI observability counts how often each tool is called, but can't tell how often a tool fails. Two integrations send tool spans that are missing what's needed for that:

  • Claude Agent SDK: the tool $ai_span was sent as soon as the model asked for the tool, before it ran. It never had the tool's output, error flag or latency.
  • LangChain: tool spans had no $ai_span_type, so they couldn't be told apart from chain spans. A tool that reports its failure as a result (handle_tool_error, ToolMessage(status="error")) ended without an exception, so the span wasn't marked as an error. Converted ToolMessages also dropped tool_call_id, name and status, so a tool result in the next generation's $ai_input couldn't be matched to the tool that produced it.

Changes:

  • Claude Agent SDK (processor.py, client.py): each tool span waits for its ToolResultBlock (matched by tool_use_id, including subagent results), then goes out with $ai_output_state (hidden in privacy mode, like $ai_input_state), $ai_is_error, $ai_error and $ai_latency. Spans still waiting are sent without a result before the trace, when the query ends, if the caller stops iterating early, and on disconnect(). The query loop is now inside try/finally for the early-stop case, so most of that hunk is re-indentation (git diff -w shows the real change).
  • LangChain (callbacks.py): tool runs set $ai_span_type: "tool". A tool that returns a ToolMessage with status == "error" marks its span $ai_is_error with $ai_error. Converted ToolMessages keep tool_call_id, name when set, and status: "error".
  • utils.py: _format_tool_error builds $ai_error from tool result content, joining text blocks. In privacy mode it returns the fixed string ToolError, because tool output can contain user data. It sends that label instead of leaving $ai_error out so the span still reads as an error in the errors view. This is a judgement call for reviewers.

Before / After

The same two runs against a mocked client: a Claude Agent SDK Read call whose result is is_error=True, and a LangChain tool that raises ToolException with handle_tool_error=True. $ai_span properties, trimmed:

Before (main)

Claude Agent SDK: {"$ai_span_name": "Read", "$ai_span_type": "tool"}
LangChain:        {"$ai_span_name": "get_weather",
                   "$ai_output_state": {"content": "weather service unavailable", "status": "error"},
                   "$ai_latency": 0.0}

After (this branch)

Claude Agent SDK: {"$ai_span_name": "Read", "$ai_span_type": "tool",
                   "$ai_is_error": true, "$ai_error": "No such file or directory",
                   "$ai_output_state": [{"type": "text", "text": "No such file or directory"}],
                   "$ai_latency": 0.0}
LangChain:        {"$ai_span_name": "get_weather", "$ai_span_type": "tool",
                   "$ai_is_error": true, "$ai_error": "weather service unavailable",
                   "$ai_output_state": {"content": "weather service unavailable", "status": "error"},
                   "$ai_latency": 0.0}

This pairs with an ingestion change in PostHog that reads tool results out of $ai_input to compute per-tool failure rates. With tool_call_id kept, LangChain results can be matched to tool names there too.

💚 How did you test it?

  • uv run pytest posthog/test/ai: 1881 passed, 15 skipped.
  • New tests: test_processor.py (result attached, error, privacy mode, subagent match, sent once and before the trace, sent when the caller stops early), test_client.py (out-of-order results against both the processor and the client, disconnect() flush, span assertions in the multi-turn test), and test_callbacks.py (span type, error ToolMessage, message conversion).
  • ruff format, ruff check.
  • The before/after above came from a scratch script run against main and this branch with a mocked client. I haven't run it against a live Claude Agent SDK or LangChain agent.

📝 Checklist

  • All commits are signed (see Commit signing)
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Written by Claude Opus 5.5 and Codex in Claude Code, directed by Daniel, as a handover to the AI observability team. Tool spans now wait for their result rather than going out with a placeholder and an update event, because ingestion and the trace view read one span per tool call.

🤖 Generated with Claude Code

Claude Agent SDK tool spans now wait for the matching tool result and carry
the output, error flag, error text and latency. LangChain tool spans set
$ai_span_type to tool, mark tools that return an error ToolMessage, and
converted ToolMessages keep tool_call_id, name and error status, so tool
results can be matched to tool names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DanielVisca
DanielVisca requested a review from a team as a code owner November 20, 2026 03:05
@DanielVisca DanielVisca self-assigned this Nov 20, 2026
@veria-ai

veria-ai Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Reviews

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed f26dc95: 1 Must fix, 3 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Comment thread posthog/ai/utils.py Resolved
Comment thread posthog/ai/langchain/callbacks.py Resolved
Comment thread posthog/ai/langchain/callbacks.py Resolved
@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

SDK performance: posthog-python

Benchmarks completed. No changes flagged for review.

Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate.

View all results
Test Measurement Before PR With PR Change Assessment
Capture and deliver events (32-byte payload, burst)
Shard: capture
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Capture and deliver events (32-byte payload, steady)
Shard: capture
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Queue an event (32-byte payload)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​; head:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​
Queue an event (32-byte payload)
Shard: capture
Allocations per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​; head:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​
Queue an event (4,096-byte payload)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​; head:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​
Queue an event (4,096-byte payload)
Shard: capture
Allocations per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​; head:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Refresh 1,000 flag definitions
Shard: flags-memory
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Capture and deliver events (32-byte payload, burst)
Shard: capture
Allocated memory per operation 133.67 KiB/op 133.67 KiB/op No observed change Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Allocations per operation 1,345.51 count/op 1,345.51 count/op No observed change Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Time per operation 77.83 µs/op 78.1 µs/op 0.3% more time Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Idle process memory (RSS) 37.62 MiB 37.68 MiB 0.2% more memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Observed peak process memory (RSS) 40.06 MiB 40.12 MiB 0.2% more memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Process memory after settling (RSS) 40.06 MiB 40.12 MiB 0.2% more memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Process CPU time per operation 80.99 µs/op 79.72 µs/op 1.6% less time Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Lifetime peak process memory (RSS) 40.38 MiB 40.38 MiB No observed change Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Allocated memory per operation 185.75 KiB/op 185.75 KiB/op <0.1% less memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Allocations per operation 1,337.51 count/op 1,337.51 count/op <0.1% fewer Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Time per operation 102.27 µs/op 102.1 µs/op 0.2% less time Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Idle process memory (RSS) 41.27 MiB 41.27 MiB <0.1% less memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Observed peak process memory (RSS) 47.05 MiB 47.05 MiB <0.1% less memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Process memory after settling (RSS) 47.05 MiB 47.05 MiB <0.1% less memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Process CPU time per operation 98.7 µs/op 99.35 µs/op 0.6% more time Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Lifetime peak process memory (RSS) 51.38 MiB 51.43 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Allocated memory per operation 138.73 KiB/op 138.73 KiB/op <0.1% less memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Allocations per operation 1,390.29 count/op 1,390.28 count/op <0.1% fewer Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Time per operation 88.74 µs/op 88.87 µs/op 0.1% more time Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Idle process memory (RSS) 37.56 MiB 37.59 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Observed peak process memory (RSS) 37.73 MiB 37.75 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Process memory after settling (RSS) 37.73 MiB 37.75 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Process CPU time per operation 91.06 µs/op 90.75 µs/op 0.3% less time Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Lifetime peak process memory (RSS) 40.38 MiB 40.38 MiB No observed change Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Allocated memory per operation 190.9 KiB/op 190.9 KiB/op No observed change Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Allocations per operation 1,382.19 count/op 1,382.19 count/op No observed change Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Time per operation 106.4 µs/op 105.95 µs/op 0.4% less time Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Idle process memory (RSS) 41.01 MiB 41.04 MiB <0.1% more memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Observed peak process memory (RSS) 41.48 MiB 41.48 MiB <0.1% more memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Process memory after settling (RSS) 41.07 MiB 41.09 MiB <0.1% more memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Process CPU time per operation 102.3 µs/op 104.41 µs/op 2.1% more time Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Lifetime peak process memory (RSS) 41.43 MiB 41.38 MiB 0.1% less memory Not flagged
Queue an event (32-byte payload)
Shard: capture
Caller-thread CPU time per operation 57.61 µs/op 57.47 µs/op 0.2% less time Not flagged
Queue an event (32-byte payload)
Shard: capture
Time per operation 57.6 µs/op 57.47 µs/op 0.2% less time Not flagged
Queue an event (4,096-byte payload)
Shard: capture
Caller-thread CPU time per operation 57.77 µs/op 57.72 µs/op <0.1% less time Not flagged
Queue an event (4,096-byte payload)
Shard: capture
Time per operation 57.76 µs/op 57.71 µs/op <0.1% less time Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Allocated memory per operation 225.43 KiB/op 225.43 KiB/op No observed change Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Allocations per operation 2,279.42 count/op 2,279.42 count/op No observed change Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Time per operation 82.6 µs/op 82.75 µs/op 0.2% more time Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Process CPU time per operation 82.59 µs/op 82.75 µs/op 0.2% more time Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Allocated memory per operation 2.17 MiB/op 2.17 MiB/op No observed change Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Allocations per operation 22,413.69 count/op 22,413.69 count/op No observed change Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Time per operation 771.68 µs/op 771.23 µs/op <0.1% less time Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Process CPU time per operation 771.7 µs/op 771.24 µs/op <0.1% less time Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Allocated memory per operation 21.63 MiB/op 21.63 MiB/op No observed change Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Allocations per operation 223,655.44 count/op 223,655.44 count/op No observed change Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Time per operation 7.66 ms/op 7.64 ms/op 0.3% less time Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Process CPU time per operation 7.66 ms/op 7.64 ms/op 0.3% less time Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Idle process memory (RSS) 43.84 MiB 43.79 MiB 0.1% less memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Observed peak process memory (RSS) 44.16 MiB 44.12 MiB 0.1% less memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Process memory after settling (RSS) 44.16 MiB 44.12 MiB 0.1% less memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Lifetime peak process memory (RSS) 44.16 MiB 43.93 MiB 0.5% less memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Idle process memory (RSS) 42.99 MiB 42.99 MiB <0.1% more memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Observed peak process memory (RSS) 43 MiB 43 MiB <0.1% more memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Process memory after settling (RSS) 43 MiB 43 MiB <0.1% more memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Lifetime peak process memory (RSS) 42.76 MiB 43.32 MiB 1.3% more memory Not flagged
Evaluate one boolean flag
Shard: flags-single
Allocated memory per operation 15.74 KiB/op 15.74 KiB/op No observed change Not flagged
Evaluate one boolean flag
Shard: flags-single
Allocations per operation 162.77 count/op 162.77 count/op No observed change Not flagged
Evaluate one boolean flag
Shard: flags-single
Time per operation 14.1 µs/op 14.12 µs/op 0.1% more time Not flagged
Evaluate one boolean flag
Shard: flags-single
Process CPU time per operation 14.1 µs/op 14.12 µs/op 0.2% more time Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Allocated memory per operation 24.6 KiB/op 24.6 KiB/op No observed change Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Allocations per operation 239.28 count/op 239.28 count/op No observed change Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Time per operation 22.67 µs/op 22.58 µs/op 0.4% less time Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Process CPU time per operation 22.67 µs/op 22.58 µs/op 0.4% less time Not flagged
Evaluate a flag (early match)
Shard: flags-single
Allocated memory per operation 39.81 KiB/op 39.81 KiB/op No observed change Not flagged
Evaluate a flag (early match)
Shard: flags-single
Allocations per operation 388.27 count/op 388.27 count/op No observed change Not flagged
Evaluate a flag (early match)
Shard: flags-single
Time per operation 28.42 µs/op 28.26 µs/op 0.6% less time Not flagged
Evaluate a flag (early match)
Shard: flags-single
Process CPU time per operation 28.42 µs/op 28.26 µs/op 0.6% less time Not flagged
Evaluate a flag (late match)
Shard: flags-single
Allocated memory per operation 60.59 KiB/op 60.59 KiB/op No observed change Not flagged
Evaluate a flag (late match)
Shard: flags-single
Allocations per operation 584.27 count/op 584.27 count/op No observed change Not flagged
Evaluate a flag (late match)
Shard: flags-single
Time per operation 39.47 µs/op 39.73 µs/op 0.7% more time Not flagged
Evaluate a flag (late match)
Shard: flags-single
Process CPU time per operation 39.47 µs/op 39.72 µs/op 0.6% more time Not flagged
Evaluate a flag (no match)
Shard: flags-single
Allocated memory per operation 59.22 KiB/op 59.22 KiB/op No observed change Not flagged
Evaluate a flag (no match)
Shard: flags-single
Allocations per operation 567.77 count/op 567.77 count/op No observed change Not flagged
Evaluate a flag (no match)
Shard: flags-single
Time per operation 35.88 µs/op 35.65 µs/op 0.6% less time Not flagged
Evaluate a flag (no match)
Shard: flags-single
Process CPU time per operation 35.88 µs/op 35.65 µs/op 0.6% less time Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Allocated memory per operation 19.04 KiB/op 19.04 KiB/op No observed change Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Allocations per operation 192.27 count/op 192.27 count/op No observed change Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Time per operation 16.77 µs/op 17.23 µs/op 2.7% more time Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Process CPU time per operation 16.77 µs/op 17.23 µs/op 2.7% more time Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Allocated memory per operation 20.66 KiB/op 20.66 KiB/op No observed change Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Allocations per operation 197.77 count/op 197.77 count/op No observed change Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Time per operation 17.99 µs/op 18.06 µs/op 0.3% more time Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Process CPU time per operation 17.99 µs/op 18.05 µs/op 0.3% more time Not flagged
Evaluate a rollout flag
Shard: flags-single
Allocated memory per operation 14.89 KiB/op 14.89 KiB/op No observed change Not flagged
Evaluate a rollout flag
Shard: flags-single
Allocations per operation 152.77 count/op 152.77 count/op No observed change Not flagged
Evaluate a rollout flag
Shard: flags-single
Time per operation 13.32 µs/op 13.38 µs/op 0.4% more time Not flagged
Evaluate a rollout flag
Shard: flags-single
Process CPU time per operation 13.32 µs/op 13.38 µs/op 0.4% more time Not flagged
Evaluate a multivariate flag
Shard: flags-single
Allocated memory per operation 15.9 KiB/op 15.9 KiB/op No observed change Not flagged
Evaluate a multivariate flag
Shard: flags-single
Allocations per operation 166.77 count/op 166.77 count/op No observed change Not flagged
Evaluate a multivariate flag
Shard: flags-single
Time per operation 14.93 µs/op 14.85 µs/op 0.5% less time Not flagged
Evaluate a multivariate flag
Shard: flags-single
Process CPU time per operation 14.93 µs/op 14.85 µs/op 0.5% less time Not flagged

Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals.

Each shard uses its own runner; compare Before PR against With PR within a row, not across shards.

Compared 5994ff313c6dfb0cbbe3d29f367fb3b7c0a80a94 → 2846827ae62b0d9f664266a9d23f78afb1612f83.

Full report and diagnostics

102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in comparison.json, including per-runner environments, resource limits, measurement methods, samples and checkpoints.

Comment thread posthog/ai/langchain/callbacks.py Resolved
DanielVisca and others added 2 commits November 19, 2026 19:15
…data positional args

- Redact structured tool error content before serializing $ai_error.
- Close processor.query() when the public query() stream closes, so pending tool spans are sent.
- Make SpanMetadata.span_type keyword-only so GenerationMetadata positional args keep their order.
- Regenerate the public API snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run every tool error value through finalize_ai_content, not just structured
content, so a data URL result is redacted like $ai_output_state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DanielVisca DanielVisca added the stamphog Request stamphog review label Nov 20, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved — this change needs a human reviewer.

Re-adding the stamphog label gives the same result unless the changed files change.

stamphog can't auto-review this pull request. The deny-list gate refused it because it touches references/public_api_snapshot.txt, which matches the public_api category. Because of that match, the tier gate placed the whole PR in the never-auto-approve tier (T2-never). The size gate passed, so the refusal isn't about size.

The author's note already asks the AI observability team to review and own it, so the next step is to get that human review. Any change to the public API snapshot needs a person to confirm it was intended.

Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: public_api in references/public_api_snapshot.txt
every_file_attested ✓ not a bot PR
size ✓ 395L, 5F substantive, 746L/11F incl. docs/generated/snapshots — within ceiling
tier ✗ T2-never because the deny-list matched public_api
stamphog 2.9.1 .stamphog/policy.yml @ unknown · reviewed head 8f4da4b

@stamphog stamphog Bot removed the stamphog Request stamphog review label Nov 20, 2026
…tadata field

Keeps the public SpanMetadata/GenerationMetadata API unchanged; the public API
snapshot is back to main's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DanielVisca DanielVisca added the stamphog Request stamphog review label Nov 20, 2026
@stamphog stamphog Bot removed the stamphog Request stamphog review label Nov 20, 2026
@stamphog

stamphog Bot commented Nov 20, 2026

Copy link
Copy Markdown

No review started. stamphog started a review of this pull request less than 10 minutes ago, and a label can request at most one review per 10 minutes. The stamphog label has been removed. Add it again at 03:48 UTC (in about 4 minutes) to request a fresh review.

@DanielVisca DanielVisca added the stamphog Request stamphog review label Nov 20, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

The telemetry and privacy-sensitive changes have current-head independent assurance in the trusted context. Source inspection supports the described privacy protections, and no concrete unresolved substantive concerns were supplied or identified.

  • PostHog Review (Standard) reviewed the current head: no findings.
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
every_file_attested ✓ not a bot PR
size ✓ 394L, 5F substantive, 739L/10F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1d-complex (739L, 10F, two-areas, feat)
stamphog 2.9.1 .stamphog/policy.yml @ 2846827 · reviewed head 2846827

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stamphog Request stamphog review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant