Skip to content

fix(capture): keep $release_id on a minimized $feature_flag_called event - #1276

Draft
posthog[bot] wants to merge 1 commit into
mainfrom
posthog/release-id-on-minimized-flag-called
Draft

posthog[bot] wants to merge 1 commit into
mainfrom
posthog/release-id-on-minimized-flag-called

Conversation

@posthog

@posthog posthog Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Brings posthog-python's Capture contract into compliance with the capture spec's "Environment-sourced release identifier" requirement in sdk-specs:

$release_id is a static, low-cardinality runtime identity property in the sense of the feature-flag-called-tracker minimal-event allowlist, so a minimized $feature_flag_called event SHALL retain it when the SDK stamped it.

The spec's @server scenario "A minimized flag-called event keeps the release id" was failing. The compliance matrix note for this row records it as the single remaining gap on the contract:

The one gap is the minimized flag-called event: $release_id is not in _MINIMAL_FLAG_CALLED_EVENT_PROPERTIES, and the allowlist filter runs after the release-id stamp, so a minimized $feature_flag_called strips the property the capture spec says it SHALL retain.

Remediation: add "$release_id" to _MINIMAL_FLAG_CALLED_EVENT_PROPERTIES (it is a static, low-cardinality runtime identity property, exactly the category the allowlist already keeps $lib/$os for).

In practice: an app that sets POSTHOG_RELEASE_ID gets $release_id on every event except its minimized flag-called events, so flag calls can't be broken down or filtered by release.

🔨 What changed

  • $release_id is added to _MINIMAL_FLAG_CALLED_EVENT_PROPERTIES in posthog/client.py, alongside the $lib/$os/$python_* runtime identity properties already kept. The v2 allowlist derives from this set, so it is covered too.
  • AsyncPosthog already retained it (it applies the allowlist before filling SDK defaults), so no change was needed there.

Nothing else about the minimal shape changes: an unset or blank POSTHOG_RELEASE_ID still sends no $release_id at all, and every other non-allowlisted property is still stripped.

Backwards compatibility

Backward-compatible and additive. Minimized $feature_flag_called events gain one static, low-cardinality property that the same client already sends on every other event; no public API, signature, or option changes, and no property is removed. Clients that never set POSTHOG_RELEASE_ID see no change at all.

💚 How did you test it?

  • Updated test_release_id.py: its test_minimal_flag_called_events_keep_their_strict_allowlist asserted the pre-spec stripping behavior, so it now asserts that a minimized event keeps $release_id while still dropping non-allowlisted properties, plus a new case covering an unset release id.
  • Added test_gated_non_experiment_flag_keeps_the_release_id to test_feature_flag_called_minimization.py, which drives the real get_feature_flag_result path under the server-controlled minimization gate and asserts both the value and that the event stays within the allowlist.
  • ruff format --check ., ruff check ., mypy on the changed module, and the full pytest suite. The only failures are 10 pre-existing AI-integration tests (posthog/test/ai/langchain, posthog/test/ai/openai) that need provider credentials/network in this sandbox; they are unrelated to this change.

➡️ Follow-up work

Out of scope here, but noted while verifying: AsyncPosthog._prepare_event applies the minimal allowlist before _fill_event_defaults, while Client._enqueue applies it after every enrichment step. Both satisfy this requirement today, but the orderings differ from each other and from the spec's "apply the allowlist after enrichment, before before_send" ordering, so the async client's minimal events can retain SDK-filled properties the sync client drops. Worth a separate look.

📝 Checklist

  • All commits are signed
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Changeset added (.sampo/changesets/release-id-on-minimized-flag-called.md)

🤖 Agent context

Autonomy: Fully autonomous

Opened by a scheduled agent run that reads the SDK compliance matrices in PostHog/sdk-specs and implements one open gap per run. This gap was picked over the other open posthog-python items because it is on a core, widely-used contract (Capture), its remediation is a one-line, strictly additive allowlist change with a matching spec scenario, and it had no in-flight PR or branch. Runner-up candidates considered and deferred: a public get_distinct_id() (larger public API surface), before_send hook chaining (spec frames it as optional), and HTTP 413 batch shrink-and-retry (also optional in the spec). The Flush retry gap was excluded outright — its compliance verdict is Breaking.

Tools used: shell (git, ripgrep, pytest, ruff, mypy) and the GitHub CLI. No human directed this change, so the PR is left unassigned for the owning team to triage, and it needs human review before merge.


Created with PostHog Desktop

The capture spec's environment-sourced release identifier requirement says
$release_id is a static, low-cardinality runtime identity property in the
sense of the minimal flag-called allowlist, so a minimized
$feature_flag_called event retains it when the SDK stamped it.

The SDK stamped it in _enqueue and then stripped it, because the allowlist
filter runs after the stamp and did not list the key. Add it next to the
other runtime identity properties the allowlist already keeps.

Generated-By: PostHog Desktop
Task-Id: 700ce639-9125-4a88-a919-8a24ad6edf12
@github-actions

Copy link
Copy Markdown
Contributor

SDK performance: posthog-python

Benchmarks completed. No changes flagged for review.

Flags require at least a 5% change in the median and all six pairs agreeing on direction. This is a noise filter, not proof of a performance change or a CI gate.

View all results
Test Measurement Before PR With PR Change Assessment
Capture and deliver events (32-byte payload, burst)
Shard: capture
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Capture and deliver events (32-byte payload, steady)
Shard: capture
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Queue an event (32-byte payload)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​; head:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​
Queue an event (32-byte payload)
Shard: capture
Allocations per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​; head:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​
Queue an event (4,096-byte payload)
Shard: capture
Allocated memory per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​; head:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​
Queue an event (4,096-byte payload)
Shard: capture
Allocations per operation — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​; head:​ Background preparation has no public completion boundary before flush; see capture.​drain for full-pipeline allocations.​
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Refresh 1,000 flag definitions
Shard: flags-memory
Managed heap after settling — — — unavailable; base:​ unsupported; head:​ unsupported; base:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​; head:​ CPython has no uninstrumented total managed-heap byte counter; tracing would distort this RSS pass.​
Capture and deliver events (32-byte payload, burst)
Shard: capture
Allocated memory per operation 133.67 KiB/op 133.67 KiB/op <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Allocations per operation 1,345.51 count/op 1,345.51 count/op <0.1% more Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Time per operation 77.19 µs/op 77.23 µs/op <0.1% more time Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Idle process memory (RSS) 37.59 MiB 37.62 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Observed peak process memory (RSS) 40.04 MiB 40.07 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Process memory after settling (RSS) 40.04 MiB 40.07 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Process CPU time per operation 76.86 µs/op 79.65 µs/op 3.6% more time Not flagged
Capture and deliver events (32-byte payload, burst)
Shard: capture
Lifetime peak process memory (RSS) 40.37 MiB 40.37 MiB No observed change Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Allocated memory per operation 185.75 KiB/op 185.75 KiB/op <0.1% less memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Allocations per operation 1,337.52 count/op 1,337.51 count/op <0.1% fewer Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Time per operation 100.51 µs/op 100.92 µs/op 0.4% more time Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Idle process memory (RSS) 41.13 MiB 41.25 MiB 0.3% more memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Observed peak process memory (RSS) 46.92 MiB 47.04 MiB 0.3% more memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Process memory after settling (RSS) 46.92 MiB 47.04 MiB 0.3% more memory Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Process CPU time per operation 98.28 µs/op 98.44 µs/op 0.2% more time Not flagged
Capture and deliver events (4,096-byte payload, burst)
Shard: capture
Lifetime peak process memory (RSS) 51.32 MiB 51.4 MiB 0.2% more memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Allocated memory per operation 138.73 KiB/op 138.73 KiB/op <0.1% less memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Allocations per operation 1,390.3 count/op 1,390.3 count/op <0.1% fewer Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Time per operation 88.28 µs/op 88.6 µs/op 0.4% more time Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Idle process memory (RSS) 37.6 MiB 37.61 MiB <0.1% more memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Observed peak process memory (RSS) 37.77 MiB 37.76 MiB <0.1% less memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Process memory after settling (RSS) 37.77 MiB 37.76 MiB <0.1% less memory Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Process CPU time per operation 90.24 µs/op 90.82 µs/op 0.6% more time Not flagged
Capture and deliver events (32-byte payload, steady)
Shard: capture
Lifetime peak process memory (RSS) 40.37 MiB 40.37 MiB No observed change Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Allocated memory per operation 190.9 KiB/op 190.9 KiB/op <0.1% more memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Allocations per operation 1,382.19 count/op 1,382.19 count/op <0.1% more Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Time per operation 106.39 µs/op 105.6 µs/op 0.7% less time Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Idle process memory (RSS) 41.11 MiB 40.98 MiB 0.3% less memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Observed peak process memory (RSS) 41.58 MiB 41.43 MiB 0.4% less memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Process memory after settling (RSS) 41.17 MiB 41.03 MiB 0.3% less memory Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Process CPU time per operation 102.31 µs/op 102.93 µs/op 0.6% more time Not flagged
Capture and deliver events (4,096-byte payload, steady)
Shard: capture
Lifetime peak process memory (RSS) 41.51 MiB 41.38 MiB 0.3% less memory Not flagged
Queue an event (32-byte payload)
Shard: capture
Caller-thread CPU time per operation 57.41 µs/op 57.27 µs/op 0.2% less time Not flagged
Queue an event (32-byte payload)
Shard: capture
Time per operation 57.4 µs/op 57.31 µs/op 0.2% less time Not flagged
Queue an event (4,096-byte payload)
Shard: capture
Caller-thread CPU time per operation 57.37 µs/op 57.73 µs/op 0.6% more time Not flagged
Queue an event (4,096-byte payload)
Shard: capture
Time per operation 57.36 µs/op 57.73 µs/op 0.6% more time Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Allocated memory per operation 225.43 KiB/op 225.43 KiB/op No observed change Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Allocations per operation 2,279.42 count/op 2,279.42 count/op No observed change Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Time per operation 82.68 µs/op 82.7 µs/op <0.1% more time Not flagged
Evaluate all 10 flags
Shard: flags-scaling
Process CPU time per operation 82.68 µs/op 82.7 µs/op <0.1% more time Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Allocated memory per operation 2.17 MiB/op 2.17 MiB/op No observed change Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Allocations per operation 22,413.69 count/op 22,413.69 count/op No observed change Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Time per operation 775.88 µs/op 777.62 µs/op 0.2% more time Not flagged
Evaluate all 100 flags
Shard: flags-scaling
Process CPU time per operation 775.9 µs/op 777.64 µs/op 0.2% more time Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Allocated memory per operation 21.63 MiB/op 21.63 MiB/op No observed change Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Allocations per operation 223,655.44 count/op 223,655.44 count/op No observed change Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Time per operation 7.73 ms/op 7.72 ms/op <0.1% less time Not flagged
Evaluate all 1,000 flags
Shard: flags-scaling
Process CPU time per operation 7.73 ms/op 7.72 ms/op <0.1% less time Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Idle process memory (RSS) 43.81 MiB 43.85 MiB <0.1% more memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Observed peak process memory (RSS) 44.13 MiB 44.15 MiB <0.1% more memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Process memory after settling (RSS) 44.13 MiB 44.15 MiB <0.1% more memory Not flagged
Evaluate 1,000 flags under concurrent load
Shard: flags-memory
Lifetime peak process memory (RSS) 43.85 MiB 43.9 MiB 0.1% more memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Idle process memory (RSS) 42.96 MiB 42.99 MiB <0.1% more memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Observed peak process memory (RSS) 42.97 MiB 43 MiB <0.1% more memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Process memory after settling (RSS) 42.97 MiB 43 MiB <0.1% more memory Not flagged
Refresh 1,000 flag definitions
Shard: flags-memory
Lifetime peak process memory (RSS) 42.83 MiB 42.74 MiB 0.2% less memory Not flagged
Evaluate one boolean flag
Shard: flags-single
Allocated memory per operation 15.74 KiB/op 15.74 KiB/op No observed change Not flagged
Evaluate one boolean flag
Shard: flags-single
Allocations per operation 162.77 count/op 162.77 count/op No observed change Not flagged
Evaluate one boolean flag
Shard: flags-single
Time per operation 8.29 µs/op 8.17 µs/op 1.4% less time Not flagged
Evaluate one boolean flag
Shard: flags-single
Process CPU time per operation 8.29 µs/op 8.17 µs/op 1.4% less time Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Allocated memory per operation 24.6 KiB/op 24.6 KiB/op No observed change Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Allocations per operation 239.28 count/op 239.28 count/op No observed change Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Time per operation 11.65 µs/op 11.58 µs/op 0.6% less time Not flagged
Evaluate a cohort-filtered flag
Shard: flags-single
Process CPU time per operation 11.65 µs/op 11.58 µs/op 0.6% less time Not flagged
Evaluate a flag (early match)
Shard: flags-single
Allocated memory per operation 39.81 KiB/op 39.81 KiB/op No observed change Not flagged
Evaluate a flag (early match)
Shard: flags-single
Allocations per operation 388.27 count/op 388.27 count/op No observed change Not flagged
Evaluate a flag (early match)
Shard: flags-single
Time per operation 16.57 µs/op 16.37 µs/op 1.2% less time Not flagged
Evaluate a flag (early match)
Shard: flags-single
Process CPU time per operation 16.57 µs/op 16.37 µs/op 1.2% less time Not flagged
Evaluate a flag (late match)
Shard: flags-single
Allocated memory per operation 60.59 KiB/op 60.59 KiB/op No observed change Not flagged
Evaluate a flag (late match)
Shard: flags-single
Allocations per operation 584.27 count/op 584.27 count/op No observed change Not flagged
Evaluate a flag (late match)
Shard: flags-single
Time per operation 23.82 µs/op 23.58 µs/op 1% less time Not flagged
Evaluate a flag (late match)
Shard: flags-single
Process CPU time per operation 23.82 µs/op 23.58 µs/op 1% less time Not flagged
Evaluate a flag (no match)
Shard: flags-single
Allocated memory per operation 59.22 KiB/op 59.22 KiB/op No observed change Not flagged
Evaluate a flag (no match)
Shard: flags-single
Allocations per operation 567.77 count/op 567.77 count/op No observed change Not flagged
Evaluate a flag (no match)
Shard: flags-single
Time per operation 22.39 µs/op 22.31 µs/op 0.3% less time Not flagged
Evaluate a flag (no match)
Shard: flags-single
Process CPU time per operation 22.39 µs/op 22.31 µs/op 0.3% less time Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Allocated memory per operation 19.04 KiB/op 19.04 KiB/op No observed change Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Allocations per operation 192.27 count/op 192.27 count/op No observed change Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Time per operation 9.37 µs/op 9.34 µs/op 0.3% less time Not flagged
Evaluate a property-filtered flag
Shard: flags-single
Process CPU time per operation 9.37 µs/op 9.34 µs/op 0.3% less time Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Allocated memory per operation 20.66 KiB/op 20.66 KiB/op No observed change Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Allocations per operation 197.77 count/op 197.77 count/op No observed change Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Time per operation 9.83 µs/op 9.81 µs/op 0.2% less time Not flagged
Evaluate a regex-filtered flag
Shard: flags-single
Process CPU time per operation 9.82 µs/op 9.81 µs/op 0.2% less time Not flagged
Evaluate a rollout flag
Shard: flags-single
Allocated memory per operation 14.89 KiB/op 14.89 KiB/op No observed change Not flagged
Evaluate a rollout flag
Shard: flags-single
Allocations per operation 152.77 count/op 152.77 count/op No observed change Not flagged
Evaluate a rollout flag
Shard: flags-single
Time per operation 7.76 µs/op 7.72 µs/op 0.5% less time Not flagged
Evaluate a rollout flag
Shard: flags-single
Process CPU time per operation 7.76 µs/op 7.72 µs/op 0.5% less time Not flagged
Evaluate a multivariate flag
Shard: flags-single
Allocated memory per operation 15.9 KiB/op 15.9 KiB/op No observed change Not flagged
Evaluate a multivariate flag
Shard: flags-single
Allocations per operation 166.77 count/op 166.77 count/op No observed change Not flagged
Evaluate a multivariate flag
Shard: flags-single
Time per operation 8.5 µs/op 8.51 µs/op 0.1% more time Not flagged
Evaluate a multivariate flag
Shard: flags-single
Process CPU time per operation 8.5 µs/op 8.51 µs/op 0.1% more time Not flagged

Values are medians of paired measurements. Not flagged does not mean unchanged. Unavailable means a measurement could not be provided, not zero. Estimated allocation measurements retain their instrumentation uncertainty. RSS measures the whole process, not just SDK allocations; observed peaks are sampled checkpoints. Paired ranges in the diagnostics describe variability, not confidence intervals.

Each shard uses its own runner; compare Before PR against With PR within a row, not across shards.

Compared 7c87c8ea152845a2514dd9caf59641539ce83d6b → 6e2b8a0de5009216d988040eeca492a348d201fd.

Full report and diagnostics

102 metric rows; 102 rows shown in all results. Omitted or shortened data remain in comparison.json, including per-runner environments, resource limits, measurement methods, samples and checkpoints.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants