Repository navigation
Conversation
Each cached entry used four heap objects: the promise, its done channel, a list.Element and the key boxed into it, plus a sync.Map node. Entries are now one object linked into an intrusive LRU list, kept in a plain map under the cache mutex that every hit already takes. The done channel and error live in an inflight call that is dropped once the fetch succeeds. This also fixes byte accounting drift. Evicting the front of the list called LoadAndDelete on the key, which could delete a newer promise that had replaced an expired one mid-fetch, leaving the old bytes counted. Eviction now removes exactly the entry it unlinks. Failed fetches are no longer cached until the TTL expires, and a caller waiting on another caller's fetch now returns when its own context is cancelled. Signed-off-by: Alan Protasio <alanprot@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>
A refresh used to skip eviction, so a larger refreshed value could leave cachedBytes over max_bytes until the next fresh miss or periodic purge. Run eviction after every completed fetch, and update the TTL test, whose first refresh now also sweeps the other expired entries. Signed-off-by: Alan Protasio <alanprot@gmail.com>
The lint target's faillint check blocks sync/atomic in favor of go.uber.org/atomic, which provides the same generic Pointer type. Signed-off-by: Alan Protasio <alanprot@gmail.com>
A panicking fetch left its entry in flight with its done channel never closed. Callers joined to it blocked until their own deadline, and every later caller for that key joined the dead fetch, so the key stayed unusable until the entry was evicted for space. QueryStream recovers runtime panics, so the ingester keeps running with the stuck key. The fetch now runs under a deferred cleanup that removes the entry if it is still current, gives waiters errFetchPanicked and closes done before the panic continues. The next caller fetches again. Signed-off-by: Alan Protasio <alanprot@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does:
Reduces the memory the ingester's expanded postings cache uses per entry, and fixes a byte-accounting drift in its LRU.
Each cached entry used to be four heap objects plus a
sync.Mapnode: thecacheEntryPromise, itsdonechannel, alist.Element, and the key boxed into that element. For small entries the real heap cost was about 3x the bytes the cache accounts for, somax_bytesunderstated memory badly.Changes:
cacheEntryPromisenow carries the key and intrusiveprev/nextpointers, so there is nolist.Elementand no boxed key.cachedMtx.Lock()to move the entry to the back of the list, sosync.Mapbought no concurrency. Map and list now change together under one lock, and an entry is in the map if and only if it is linked in the list.donechannel and the error move into a smallinflightvalue. It is dropped when the fetch succeeds, so completed entries hold no channel. All concurrent callers for a key wait on the sameinflight.getPromiseForKey. It waits inresult()and returnsctx.Err()if its own context is cancelled, while the fetch continues for other callers.errFetchPanicked, and lets the panic continue, so the next caller fetches again.int64nanoseconds instead oftime.Time.Per-entry memory, excluding the key bytes and the series IDs (both unchanged). Heap bytes per entry measured with 200k entries on amd64, Go 1.27:
cacheEntryPromise(88 B struct before, 80 B after)donechannel (hchan)list.Elementplus the key boxed intoanyprev/next)sync.Mapnode before, plainmapslot afterWith a ~190-byte key and one series ID (192 B + 8 B allocated) this adds up to the ~600 B and ~320 B per entry in the benchmark below. The
inflightvalue (24 B plus its channel) exists only while a fetch is running and is then garbage.Accounting drift fixed. In the expired-refresh path, a goroutine swapped a new promise in for an expired one and started fetching. If
expire()then popped the old promise's list element,LoadAndDelete(key)removed the new promise, whose size was still 0, so the old promise's bytes stayed incachedBytespermanently. Orphan and duplicate list elements followed. Over time the cache counted bytes for entries it no longer held, which shrinks its effective capacity.-racealso reported unsynchronized writes totsandsizeBytes. Eviction now removes exactly the entry it unlinks, and a completed fetch only adds its bytes if its entry is still the one in the map.Behaviour kept as before: TTL measured from creation time,
fullandexpiredeviction reasons and metrics, andTtl <= 0disabling retention.Behaviour change: eviction now runs after every completed fetch, including the refresh of an expired key. Previously a refresh skipped eviction, so a larger refreshed value could push
cachedBytesovermax_bytesuntil the next fresh miss or the periodic purge, and expired neighbours stayed until then.TestFifoCacheExpire/TTLasserted the old behaviour and is updated: the first refresh now also sweeps the other expired entries.Benchmarks (
-tags netgo,slicelabels, 48 cores)Heap per entry, 100k entries with ~190-byte keys (
BenchmarkLruCache_BytesPerEntry, new):Existing benchmarks,
benchstatover 5 runs:LruCacheHitUnderPressuresec/opCacheGetPromisesec/opLruCacheHitUnderPressureallocs/opCacheGetPromiseallocs/opCacheGetPromiseB/opTests
TestLruCache_ConsistentUnderConcurrency: 64 goroutines, a clock advancing past the TTL while fetches are in flight, and a backgroundexpire(). Checks thatcachedBytesequals the sum of entry sizes, that map and list sizes match, and that list links are consistent. The equivalent check against the current implementation fails in the expiring cases.TestLruCache_DoesNotCacheErrorsTestPostingsCache_WaiterHonorsContextWhileFetchRunsTestLruCache_FetchPanicDoesNotPoisonKeypkg/storage/tsdbtests pass, including under-race, and the ingester postings cache tests pass.Not in this PR: the accounting still charges only
len(key) + 8*len(ids). The remaining real cost is about 1.7x the accounted bytes for 1-ID entries. Charging a fixed per-entry overhead would makemax_bytesreflect real heap; that changes how much a givenmax_bytesholds, so I'd rather do it separately.Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]docs/configuration/v1-guarantees.mdupdated if this PR introduces experimental flags