Repository navigation
refactor: streamline user cache management by consolidating permission and role cache clearing functions - #7963
Conversation
| def clear_user_permission_and_role_cache(user_ids): | ||
| """同时清理指定用户的权限缓存与角色缓存。""" | ||
| clear_user_permission_cache(user_ids) | ||
| clear_user_role_cache(user_ids) |
There was a problem hiding this comment.
Overall, the added helpers are reasonable, but there are several correctness and edge-case issues worth fixing.
Potential issues
-
One-shot iterators/generators are consumed twice
Inclear_user_permission_and_role_cache, the sameuser_idsobject is passed to two functions. If it is a generator/iterator/map/filter, the first call consumes it, so the second call receives an exhausted iterator and clears nothing.Example:
clear_user_permission_and_role_cache(iter([1, 2, 3])) # permission cache may be cleared, role cache will not be
Fix by normalizing once before calling both clear functions.
-
bytesnormalization is incorrectstr(b"123") # "b'123'"
This will not match a cache key built from
"123". If bytes are allowed, decode them explicitly. -
Empty values are not actually fully removed
The docstring says “剔除空值”, but onlyNoneis skipped. Empty string""is still converted to""and included. If empty strings are invalid user IDs, skip them explicitly. Be careful not to skip valid0IDs. -
String conversion must match the cache key builder
_normalize_user_idsconverts every ID tostr. This is only correct ifget_key()expects a string. If the cache key builder expects anint,UUID, or another representation, the generated keys will not match the keys being cached. -
Possible double-versioning
cache.delete_many(..., version=version)applies Django’s cache versioning. Make sureget_key(user_id)returns the base/unversioned key, not an already finalized key. Ifget_keyalready applies the version or callscache.make_key,version=versionwill double-apply it. -
Default cache backend only
_delete_user_cacheuses the globalcacheobject. If permission/role caches are stored in a named cache alias, e.g.caches["permission"], these clear functions will delete from the wrong backend. -
Invalidation order may cause stale permission cache repopulation
If permission cache is derived from role cache,clear_user_permission_and_role_cachecurrently clears permission first, then role. A concurrent request could repopulate permission cache from the still-stale role cache between those two operations. Safer order:clear_user_role_cache(user_ids) clear_user_permission_cache(user_ids)
-
Large input handling
_normalize_user_idsmaterializes the whole input into a list and does not deduplicate. For large inputs, this can create a largedelete_manycall. Consider deduplicating and optionally chunking deletions. -
No return value / observability
The clear functions returnNone. If useful, return or log the number of attempted keys, or log failures from the cache backend.
Suggested adjustment
def _normalize_user_ids(user_ids):
if isinstance(user_ids, bytes):
user_ids = [user_ids.decode("utf-8")]
elif isinstance(user_ids, str):
user_ids = [user_ids]
else:
try:
iter(user_ids)
except TypeError:
user_ids = [user_ids]
normalized = []
seen = set()
for user_id in user_ids:
if user_id is None or user_id == "":
continue
if isinstance(user_id, bytes):
user_id = user_id.decode("utf-8")
key = str(user_id)
if key not in seen:
seen.add(key)
normalized.append(key)
return normalized
def clear_user_permission_and_role_cache(user_ids):
user_ids = _normalize_user_ids(user_ids)
# Clear role first if permission cache is derived from role cache.
clear_user_role_cache(user_ids)
clear_user_permission_cache(user_ids)Also add tests for: None, empty list, single string, single integer, generator, duplicate IDs, empty string, bytes, and 0 if 0 is a valid ID.
| key = Cache_Version.PERMISSION_LIST.get_key(user_id=user_id) | ||
| cache.delete(key, version=version) | ||
| clear_user_permission_cache(users_id) | ||
| return instance |
There was a problem hiding this comment.
Code Review
Summary
The diff replaces inline cache-invalidation logic with a helper clear_user_permission_cache([user_id]). This is a reasonable refactor, but there are several issues worth verifying.
Potential Issues
-
Behavioral change in
auth_resource_batch(first hunk)- The removed code pasted the same
user_idfor every resource (a pre-existing bug), but the new helper callclear_user_permission_cache([user_id])also only clears one user. Confirm that all affected users get their cache invalidated, not just one. If the batch operation affects multiple users, this will leave stale caches.
- The removed code pasted the same
-
Empty-list handling
- In the last hunk,
users_idmay be empty or falsy. Verify thatclear_user_permission_cache([])is safe and does not raise / does not issue a pointless round-trip to the cache backend. If it unconditionally loops over an empty list, it's a no-op — acceptable — but consider guarding (if users_id:) if the helper has side effects.
- In the last hunk,
-
Consistency of import
Cache_Versionanddjango.core.cacheimports are removed. Ensure no other references tocacheorCache_Versionremain in this module (would causeNameErrorat runtime). Grep before merging.
-
Silent failure semantics
- Previously the delete was explicit; now it depends on the helper's error handling. Confirm the helper doesn't swallow exceptions silently or that swallowing is intended (stale permissions = security risk).
-
Duplicate detection in
auth_resource_batch- Not part of this diff, but note: the batch method still iterates using a single
user_id, which combined with the cache-clearing change suggests the method may implicitly assume one user. Worth double-checking scope.
- Not part of this diff, but note: the batch method still iterates using a single
Optimization Suggestions
- Deduplicate
users_idbefore passing to the helper:clear_user_permission_cache(set(users_id))— avoids redundant cache deletes when the same user appears multiple times. - Use delete_many / pipeline inside
clear_user_permission_cachefor bulk invalidation rather than per-key deletes, if the cache backend supports it (e.g. Redisdelete(*keys)). - Add a docstring/test for
clear_user_permission_cachenoting that it accepts a list of user IDs, since callers pass both[user_id]andusers_id.
Minor Nit
clear_user_permission_cache([user_id])— the list wrapper is fine, but if the helper clearly expects an iterable, keep it consistent across all call sites (it already is).
| for user_id in member_user_ids: | ||
| cache.delete(Cache_Version.PERMISSION_LIST.get_key(user_id=user_id), version=version) | ||
| clear_user_permission_cache(member_user_ids) | ||
| return instance |
There was a problem hiding this comment.
Here's a review of the diff:
Observations
1. clear_user_permission_cache likely expects a list, but receives a QuerySet or set
- In the first hunk,
member_user_idsis a QuerySet (.values_list(...)withoutlist()). - In the second hunk,
member_user_idsis now aset. - If the helper internally does something like
",".join(member_user_ids)or indexes into it, asetwill fail (not subscriptable) or produce nondeterministic ordering. Verify the helper's signature; pass alist(...)explicitly for safety:clear_user_permission_cache(list(member_user_ids))
2. Behavioral change in caching semantics — cache.delete(..., version=...) vs helper
- The original code deleted keys using an explicit
version=Cache_Version.PERMISSION_LIST.get_version(). Ensure the new helper uses the exact same key derivation and version logic. Ifclear_user_permission_cachedoesn't replicate the version handling, permission cache invalidation silently breaks (a serious correctness bug).
3. QuerySet evaluation order / DB trips
- Previously,
.values_list("user_id", flat=True)was iterated lazily but only once per group. - Now in the second hunk,
member_user_ids.update(queryset)forces iteration — fine, but note that eachSystemUserGroupRelation.filter(...)still triggers a separate query peruser_group_id. Consider collapsing:This reduces N queries to one — a real optimization.member_user_ids = set( QuerySet(SystemUserGroupRelation) .filter(group_id__in=user_group_ids) .values_list("user_id", flat=True) )
4. Same optimization applies to the first hunk
member_user_ids = list(
SystemUserGroupRelation.objects.filter(group_id=user_group_id)
.values_list("user_id", flat=True)
)and possibly guard on empty (if user_group_id is None: ...). At minimum, skip calling the helper when the sequence is empty to avoid needless work.
5. First hunk: QuerySet(...).bulk_create(...) if len(save_list) > 0 else None — pre-existing issue
len(save_list) > 0→ preferif save_list:. AlsoQuerySet(...)wrapping looks unusual; typicallyModel.objects.bulk_create(...). Not part of the diff, but a latent irregularity worth flagging.
6. Unused/removed imports
- Removing
cacheandCache_Versionfrom this module is fine only if they are not used elsewhere in the file (the diff shows they were removed at the top-level import site but used further down at line ~460 region and possibly other spots). Grep the file forcache.andCache_Version.before merging — otherwise you'll getNameError.
7. Type consistency
- First hunk passes a QuerySet to
clear_user_permission_cache; second hunk passes a set. For readability and to guarantee the helper contract, normalize both tolist(or whatever the helper documents).
8. No error handling around cache clearing
- If
clear_user_permission_cacheraises (e.g., Redis down), the transaction may roll back the DB changes or surface an unexpected exception after DB writes are already done. Consider whether cache-clear failure should be non-fatal (log-and-continue) for permission updates.
9. with_valid parameter unused
- In both
editmethods,with_validisn't referenced — pre-existing, but worth removing or documenting.
Suggested patch refinement
# Hunk 1
member_user_ids = list(
SystemUserGroupRelation.objects
.filter(group_id=user_group_id)
.values_list("user_id", flat=True)
)
if member_user_ids:
clear_user_permission_cache(member_user_ids)
# Hunk 2
member_user_ids = list(
SystemUserGroupRelation.objects
.filter(group_id__in=user_group_ids)
.values_list("user_id", flat=True)
.distinct()
)
if member_user_ids:
clear_user_permission_cache(member_user_ids)Summary
- Correctness risk (highest): helper must replicate original version/key semantics; verify it handles a QuerySet/set gracefully.
- Performance win available: collapse per-group queries into a single
group_id__in=...query withdistinct(). - Style: pass
list(...), guard against empty sequences, confirm removed imports aren't referenced elsewhere.
…n and role cache clearing functions
0a7fc1b to
d6e32dc
Compare
| def clear_user_permission_and_role_cache(user_ids): | ||
| """同时清理指定用户的权限缓存与角色缓存。""" | ||
| clear_user_permission_cache(user_ids) | ||
| clear_user_role_cache(user_ids) |
There was a problem hiding this comment.
Here are my observations on the diff:
Potential Issues
-
cache.delete_many(..., version=version)— version parameter support: Django'scache.delete_many()does not accept aversionkeyword argument (onlyget,set,add,get_or_set,incr,decr, anddelete/set_many-style methods handle versions inconsistently;delete_manyin the default backend ignores/has noversion). This will likely raiseTypeError: delete_many() got an unexpected keyword argument 'version'. If your custom cache backend supports it, fine — but this is fragile and worth verifying. The version is normally baked into the key viaget_key, so passingversionhere may be redundant or incorrect. -
Assumption about
cache_item.valueshape:_delete_user_cacheassumesCache_Version.PERMISSION_LIST.valueis a(version, get_key)tuple. This coupling is implicit. If.valuereturns something else (e.g., a dataclass/enum with named fields), this breaks silently or loudly. Consider unpacking defensively or documenting the contract. -
_normalize_user_idssemantics forbytes: Treatingbyteslikestrwraps it as a single item, thenstr(b'abc')yields"b'abc'", which is almost certainly not the intended key. Clarify whether bytes are a legitimate input; if not, don't special-case them like strings. -
clear_user_permission_and_role_cache— partial failure: Ifclear_user_permission_cachesucceeds butclear_user_role_cacheraises, you get inconsistent cache state. Not necessarily a bug, but callers should be aware there's no atomicity. -
No
keys=batching / large iterables:delete_manywith a very largeuser_idslist can build a huge key list in memory. For bulk imports/migrations, consider chunking. -
Double normalization cost: Each public function calls
_normalize_user_ids, andclear_user_permission_and_role_cachecalls both helpers—so normalization happens twice on the same input. Minor, but could normalize once and pass lists down.
Optimization / Style Suggestions
-
Simplify
_normalize_user_idsiterator handling:def _normalize_user_ids(user_ids): if user_ids is None: return [] if isinstance(user_ids, (str, bytes)): user_ids = [user_ids] try: iter(user_ids) except TypeError: user_ids = [user_ids] return [str(uid) for uid in user_ids if uid is not None]
Note using a list comprehension also slightly reduces overhead vs. manual
append. -
Deduplicate ids:
setconversion before building keys avoids redundant work ondelete_many:unique = {str(uid) for uid in ...}. Worth it for large inputs. -
Comment language consistency: The docstrings use Chinese (
将用户ID...) while the module header/comments elsewhere are English. Align with project conventions for maintainability. -
Delete-on-empty short-circuit is good — keep it. Consider returning the count from
delete_manyfor observability/tests. -
Naming:
get_dataparameter (in existing code above the diff) shadows nothing problematic, but_delete_user_cachemight better be named_delete_user_cachessince it handles multiple users/keys. -
Type hints: Adding
Iterable[Union[str, int]] -> List[str]annotations to the new helpers would improve clarity, consistent with the existingkey: strannotation style.
Summary of highest-priority items
- Verify
cache.delete_many()actually acceptsversion=in your backend (item 1). - Confirm
Cache_Version.*.value == (version, get_key)contract (item 2). - Reconsider
byteshandling in_normalize_user_ids(item 3).
| for user_id in member_user_ids: | ||
| cache.delete(Cache_Version.PERMISSION_LIST.get_key(user_id=user_id), version=version) | ||
| clear_user_permission_cache(member_user_ids) | ||
| return instance |
There was a problem hiding this comment.
The changes are largely sound (moving cache invalidation into a helper, and collecting user IDs before clearing). A few observations:
Potential issues:
-
member_user_idstype inconsistency in the first hunk. Inedit()(around line 253),clear_user_permission_cache(member_user_ids)receives a lazyValuesListQuerySet. If the helper iterates it once, that's fine; but if it needs to iterate multiple times orlen(), a queryset is still re-evaluated per iteration. Consider wrapping inset(...)for consistency with the second hunk:member_user_ids = set( QuerySet(SystemUserGroupRelation).filter(group_id=user_group_id).values_list("user_id", flat=True) )
-
Behavioral difference of dedup / ordering. The old loop iterated per-group and deleted per-user immediately, so duplicate user_ids across groups could cause redundant deletes but were harmless. The new
set()approach is better (dedups, single batched call), but ensureclear_user_permission_cacheaccepts an iterable/set and not a queryset specifically. -
Transaction/consistency. If
bulk_createand cache clearing aren't inside the same transaction, a failure between them can leave stale cache. Confirm the caller wraps this intransaction.atomic()(or that stale-cache-on-failure is acceptable). -
save_list/update_listunused variables? In the shown snippet,update_listis built but never used in the visible code after theif len(update_list) > 0 else Noneexpression — verify the ternary isn't dead code (the result isn't assigned anywhere in either hunk shown). -
Batch-size for
bulk_create. For large rule sets, passbatch_size=tobulk_createto avoid one giant INSERT and DB parameter limits. -
Empty-set safety.
clear_user_permission_cache(set())should be a no-op; confirm the helper early-returns on empty input to avoid unnecessary round-trips (e.g., adelete_manywith empty keys).
Style/nits:
QuerySet(Model).filter(...)instead ofModel.objects.filter(...)works but is unconventional;Model.objects.filteris more idiomatic.- The
... if len(x) > 0 else Noneexpression statements are confusing; prefer explicitif save_list: bulk_create(...). - Confirm the import switch (
Cache_Version/cacheremoved) doesn't break other usages in the file — grep before merging.
Optimization: skimming two hunks only; consider also invalidating when members are removed from a group (not shown), otherwise stale permission cache may persist.
| clear_user_permission_cache([user_id]) | ||
|
|
||
| return None | ||
|
|
There was a problem hiding this comment.
Here's my review:
Potential Issues
-
clear_user_permission_and_role_cache([user.id])called after earlyreturnpath is inconsistent — In the first hunk (line ~827), the cache clear is only invoked on the "create default mapping" branch, immediately beforereturn. This means other branches in the same function that also mutate user-role mappings may not invalidate the cache. Verify the function has a single consistent invalidation point, otherwise stale permissions persist. -
Loss of transactional safety / cache-vs-DB ordering — The old code deleted cache keys after DB writes in
update_user_role, and the new call preserves that order. However, if the surrounding code is inside a transaction (transaction.atomic) and later rolls back, the cache will already have been cleared (harmless but wasteful) — conversely if cache clearing raises, the DB write commits without invalidation. Consider clearing after commit (transaction.on_commit(...)). -
Single-element list argument — Passing
[user.id]/[user_id]to batch-style helpers suggests the helper is designed for multi-user invalidation. Double-check the helper signature acceptsList[str|UUID]and thatuser.idtype matches (str vs. UUID). A mismatch could silently produce no-op deletes. -
Parameter shadowing in
set_user_groups— The new code usesclear_user_permission_cache([user_id]), while the old code usedCache_Version.PERMISSION_LIST.get_key(user_id=user_id). Confirm the helper internally applies the same key-building logic (including version prefix); otherwise the cache won't actually be cleared. -
Behavior change: version bump removal — Old code did
cache.delete(..., version=<current_version>). If the helper instead bumps the version or clears with a different version source, semantics differ subtly. Confirm the helper reads the sameCache_Versionand doesn't skip versioned namespace handling. -
Local re-import inside function removed — The diff removes inline
Cache_Versionusage; ensureCache_Versionimport is now unused elsewhere in this file (lint/dead-import), or still needed (if so, keep it). -
Error handling — The original inline deletes would surface any exceptions from
cache.delete. Delegating toclear_user_permission_*_cachecentralizes logic (good), but verify the helper doesn't swallow exceptions quietly — a silent failure leaves users with stale permissions. -
The
returnplacement in the first hunk — Placing the cache clear immediately beforereturncouples it specifically to that branch. If additional branches return early elsewhere inupdate_user_role, they must also invalidate. Prefer atry/finallyor post-processing hook.
Optimization Suggestions
- Consider replacing per-branch invalidation with a single invalidation at the end of
update_user_role, or use a signal/on_commitpattern so all mutation paths are covered. - If
clear_user_permission_and_role_cacheandclear_user_permission_cachedo simpledeletes, ensure they use batch/pipeline operations when passed multiple IDs to avoid N round-trips. - Add unit tests asserting that cache keys are invalidated on all role-mutation branches (including early returns).
- Verify import ordering (alphabetical):
common.utils.cache_utilshould sit betweencommon.utils.commonandcommon.utils.rsa_util— currently it does, good.
Summary
The refactor improves consistency by consolidating cache-invalidation logic, which is a good direction. The main risks are branch coverage (not all early returns invalidate), type/signature assumptions on the helper, and transaction/cache-ordering semantics. Validate these before merging.
refactor: streamline user cache management by consolidating permission and role cache clearing functions