Repository navigation
linux: do the hash multiplies in unsigned - #397
Merged
Merged
Conversation
hja-oss
reviewed
Nov 16, 2026
Comment on lines
54
to
58
| #define SFHASHDEVINO(maj, min, ino, mod) \ | ||
| ((int)(((int)((((int)(maj + 1)) * ((int)((min + 1)))) + ino) * 31415) & \ | ||
| ((int)(((int)((((unsigned int)(maj + 1)) * ((unsigned int)((min + 1)))) + \ | ||
| ino) * \ | ||
| 31415u) & \ | ||
| (mod - 1))) |
Contributor
There was a problem hiding this comment.
Could we keep the calculation unsigned until after masking?
Suggested change
| #define SFHASHDEVINO(maj, min, ino, mod) \ | |
| ((int)(((int)((((int)(maj + 1)) * ((int)((min + 1)))) + ino) * 31415) & \ | |
| ((int)(((int)((((unsigned int)(maj + 1)) * ((unsigned int)((min + 1)))) + \ | |
| ino) * \ | |
| 31415u) & \ | |
| (mod - 1))) | |
| #define SFHASHDEVINO(maj, min, ino, mod) \ | |
| ((int)(((((unsigned int)(maj) + 1u) * \ | |
| ((unsigned int)(min) + 1u) + (unsigned int)(ino)) * \ | |
| 31415u) & ((mod) - 1))) |
The same could apply to SFHASHRDEVI below.
Contributor
Author
There was a problem hiding this comment.
good call, applied it to both SFHASHDEVINO and SFHASHRDEVI so its unsigned all the way until the mask now. buckets come out the same, ubsan build is still clean
HASHPID, HASHPINFO, SFHASHDEVINO, SFHASHRDEVI and the IPv6 IPC hash multiplied in int, so any PID above 68356, most inode and device numbers, and most IPv6 addresses overflowed a signed int, which is undefined behaviour. The bucket mask hides it on usual builds, but UBSan reports it on every run and lsof's own tests hit it hundreds of times. Do the multiplies in unsigned int instead. The low bits are the same as before, so every value still lands in the same bucket.
Ar-maan05
force-pushed
the
linux-hash-unsigned
branch
from
November 17, 2026 02:05
1bcd0be to
ac3630b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The linux hash macros multiply by 31415 in
int, so they overflow for basically any real pid, inode or ipv6 address. On a UBSan build plainlsof -n -P -Eon master prints:the bucket mask hides it on normal builds so nothing visibly breaks, its just UB. This makes HASHPID, HASHPINFO, SFHASHDEVINO/SFHASHRDEVI and the IPv6 IPC hash multiply in unsigned. Low bits are the same so every bucket stays the same.
Tested: clang -fsanitize=undefined build,
lsof -n -P -Eand lsof on a few file names are clean now (7 overflow sites before),make checkpasses (39 pass, 3 skip).