Skip to content

Fix public suffix matching for IDN labels unassigned in IDNA2003 - #954

Merged
arturobernalg merged 1 commit into
apache:masterfrom
arturobernalg:psl-idn-unassigned
Nov 19, 2026
Merged

arturobernalg merged 1 commit into
apache:masterfrom
arturobernalg:psl-idn-unassigned

Conversation

@arturobernalg

Copy link
Copy Markdown
Member

java.net.IDN uses Unicode 3.2 and fails to decode certain A-labels containing characters introduced in later Unicode versions.

As a result, PublicSuffixMatcher may fail to recognize valid public suffix rules, potentially allowing cookies to be shared across unrelated domains.

This change uses IDN.ALLOW_UNASSIGNED to ensure consistent IDN normalization in public suffix matching and hostname verification.

Reported by Jonas Mandalunes.

Problem:

java.net.IDN enforces the Unicode 3.2 repertoire (IDNA2003) by
default. A Public Suffix List label whose script was assigned to
Unicode after 3.2 - for example Balinese (Unicode 5.0), the script
behind the real PSL entry xn--9tfky.id - is "unassigned" under that
repertoire. IDN.toUnicode silently fails to decode such a label
instead of throwing, so DnsUtils.normalizeUnicode and
PublicSuffixMatcher.getDomainRoot never see the label in the Unicode
form the PSL rules are keyed by, and the matcher falls through to
treating it as an ordinary registrable label one level short of the
real suffix.

This is a real, exploitable defect: a cookie set with
Domain=xn--9tfky.id by one tenant under that suffix is accepted by
another, unrelated tenant under the same suffix, defeating the
Public Suffix List's whole purpose on the cookie path. The same
decode is shared with DefaultHostnameVerifier (matchDNSName/matchCN
normalize through the same DnsUtils.normalizeUnicode before calling
PublicSuffixMatcher.verifyInternal), so a non-wildcard TLS identity
for the bare suffix itself is wrongly accepted too.

Solution:

Pass IDN.ALLOW_UNASSIGNED to the IDN.toUnicode/toASCII calls in
DnsUtils.normalizeUnicode and PublicSuffixMatcher.getDomainRoot.
DnsUtils.normalizeUnicode is the fix that actually matters: it is
the normalization verify() and DefaultHostnameVerifier both go
through. getDomainRoot has no production caller in this module
today, but is public API and should stay consistent with verify()
for external callers.

A separate, pre-existing defect was found while testing this: a
wildcard SAN such as *.xn--9tfky.id (or, identically, a plain ASCII
*.id) is accepted by verifyInternal for a host anywhere under that
suffix, because the literal '*' is not stripped before the suffix
lookup. It reproduces identically with or without IDN involved, so
it is unrelated to this fix and intentionally left alone here;
resolveDomainRoot is not touched.

Tests:

- TestPublicSuffixMatcher#testUnassignedCodePoint: getDomainRoot,
  matches and verify against a crafted PSL rule whose Unicode label
  is unassigned under 3.2 (xn--9tfky.test / Balinese, added to the
  suffixlistmatcher.txt fixture), in both ACE and Unicode form, plus
  the same verify() check against the real, live Mozilla Public
  Suffix List entry (xn--9tfky.id).
- TestDefaultHostnameVerifier#testMatchDNSNameUnassignedCodePoint:
  matchDNSName, through the real normalize-then-verifyInternal call
  path (not matchIdentity directly, which does not normalize),
  rejects a non-wildcard certificate for the bare suffix, matching
  existing ASCII behavior for a control such as gov.uk.
- TestRFC6265CookieSpec#testNoArgSpecsRejectUnassignedCodePointSuffix:
  the real, no-arg RFC6265LaxSpec/RFC6265StrictSpec (which wire the
  default PublicSuffixMatcher), same pattern as the existing
  testNoArgSpecsUsePublicSuffixMatcher - a cookie for the bare
  suffix does not match a different tenant under it, while a cookie
  for a genuine registrable domain under the suffix still matches
  its siblings.

All three were run RED against unpatched master (production sources
reverted to the upstream master blob, in an otherwise identical
worktree) and GREEN with the fix, deterministically (3 repeated
runs, no sockets, no timing). Full httpclient5 module: 1169/1169.
httpclient5-testing is untouched by this revision (an earlier,
larger version of this change added integration-server tests there;
they duplicated what the CookieSpec-level test already covers and
were removed).

Compatibility:

Observable change: a non-wildcard TLS identity, or a cookie Domain,
exactly equal to a PSL suffix whose label was previously silently
left undecoded is no longer treated as an ordinary registrable
domain. This is a correction, not a new restriction: it brings these
labels in line with how any other multi-label suffix (assigned or
ASCII) already behaves.

Reporter: Jonas Mandalunes (attribution as given).
@arturobernalg
arturobernalg requested a review from ok2c November 19, 2026 10:45
@arturobernalg
arturobernalg merged commit 1ee8fb3 into apache:master Nov 19, 2026
12 checks passed
@arturobernalg

Copy link
Copy Markdown
Member Author

Cherry-picked to 5.6.x

@ok2c

ok2c commented Nov 19, 2026

Copy link
Copy Markdown
Member

@arturobernalg Please inform the original reporter. The reporter actually had patches prepared. Next time we should give people a chance to submit their patches as pull requests

@arturobernalg

Copy link
Copy Markdown
Member Author

@arturobernalg Please inform the original reporter. The reporter actually had patches prepared. Next time we should give people a chance to submit their patches as pull requests

@olegk Fair point. Started on this because it looked CVE-level — cross-tenant cookie leak plus a TLS hostname-verification bypass, both real and demonstrated. Didn't want that sitting around unfixed. Once I had a validated patch in hand, opening the PR was the fastest way to get it in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants