Scope cookies to the host that set them - #2075
Open
rzo1 wants to merge 1 commit into
Open
Conversation
rzo1
force-pushed
the
fix/cookie-host-scope
branch
from
August 27, 2026 12:51
25836d6 to
fd5035f
Compare
dpol1
requested changes
Aug 28, 2026
dpol1
left a comment
Member
There was a problem hiding this comment.
Two things before merging: the new domain guard is bypassable, and the null origin disables every host-only cookie (no Domain, in practice most session cookies) for anyone running http.use.cookies. Threads.
Side note on "both protocol implementations": httpclient went with #2071, only okhttp calls this now.
rzo1
force-pushed
the
fix/cookie-host-scope
branch
from
August 28, 2026 08:10
fd5035f to
15e9338
Compare
CookieConverter only checked the domain when the cookie carried a Domain attribute, so a cookie without one was sent to any target URL, a Domain which covers every host under it such as "com" or "co.uk" was accepted as a scope, and checkDomainMatchToUrl returned true when it threw. getCookies now takes the URL whose response set the cookies, keeps a cookie without a Domain attribute for that host only, and fails closed on error. A Domain attribute is normalised with IDN.toASCII and validated before it is used, so the unicode and punycode forms of a name are interchangeable and a malformed value such as "com.." is rejected rather than matching every host under "com". The public suffix list shipped with crawler-commons, its private section included, decides whether a domain may cover subdomains at all; a domain the list does not know about, e.g. an internal name under ".local", falls back to the plain suffix match so that unlisted domains keep working. A domain which can not own subdomains is ignored and the cookie is bound to the host, as RFC 6265 5.3 requires, so a single label intranet domain keeps its cookies while "com" no longer scopes anything. An address is only matched by itself, per RFC 6265 5.1.3, instead of covering the hosts under it as "2.3.4" covered "9.2.3.4". The protocols now record which host set the cookies. When a response carries Set-Cookie, okhttp and playwright write its url next to the header as set-cookie-origin, dropping any key of that name from the response first so that a response can not choose where its cookies are sent. The key has to travel with protocol.set-cookie for host-only cookies to be sent back, so it is added to the metadata.persist and metadata.transfer examples, and a warning naming both keys is logged once when cookies are present without it. The metadata.transfer example in internals.adoc named set-cookie instead of protocol.set-cookie and is corrected too. A Set-Cookie header whose first token has no "=" threw StringIndexOutOfBoundsException out of getCookies, failing the whole fetch, and is now skipped. Cookies dropped on the path, secure and expiry checks are logged like the other ones.
rzo1
force-pushed
the
fix/cookie-host-scope
branch
from
August 28, 2026 08:30
15e9338 to
8f00119
Compare
dpol1
approved these changes
Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CookieConverteronly checked applicability for cookies carrying aDomainattribute. A cookie without one, which is the usual session cookie, skipped the check and was sent to whatever host was being fetched. The domain check also passed on exception and accepted aDomaincovering every host under it, such ascomorco.uk.A three-argument
getCookiestaking the originating URL is added; the two-argument method is deprecated and kept. ADomainattribute is now normalised withIDN.toASCIIand validated before use, so the unicode and punycode forms of a name are interchangeable and a malformed value such ascom..is rejected instead of matching every host undercom. The public suffix list already shipped with crawler-commons, private section included, decides whether a domain may cover subdomains at all; a domain the list does not know about, e.g. an internal name under.local, falls back to the plain suffix match so unlisted domains keep working. A domain which cannot own subdomains is ignored and the cookie is bound to the host, per RFC 6265 5.3. An address is matched only by itself, per RFC 6265 5.1.3, whereDomain=2.3.4previously covered9.2.3.4.Two smaller ones found on the way: a
Set-Cookieheader whose first token has no=threwStringIndexOutOfBoundsExceptionout ofgetCookiesand failed the whole fetch, and is now skipped; cookies dropped on the path, secure and expiry checks are logged like the other ones.The protocols record which host set the cookies. When a response carries
Set-Cookie, okhttp and playwright write its URL alongside asset-cookie-origin, dropping any inbound key of that name first so a response cannot choose where its cookies are sent. That key has to travel withprotocol.set-cookiefor host-only cookies to be sent back, so it is added to themetadata.persistandmetadata.transferexamples and a warning naming both keys is logged once when cookies arrive without it. Also corrects themetadata.transferexample ininternals.adoc, which namedset-cookiewhere the key isprotocol.set-cookie.For all changes
Is there a issue associated with this PR? Is it referenced in the commit message? - no issue
Does your PR title start with
#XXXXwhereXXXXis the issue number you are trying to resolve? - no issueHas your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?
Is the code properly formatted with
mvn git-code-format:format-code -Dgcf.globPattern="**/*" -Dskip.format.code=false?For code changes
mvn clean verify? -coremodule tests onlyCookieConverterTestextended