feat: Add IPv6 zone ID detection and parsing support in HTTPAdapter#7065
Open
tboy1337 wants to merge 2 commits into
Open
feat: Add IPv6 zone ID detection and parsing support in HTTPAdapter#7065tboy1337 wants to merge 2 commits into
tboy1337 wants to merge 2 commits into
Conversation
|
We really need this issue fixed. Our customers are waiting for it. Others have also requested for a fix. Maintainers have so far completely ignored all requests (no pun intended) and remained silent. |
cf33244 to
fb86a0e
Compare
50246e5 to
a16aad9
Compare
d19ecbf to
628d097
Compare
Author
|
I think this is worthy of a review now @nateprewitt @sigmavirus24 |
This comment has been minimized.
This comment has been minimized.
Author
|
Hey @ani-sinha — since you mentioned your customers are waiting on this, would you or a colleague be able to test this branch and report back here? Even a simple "tested, works" or "tested, doesn't work" comment would really help get maintainer attention on this. |
|
@tboy1337: Works for me. Using latest stable release (2.33.1): >>> import requests
>>>
>>> requests.get('http://[fe80::211:32ff:fe89:dcaf%en0]/')
Traceback (most recent call last):
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connection.py", line 204, in _new_conn
sock = connection.create_connection(
(self._dns_host, self.port),
...<2 lines>...
socket_options=self.socket_options,
)
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
~~~~~~~~~~~~^^^^
OSError: [Errno 65] No route to host
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connectionpool.py", line 787, in urlopen
response = self._make_request(
conn,
...<10 lines>...
**response_kw,
)
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connectionpool.py", line 493, in _make_request
conn.request(
~~~~~~~~~~~~^
method,
^^^^^^^
...<6 lines>...
enforce_content_length=enforce_content_length,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connection.py", line 500, in request
self.endheaders()
~~~~~~~~~~~~~~~^^
File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/http/client.py", line 1353, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/http/client.py", line 1113, in _send_output
self.send(msg)
~~~~~~~~~^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/http/client.py", line 1057, in send
self.connect()
~~~~~~~~~~~~^^
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connection.py", line 331, in connect
self.sock = self._new_conn()
~~~~~~~~~~~~~~^^
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connection.py", line 219, in _new_conn
raise NewConnectionError(
self, f"Failed to establish a new connection: {e}"
) from e
urllib3.exceptions.NewConnectionError: HTTPConnection(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Failed to establish a new connection: [Errno 65] No route to host
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/adapters.py", line 645, in send
resp = conn.urlopen(
method=request.method,
...<9 lines>...
chunked=chunked,
)
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connectionpool.py", line 841, in urlopen
retries = retries.increment(
method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
)
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/util/retry.py", line 535, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Max retries exceeded with url: / (Caused by NewConnectionError("HTTPConnection(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Failed to establish a new connection: [Errno 65] No route to host"))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<python-input-5>", line 1, in <module>
requests.get('http://[fe80::211:32ff:fe89:dcaf%en0]/')
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/sessions.py", line 592, in request
resp = self.send(prep, **send_kwargs)
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/sessions.py", line 706, in send
r = adapter.send(request, **kwargs)
File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/adapters.py", line 678, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Max retries exceeded with url: / (Caused by NewConnectionError("HTTPConnection(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Failed to establish a new connection: [Errno 65] No route to host"))Using your branch: >>> import requests
>>>
>>> requests.get('http://[fe80::211:32ff:fe89:dcaf%en0]/')
<Response [200]> |
Add correct handling of IPv6 scope/zone identifiers (e.g. fe80::1%25eth0) across HTTPAdapter and PreparedRequest. - Add _has_ipv6_zone_id() helper in adapters.py using a regex anchored to the URL authority section to reliably detect zone IDs without false positives - Update _urllib3_request_context() to use urllib3 parse_url (instead of urlparse) when a zone ID is present, ensuring scheme, port, and hostname are extracted correctly for IPv6 scoped addresses - Update HTTPAdapter scheme resolution to use parse_url for zone ID URLs - Add RFC 6874 zone ID regex patterns in models.py (_AUTHORITY_BRACKET_RE, _RFC6874_ZONE_ID_RE, _RAW_ZONE_ID_RE) - Fix PreparedRequest.prepare_url() to preserve RFC 6874 encoded zone IDs (%25 delimiter) verbatim and re-encode raw % zone IDs (legacy form) as %25<zone> to prevent downstream ipaddress validation and connection errors
|
@tboy1337 you likely want to rebase onto main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR addresses #6927, #6735 and #7088 by adding full support for IPv6 zone identifiers (scope IDs) in URLs, enabling correct handling of link-local IPv6 addresses such as
http://[fe80::1%eth0]/.Background
IPv6 zone identifiers disambiguate link-local addresses (the
fe80::/10range) that may exist on multiple network interfaces. The zone ID is appended to the address with a%delimiter — for example,fe80::1%eth0. Previously, requests did not handle these URLs correctly, causing connection failures or parsing errors.Two encoding forms are in common use:
%25delimiter):http://[fe80::1%25eth0]/— the standards-compliant form%delimiter (legacy/non-standard):http://[fe80::1%eth0]/— still widely found in the wildChanges Made
src/requests/adapters.py(+61 -4)_IPV6_ZONE_ID_RE(new): Module-level compiled regex that detects zone IDs anchored to the URL authority section, correctly distinguishing RFC 6874%25and raw%forms from legitimate percent-encoded bytes (e.g.%AB,%20) and from brackets appearing in the path or query string._has_ipv6_zone_id(url)(new): Thin helper that returnsboolfrom the regex, used as a routing condition inside_urllib3_request_context._urllib3_request_context()(modified): Routes zone ID URLs throughurllib3.util.parse_url(which handles%in the host correctly) and all other URLs through the existingurlparsepath, with no changes to the function's signature or callers.src/requests/models.py(+44)_AUTHORITY_BRACKET_RE,_RFC6874_ZONE_ID_RE,_RAW_ZONE_ID_RE(new): Three module-level compiled regex patterns used byPreparedRequest.prepare_url.PreparedRequest.prepare_url(modified): Afterurllib3.util.parse_urlprocesses the URL it decodes%25to%, which can produce ambiguous percent sequences (e.g.%2550→%50) that Python 3.14's stricterurlparserejects. The fix inspects the original URL string before any decoding to determine the correct zone ID form and reconstructs the host with a canonical%25delimiter, avoiding false-positive re-encoding of legitimate%XXsequences.tests/test_adapters.py(+425)Three new test classes with comprehensive parametrized and integration coverage:
TestIPv6ZoneIDDetection— 30+ parametrized cases for_has_ipv6_zone_id, including false-positive guards for percent-encoded bytes (%AB,%20), brackets in path/query, bare%25with no following characters, and numeric zone IDs (%2550).TestIPv6ZoneIDParsing— Parametrized tests verifying thatbuild_connection_pool_key_attributesreturns the correcthost,port, andschemefor zone ID URLs (both%25and raw%forms), regular IPv6, IPv4, and hostnames.TestIPv6ZoneIDRequests— Integration tests covering connection pool key isolation across different zone IDs, client certificate pass-through, fullsend()flow with a mocked pool manager, encoding normalisation (%25eth0and%eth0resolving to the same pool key),%25preservation through URL preparation, Python 3.14urlparsevalidation behaviour,request_url()path extraction, and proxy connection routing.Files Changed
src/requests/adapters.py— Core implementation (+61 -4 lines)src/requests/models.py— URL preparation fix (+44 lines)tests/test_adapters.py— Test suite (+425 lines)Backward Compatibility
Fully backward compatible. Standard URLs without zone identifiers continue through the unchanged
urlparsepath. No existing function signatures, call sites, or public APIs have been modified.