Skip to content

feat: Add IPv6 zone ID detection and parsing support in HTTPAdapter#7065

Open
tboy1337 wants to merge 2 commits into
psf:mainfrom
tboy1337:edit-14
Open

feat: Add IPv6 zone ID detection and parsing support in HTTPAdapter#7065
tboy1337 wants to merge 2 commits into
psf:mainfrom
tboy1337:edit-14

Conversation

@tboy1337

@tboy1337 tboy1337 commented Oct 23, 2025

Copy link
Copy Markdown

Summary

This PR addresses #6927, #6735 and #7088 by adding full support for IPv6 zone identifiers (scope IDs) in URLs, enabling correct handling of link-local IPv6 addresses such as http://[fe80::1%eth0]/.

Background

IPv6 zone identifiers disambiguate link-local addresses (the fe80::/10 range) that may exist on multiple network interfaces. The zone ID is appended to the address with a % delimiter — for example, fe80::1%eth0. Previously, requests did not handle these URLs correctly, causing connection failures or parsing errors.

Two encoding forms are in common use:

  • RFC 6874 (%25 delimiter): http://[fe80::1%25eth0]/ — the standards-compliant form
  • Raw % delimiter (legacy/non-standard): http://[fe80::1%eth0]/ — still widely found in the wild

Changes Made

src/requests/adapters.py (+61 -4)

  • _IPV6_ZONE_ID_RE (new): Module-level compiled regex that detects zone IDs anchored to the URL authority section, correctly distinguishing RFC 6874 %25 and raw % forms from legitimate percent-encoded bytes (e.g. %AB, %20) and from brackets appearing in the path or query string.
  • _has_ipv6_zone_id(url) (new): Thin helper that returns bool from the regex, used as a routing condition inside _urllib3_request_context.
  • _urllib3_request_context() (modified): Routes zone ID URLs through urllib3.util.parse_url (which handles % in the host correctly) and all other URLs through the existing urlparse path, with no changes to the function's signature or callers.

src/requests/models.py (+44)

  • _AUTHORITY_BRACKET_RE, _RFC6874_ZONE_ID_RE, _RAW_ZONE_ID_RE (new): Three module-level compiled regex patterns used by PreparedRequest.prepare_url.
  • PreparedRequest.prepare_url (modified): After urllib3.util.parse_url processes the URL it decodes %25 to %, which can produce ambiguous percent sequences (e.g. %2550%50) that Python 3.14's stricter urlparse rejects. The fix inspects the original URL string before any decoding to determine the correct zone ID form and reconstructs the host with a canonical %25 delimiter, avoiding false-positive re-encoding of legitimate %XX sequences.

tests/test_adapters.py (+425)

Three new test classes with comprehensive parametrized and integration coverage:

  • TestIPv6ZoneIDDetection — 30+ parametrized cases for _has_ipv6_zone_id, including false-positive guards for percent-encoded bytes (%AB, %20), brackets in path/query, bare %25 with no following characters, and numeric zone IDs (%2550).
  • TestIPv6ZoneIDParsing — Parametrized tests verifying that build_connection_pool_key_attributes returns the correct host, port, and scheme for zone ID URLs (both %25 and raw % forms), regular IPv6, IPv4, and hostnames.
  • TestIPv6ZoneIDRequests — Integration tests covering connection pool key isolation across different zone IDs, client certificate pass-through, full send() flow with a mocked pool manager, encoding normalisation (%25eth0 and %eth0 resolving to the same pool key), %25 preservation through URL preparation, Python 3.14 urlparse validation behaviour, request_url() path extraction, and proxy connection routing.

Files Changed

  • src/requests/adapters.py — Core implementation (+61 -4 lines)
  • src/requests/models.py — URL preparation fix (+44 lines)
  • tests/test_adapters.py — Test suite (+425 lines)

Backward Compatibility

Fully backward compatible. Standard URLs without zone identifiers continue through the unchanged urlparse path. No existing function signatures, call sites, or public APIs have been modified.

@tboy1337 tboy1337 changed the title Add IPv6 zone ID detection and parsing support in HTTPAdapter feat: Add IPv6 zone ID detection and parsing support in HTTPAdapter Oct 24, 2025
@ani-sinha

Copy link
Copy Markdown

We really need this issue fixed. Our customers are waiting for it. Others have also requested for a fix. Maintainers have so far completely ignored all requests (no pun intended) and remained silent.

@tboy1337 tboy1337 force-pushed the edit-14 branch 2 times, most recently from cf33244 to fb86a0e Compare April 1, 2026 08:02
@tboy1337 tboy1337 marked this pull request as draft April 1, 2026 08:05
@tboy1337 tboy1337 force-pushed the edit-14 branch 2 times, most recently from 50246e5 to a16aad9 Compare April 12, 2026 14:46
@tboy1337 tboy1337 marked this pull request as ready for review April 12, 2026 15:44
@tboy1337 tboy1337 force-pushed the edit-14 branch 3 times, most recently from d19ecbf to 628d097 Compare April 13, 2026 06:31
@tboy1337

Copy link
Copy Markdown
Author

I think this is worthy of a review now @nateprewitt @sigmavirus24

@Jah-yee

This comment has been minimized.

@tboy1337

tboy1337 commented May 2, 2026

Copy link
Copy Markdown
Author

Hey @ani-sinha — since you mentioned your customers are waiting on this, would you or a colleague be able to test this branch and report back here? Even a simple "tested, works" or "tested, doesn't work" comment would really help get maintainer attention on this.

@jleroy

jleroy commented May 4, 2026

Copy link
Copy Markdown

@tboy1337: Works for me.

Using latest stable release (2.33.1):

>>> import requests
>>>
>>> requests.get('http://[fe80::211:32ff:fe89:dcaf%en0]/')
Traceback (most recent call last):
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connection.py", line 204, in _new_conn
    sock = connection.create_connection(
        (self._dns_host, self.port),
    ...<2 lines>...
        socket_options=self.socket_options,
    )
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
OSError: [Errno 65] No route to host

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
        conn,
    ...<10 lines>...
        **response_kw,
    )
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connectionpool.py", line 493, in _make_request
    conn.request(
    ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<6 lines>...
        enforce_content_length=enforce_content_length,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connection.py", line 500, in request
    self.endheaders()
    ~~~~~~~~~~~~~~~^^
  File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/http/client.py", line 1353, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/http/client.py", line 1113, in _send_output
    self.send(msg)
    ~~~~~~~~~^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/http/client.py", line 1057, in send
    self.connect()
    ~~~~~~~~~~~~^^
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connection.py", line 331, in connect
    self.sock = self._new_conn()
                ~~~~~~~~~~~~~~^^
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connection.py", line 219, in _new_conn
    raise NewConnectionError(
        self, f"Failed to establish a new connection: {e}"
    ) from e
urllib3.exceptions.NewConnectionError: HTTPConnection(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Failed to establish a new connection: [Errno 65] No route to host

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/adapters.py", line 645, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/urllib3/util/retry.py", line 535, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Max retries exceeded with url: / (Caused by NewConnectionError("HTTPConnection(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Failed to establish a new connection: [Errno 65] No route to host"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    requests.get('http://[fe80::211:32ff:fe89:dcaf%en0]/')
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/sessions.py", line 592, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/sessions.py", line 706, in send
    r = adapter.send(request, **kwargs)
  File "/Users/jleroy/.pyvenv/requests/lib/python3.14/site-packages/requests/adapters.py", line 678, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Max retries exceeded with url: / (Caused by NewConnectionError("HTTPConnection(host='fe80::211:32ff:fe89:dcaf%25en0', port=80): Failed to establish a new connection: [Errno 65] No route to host"))

Using your branch:

>>> import requests
>>>
>>> requests.get('http://[fe80::211:32ff:fe89:dcaf%en0]/')
<Response [200]>

tboy1337 added 2 commits June 11, 2026 21:48
Add correct handling of IPv6 scope/zone identifiers (e.g. fe80::1%25eth0)
across HTTPAdapter and PreparedRequest.

- Add _has_ipv6_zone_id() helper in adapters.py using a regex anchored to the
  URL authority section to reliably detect zone IDs without false positives
- Update _urllib3_request_context() to use urllib3 parse_url (instead of
  urlparse) when a zone ID is present, ensuring scheme, port, and hostname
  are extracted correctly for IPv6 scoped addresses
- Update HTTPAdapter scheme resolution to use parse_url for zone ID URLs
- Add RFC 6874 zone ID regex patterns in models.py (_AUTHORITY_BRACKET_RE,
  _RFC6874_ZONE_ID_RE, _RAW_ZONE_ID_RE)
- Fix PreparedRequest.prepare_url() to preserve RFC 6874 encoded zone IDs
  (%25 delimiter) verbatim and re-encode raw % zone IDs (legacy form) as
  %25<zone> to prevent downstream ipaddress validation and connection errors
@frittentheke

Copy link
Copy Markdown

@tboy1337 you likely want to rebase onto main.
Thanks for pursuing this chance. Would love to see IPv6 zone support in requests ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants