fix(security): resolve and pin a caller-supplied proxy in public mode

GHSA-q3h8-73xx-gwqx. In security.request_proxy=public the proxy host was
judged by its text alone, so any name pointing at an internal address was
accepted - 127-0-0-1.sslip.io as reported, or any domain with such an A
record.

- vet() resolves the host, refuses the proxy if any answer is non-global,
  and hands on the address it checked so nothing downstream resolves the
  name again. https proxies keep their name; certificate verification
  covers them. An unresolvable or slow name is refused (host_unresolvable).
- normalize() refuses an authority with characters outside RFC 3986
  (authority_invalid), so the host checked is the host the client dials.
- is_private_host() also recognises hyphen-spelled non-global addresses,
  giving an early, honest refusal to every caller of it.

documents/en/11-api.md

@@ -481,9 +481,9 @@ See [Identities and proxies](./06-identities-and-proxies.md).
 Sends the upstream request through an egress **you** supply, replacing the identity's own exit.
 
 - Refused unless an administrator sets `security.request_proxy` to `public` or `any`. The default is `deny`, and an unrecognised value is treated as `deny` — a typo in configuration must not be the thing that opens a network.
-- `public` accepts only publicly routable destinations; `any` accepts loopback and private ranges too and is only coherent when every API key holder is already trusted with the network the instance runs in.
+- `public` accepts only publicly routable destinations. A proxy given by name is resolved, every address it answers with must be public, and the request is dialled at the address that was checked rather than at the name — so `http://proxy.example.com:8080` is sent on as `http://<its address>:8080`. An `https` proxy keeps its name, because its certificate is verified against it. `any` accepts loopback and private ranges too and is only coherent when every API key holder is already trusted with the network the instance runs in.
 - Schemes: `http`, `https`, `socks5`, `socks5h`. Maximum 512 characters. Give it as a full URL, e.g. `http://host:port`.
-- A refusal is `INVALID_PARAM` with `details.reason` — one of `request_proxy_disabled`, `too_long`, `scheme_missing`, `scheme_not_supported`, `host_missing`, `port_invalid`, `host_not_public`. The value you sent is never echoed back or logged, because a rejected proxy URL is exactly when someone has pasted a real credential.
+- A refusal is `INVALID_PARAM` with `details.reason` — one of `request_proxy_disabled`, `too_long`, `scheme_missing`, `scheme_not_supported`, `authority_invalid`, `host_missing`, `port_invalid`, `host_not_public`, `host_unresolvable`. The value you sent is never echoed back or logged, because a rejected proxy URL is exactly when someone has pasted a real credential.
 - A disabled feature **refuses** rather than ignores. Silently dropping the parameter would send the request from the instance's own address while you believed it went through your proxy.
 
 The cost is real and is not a free option: the identity's cookies were minted behind one address and would now be presented from another, which is an incoherence the platforms can see. Two callers asking for the same post through different proxies are not asking the same question and are never coalesced.

documents/en/15-security.md

@@ -357,7 +357,7 @@ forgery in its plainest form.
 would already trust with the network the instance runs in. It is the largest
 single widening available in the settings table.
 
-Three details of the implementation are deliberate:
+Four details of the implementation are deliberate:
 
 - **A disabled feature refuses rather than ignores.** Silently dropping the
   parameter would send the request from the instance's own address while the
@@ -368,6 +368,15 @@ Three details of the implementation are deliberate:
 - **The value is never echoed back or logged.** A proxy URL carries
   credentials, and a rejected request is exactly when somebody is most likely to
   have pasted a real one. Only the scheme and the mode are logged.
+- **In `public` mode the name is resolved and the address pinned.** The text of
+  a hostname proves nothing about where it leads: `127-0-0-1.sslip.io`, or any
+  domain whose owner points it at `127.0.0.1`, is a public-looking name for
+  loopback. So every address the name resolves to must be public, and the
+  request is dialled at the address that was checked rather than at the name —
+  a second lookup never happens, so it cannot answer differently. An `https`
+  proxy keeps its name, because its TLS certificate is verified against it and
+  an address the name was rebound to cannot produce one. A name that does not
+  resolve within five seconds is refused with `host_unresolvable`.
 
 Accepted values are at most 512 characters and must use `http`, `https`,
 `socks5` or `socks5h`.

documents/zh/11-api.md

@@ -480,9 +480,9 @@ curl -sS "$DTK_BASE_URL/api/v1/parse?lang=zh" -X POST \
 让上游请求走**你**提供的出口，替换掉该身份自己的出口。
 
 - 除非管理员把 `security.request_proxy` 设为 `public` 或 `any`，否则一律拒绝。默认是 `deny`，而且无法识别的值也按 `deny` 处理——配置里的一个笔误不该成为打开网络的那件事。
-- `public` 只接受公网可路由的目的地；`any` 连回环和私有网段也接受，只有在每一个持有 API key 的人都已经被信任可以访问该实例所在网络时才说得通。
+- `public` 只接受公网可路由的目的地。以域名给出的代理会先被解析，解析出的每一个地址都必须是公网地址，之后请求拨向的是通过检查的那个地址而不是域名——所以 `http://proxy.example.com:8080` 往下传时会变成 `http://<它的地址>:8080`。`https` 代理保留域名，因为它的证书要按域名校验。`any` 连回环和私有网段也接受，只有在每一个持有 API key 的人都已经被信任可以访问该实例所在网络时才说得通。
 - 协议：`http`、`https`、`socks5`、`socks5h`。最长 512 字符。请写成完整 URL，例如 `http://host:port`。
-- 拒绝时返回 `INVALID_PARAM`，`details.reason` 是 `request_proxy_disabled`、`too_long`、`scheme_missing`、`scheme_not_supported`、`host_missing`、`port_invalid`、`host_not_public` 之一。你发的值永远不会被回显或写日志，因为一个被拒的代理 URL 恰恰是最可能刚粘贴了真实凭据的时刻。
+- 拒绝时返回 `INVALID_PARAM`，`details.reason` 是 `request_proxy_disabled`、`too_long`、`scheme_missing`、`scheme_not_supported`、`authority_invalid`、`host_missing`、`port_invalid`、`host_not_public`、`host_unresolvable` 之一。你发的值永远不会被回显或写日志，因为一个被拒的代理 URL 恰恰是最可能刚粘贴了真实凭据的时刻。
 - 功能被关闭时是**拒绝**而不是忽略。悄悄丢掉这个参数，会让请求从实例自己的地址发出去，而你以为它走了你的代理。
 
 代价是真实存在的，不是白送的选项：这个身份的 cookie 是在某个地址后面签发的，现在却从另一个地址出示，这正是平台看得见的那种自相矛盾。两个通过不同代理请求同一条作品的调用方，问的不是同一个问题，所以永远不会被合并。

documents/zh/15-security.md

@@ -253,11 +253,12 @@ curl -s -D - -o /dev/null http://127.0.0.1:8000/healthz
 
 只有当实例上的每一把 API 密钥都握在“你本来就愿意让他接触这台机器所在网络”的人手里时，`any` 才是自洽的。它是整张配置表里能做出的最大一次放宽。
 
-实现上有三个细节是刻意的：
+实现上有四个细节是刻意的：
 
 - **被禁用的功能是拒绝，而不是忽略。** 悄悄丢掉这个参数，会让请求从实例自己的地址发出去，而调用方以为它走了自己的代理——这比报错更糟，因为他只能从对端才发现。
 - **无法识别的配置值按 `deny` 处理。** 运维配置里的一个笔误，绝不能成为打开他们内网的那个东西。
 - **这个值从不回显，也从不写日志。** 代理 URL 里带着凭据，而请求被拒的那一刻，恰恰是人最可能刚粘了一个真值的时候。只有 scheme 和当前模式会被记录。
+- **`public` 模式下会解析域名并钉住地址。** 主机名的字面内容说明不了它指向哪里：`127-0-0-1.sslip.io`，或者任何被主人指向 `127.0.0.1` 的域名，都是一个看起来像公网的回环地址。所以域名解析出的每一个地址都必须是公网地址，而请求拨向的是通过检查的那个地址而不是域名——第二次解析根本不会发生，也就不可能得到不同的答案。`https` 代理保留域名，因为它的 TLS 证书按域名校验，被重绑定到的地址拿不出这张证书。五秒内解析不出来的域名以 `host_unresolvable` 拒绝。
 
 被接受的值最长 512 个字符，scheme 必须是 `http`、`https`、`socks5` 或 `socks5h`。
 

src/dtk/api/request_proxy.py

@@ -18,21 +18,30 @@ So the default is to refuse the parameter entirely. An operator who wants the
 feature opts in with ``security.request_proxy``, and the middle setting - the
 one to actually use - keeps the caller on publicly routable addresses.
 
-What this module does NOT do is resolve DNS. A name that resolves to a private
-address today can resolve elsewhere between this check and the connection, so a
-lookup here would buy a false sense of safety at the cost of a network call in a
-request path; :func:`dtk.urls.is_private_host` takes the same position for the
-same reason. The literal forms that exist only to evade string matching -
-decimal, hex, IPv4-mapped IPv6, trailing dots - are what it does catch, and it
-catches them by refusing anything that is not globally routable rather than by
-listing what is bad.
+In that mode the host is resolved, every answer is checked, and the URL handed
+on names the address that passed rather than the name. This module used to
+refuse to resolve at all, on the grounds that a name can point elsewhere between
+the check and the connection. The conclusion did not follow from the reason:
+checking only the text left every name that simply points inside wide open, and
+GHSA-q3h8-73xx-gwqx walked through with ``127-0-0-1.sslip.io`` - no race, just
+a public DNS service that answers with the address the name spells. The race is
+real, and pinning is what closes it: the worker, the browser service and anything
+else downstream dial the literal that was checked and never look the name up
+again. The literal forms that exist only to evade string matching - decimal,
+hex, IPv4-mapped IPv6, trailing dots - are still refused before any lookup, by
+refusing anything not globally routable rather than by listing what is bad.
 """
 
 from __future__ import annotations
 
+import asyncio
+import ipaddress
+import re
+import socket
+from collections.abc import Awaitable, Callable, Sequence
 from enum import StrEnum
 from typing import Final
-from urllib.parse import urlsplit
+from urllib.parse import SplitResult, urlsplit, urlunsplit
 
 from dtk.core.errors import DtkError, InvalidParam
 from dtk.core.logging import get_logger
@@ -49,6 +58,19 @@ ALLOWED_SCHEMES: Final[frozenset[str]] = frozenset({"http", "https", "socks5", "
 #: is someone probing the parser.
 MAX_LENGTH: Final = 512
 
+#: What RFC 3986 allows in an authority: unreserved, sub-delims, percent
+#: escapes, and the ":", "@" and brackets that give it structure.
+AUTHORITY_RE: Final = re.compile(r"[A-Za-z0-9\-._~!$&'()*+,;=%:@\[\]]+")
+
+#: How long the proxy's name gets to resolve. The lookup holds an API request
+#: open, and a proxy whose own name does not answer is not going to carry
+#: traffic either, so a slow one is refused rather than waited on.
+RESOLVE_TIMEOUT_SECONDS: Final = 5.0
+
+#: Looks a host up and returns every address it answers with. Injected so the
+#: tests decide what a name resolves to without touching the network.
+Resolver = Callable[[str], Awaitable[Sequence[str]]]
+
 
 class RequestProxyMode(StrEnum):
     """What ``security.request_proxy`` may be set to.
@@ -82,10 +104,12 @@ def _reject(reason: str, detail: str) -> DtkError:
 
 
 def normalize(value: str | None, *, mode: RequestProxyMode) -> str | None:
-    """Validate a caller-supplied proxy, or explain why it cannot be used.
+    """Judge the text of a caller-supplied proxy, or explain why it cannot be used.
 
-    Returns the URL to dial, or None when the caller supplied nothing. Never
-    returns a value the current mode does not permit.
+    Returns the URL as given, or None when the caller supplied nothing. This
+    is the offline half: it cannot tell where a name leads, so in ``public``
+    mode its answer is not yet safe to dial. Routes call :func:`vet`, which
+    finishes the job.
 
     A disabled feature REFUSES rather than ignores. Silently dropping the
     parameter would send the request from the instance's own address while the
@@ -120,6 +144,18 @@ def normalize(value: str | None, *, mode: RequestProxyMode) -> str | None:
             f"proxy scheme must be one of {', '.join(sorted(ALLOWED_SCHEMES))}",
         )
 
+    # Checked before asking urlsplit for the host, because the host it names
+    # is only worth checking if the client that dials agrees with it. wreq and
+    # the browser parse by WHATWG rules, which end the authority at a backslash
+    # where Python reads on to the last "@"; any character RFC 3986 does not
+    # allow in an authority is a place the two can disagree.
+    if not AUTHORITY_RE.fullmatch(parts.netloc):
+        raise _reject(
+            "authority_invalid",
+            "the proxy host, port or credentials contain characters that are "
+            "not allowed there; percent-encode them",
+        )
+
     try:
         host, port = parts.hostname, parts.port
     except ValueError as exc:  # a port that is not a number at all
@@ -137,16 +173,115 @@ def normalize(value: str | None, *, mode: RequestProxyMode) -> str | None:
             "link-local and intranet names are refused",
         )
 
-    # Host and scheme only. The value can carry credentials, so nothing that
-    # could hold the whole URL is emitted.
-    log.info("api.request_proxy.accepted", scheme=scheme, mode=mode.value)
     return text
 
 
+async def resolve_host(host: str) -> Sequence[str]:
+    """Every address ``host`` answers with, from the event loop's resolver.
+
+    The loop's resolver rather than ``socket.getaddrinfo``, which blocks: this
+    runs inside an API request.
+    """
+    infos = await asyncio.get_running_loop().getaddrinfo(host, None, type=socket.SOCK_STREAM)
+    return [str(info[4][0]) for info in infos]
+
+
+async def vet(
+    value: str | None,
+    *,
+    mode: RequestProxyMode,
+    resolve: Resolver | None = None,
+) -> str | None:
+    """Validate a caller-supplied proxy and fix the address it will be dialled at.
+
+    Returns the URL to dial, or None when the caller supplied nothing. Never
+    returns a value the current mode does not permit: in ``public`` mode a
+    name comes back replaced by the address that was checked, so nothing
+    downstream resolves it a second time. ``any`` mode is handed back as given,
+    because it permits every answer a lookup could give.
+    """
+    text = normalize(value, mode=mode)
+    if text is None:
+        return None
+
+    dial = text
+    if mode is RequestProxyMode.PUBLIC:
+        dial = await _pin(text, resolve or resolve_host)
+
+    # Scheme and mode only. The value can carry credentials, so nothing that
+    # could hold the whole URL is emitted.
+    log.info(
+        "api.request_proxy.accepted",
+        scheme=urlsplit(text).scheme.lower(),
+        mode=mode.value,
+        pinned=dial != text,
+    )
+    return dial
+
+
+async def _pin(text: str, resolve: Resolver) -> str:
+    """Resolve the proxy host, refuse any private answer, dial the one checked.
+
+    Every answer is checked, not just the one that will be used: a record with
+    a public and a private member is someone hedging, and refusing it costs a
+    legitimate proxy nothing.
+    """
+    parts = urlsplit(text)
+    host = parts.hostname or ""
+    try:
+        ipaddress.ip_address(host)
+    except ValueError:
+        pass
+    else:
+        # A literal was judged as the address itself; there is nothing to look up.
+        return text
+
+    try:
+        async with asyncio.timeout(RESOLVE_TIMEOUT_SECONDS):
+            answers = list(await resolve(host))
+    except (OSError, TimeoutError, ValueError) as exc:
+        # ValueError covers what the IDNA codec raises for a label it cannot
+        # encode, before any query is sent.
+        raise _reject("host_unresolvable", "the proxy host does not resolve") from exc
+    if not answers:
+        raise _reject("host_unresolvable", "the proxy host does not resolve")
+
+    if any(is_private_host(answer) for answer in answers):
+        raise _reject(
+            "host_not_public",
+            "the proxy host resolves to a loopback, private, link-local or "
+            "otherwise non-public address",
+        )
+
+    if parts.scheme.lower() == "https":
+        # The TLS handshake with the proxy is verified against its name, and
+        # that is the defence here: a name rebound to an internal address
+        # cannot present a certificate for the caller's hostname. Pinning would
+        # break exactly that check, so the name is kept.
+        return text
+
+    # IPv4 first when there is one. A container with no IPv6 route is the
+    # common deployment, and a pinned address gets no happy-eyeballs fallback.
+    ipv4 = [answer for answer in answers if ":" not in answer]
+    return _with_host(parts, (ipv4 or answers)[0])
+
+
+def _with_host(parts: SplitResult, address: str) -> str:
+    """``parts`` with its host swapped for ``address``; userinfo and port kept."""
+    userinfo, at, _ = parts.netloc.rpartition("@")
+    host = f"[{address}]" if ":" in address else address
+    port = f":{parts.port}" if parts.port is not None else ""
+    return urlunsplit(parts._replace(netloc=f"{userinfo}{at}{host}{port}"))
+
+
 __all__ = [
     "ALLOWED_SCHEMES",
     "MAX_LENGTH",
+    "RESOLVE_TIMEOUT_SECONDS",
     "SETTING_KEY",
     "RequestProxyMode",
+    "Resolver",
     "normalize",
+    "resolve_host",
+    "vet",
 ]

src/dtk/api/routes/content.py

@@ -316,7 +316,7 @@ async def parse(
             "callback_url": callback,
         },
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         # The platform comes from the link rather than from the path here, and
@@ -360,7 +360,7 @@ async def batch(
     # differ between items, and checking N times would log N acceptances. The
     # pin is checked without a platform for the same reason it is on /parse -
     # each item names its own link, and they need not agree.
-    egress = resolve_request_proxy(request, proxy)
+    egress = await resolve_request_proxy(request, proxy)
     pinned = await resolve_request_identity(request, principal, identity)
 
     results: list[dict[str, Any]] = []
@@ -470,7 +470,7 @@ async def video(
         endpoint=endpoint_name(platform, Operation.CONTENT_DETAIL),
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -542,7 +542,7 @@ async def comments(
         endpoint=endpoint_name(platform, Operation.COMMENTS),
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -618,7 +618,7 @@ async def comment_replies(
         endpoint=endpoint_name(platform, Operation.COMMENT_REPLIES),
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -678,7 +678,7 @@ async def user(
         endpoint=endpoint_name(platform, Operation.AUTHOR_PROFILE),
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -751,7 +751,7 @@ async def user_posts(
         endpoint=endpoint_name(platform, Operation.AUTHOR_POSTS),
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -828,7 +828,7 @@ async def user_likes(
         endpoint=supported(platform, Operation.AUTHOR_LIKES),
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -903,7 +903,7 @@ async def user_reposts(
         endpoint=supported(platform, Operation.AUTHOR_REPOSTS),
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -987,7 +987,7 @@ async def user_collections(
         endpoint=endpoint,
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -1062,7 +1062,7 @@ async def user_bookmarks(
         endpoint=supported(platform, Operation.AUTHOR_BOOKMARKS),
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -1124,7 +1124,7 @@ async def collection_detail(
         endpoint=supported(platform, Operation.COLLECTION_DETAIL),
         params={"collection_id": collection_id, "include_raw": include_raw},
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -1197,7 +1197,7 @@ async def collection_posts(
             "include_raw": include_raw,
         },
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -1264,7 +1264,7 @@ async def mix_posts(
             "include_raw": include_raw,
         },
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -1338,7 +1338,7 @@ async def user_followers(
         endpoint=endpoint,
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),
@@ -1411,7 +1411,7 @@ async def user_following(
         endpoint=endpoint,
         params=params,
         wait=resolve_wait(request, wait),
-        proxy=resolve_request_proxy(request, proxy),
+        proxy=await resolve_request_proxy(request, proxy),
         refresh=refresh,
         explain=await resolve_explain(request, principal, explain),
         identity=await resolve_request_identity(request, principal, identity, platform=platform),

src/dtk/api/routes/support.py

@@ -305,11 +305,13 @@ def resolve_count(
     return value
 
 
-def resolve_request_proxy(request: Request, value: str | None) -> str | None:
+async def resolve_request_proxy(request: Request, value: str | None) -> str | None:
     """Vet a caller-supplied egress against this instance's setting.
 
     The mode lives in ``security.request_proxy`` and defaults to refusing the
     parameter; :mod:`dtk.api.request_proxy` explains why, and does the checking.
+    Async because ``public`` mode resolves the proxy's name and hands back the
+    address it checked, not the name.
     An unrecognised setting value is treated as ``deny`` rather than as a
     permissive default - a typo in an operator's configuration must not be the
     thing that opens their network.
@@ -322,7 +324,7 @@ def resolve_request_proxy(request: Request, value: str | None) -> str | None:
     except ValueError:
         log.warning("api.request_proxy.unknown_mode", configured=raw)
         mode = request_proxy.RequestProxyMode.DENY
-    return request_proxy.normalize(value, mode=mode)
+    return await request_proxy.vet(value, mode=mode)
 
 
 #: The scopes and role a caller needs before they may name an identity. They

src/dtk/api/routes/tools.py

@@ -32,7 +32,7 @@ from pydantic import BaseModel, ConfigDict, Field
 
 from dtk.api.deps import Principal, enforce_rate_limit
 from dtk.api.request_proxy import RequestProxyMode
-from dtk.api.request_proxy import normalize as normalize_proxy
+from dtk.api.request_proxy import vet as vet_proxy
 from dtk.api.routes.openapi import I18N_KEY
 from dtk.api.routes.support import ok
 from dtk.core.errors import InvalidParam, NotConfigured, UpstreamRiskControl
@@ -888,7 +888,7 @@ async def mint_identity(
         mode = RequestProxyMode(raw_mode)
     except ValueError:
         mode = RequestProxyMode.DENY
-    egress = normalize_proxy(body.proxy, mode=mode)
+    egress = await vet_proxy(body.proxy, mode=mode)
 
     client = BrowserRpcClient(base_url)
     try:

src/dtk/urls/parse.py

@@ -122,11 +122,17 @@ def is_private_host(host: str) -> bool:
     Handles address literals (v4, v6, bracketed, IPv4-mapped), the usual local
     names, and single-label intranet names. Decimal or hex integer forms of an
     address (``2130706433``, ``0x7f000001``) are refused as well: they are never
-    a real hostname and exist only to slip past naive string checks.
+    a real hostname and exist only to slip past naive string checks. So is a
+    label that spells a non-global address with hyphens, see
+    :func:`_spells_private_address`.
 
-    DNS is deliberately not consulted. Resolving here would add a network call
-    to a pure function and would still be a time-of-check/time-of-use gap, so
-    the defence is the positive allowlist plus a proxy on every outbound call.
+    DNS is deliberately not consulted, which makes this a judgement of the
+    *text* and nothing more. That is enough where a positive allowlist follows
+    it. It is not enough where the caller picks the host outright: any name can
+    be given an A record of 127.0.0.1, and GHSA-q3h8-73xx-gwqx did exactly that
+    with a public wildcard-DNS service. Such a caller must resolve the name,
+    check every answer here, and dial the answer it checked -
+    :mod:`dtk.api.request_proxy` is the example.
     """
     name = host.strip().strip("[]").rstrip(".").lower()
     if not name:
@@ -145,7 +151,52 @@ def is_private_host(host: str) -> bool:
     if len(labels) == 1:
         # A bare label is an intranet name; it can never be an allowed domain.
         return True
-    return any(NUMERIC_HOST_RE.match(label) for label in labels)
+    return any(NUMERIC_HOST_RE.match(label) or _spells_private_address(label) for label in labels)
+
+
+def _spells_private_address(label: str) -> bool:
+    """True when one DNS label spells a non-global address in hyphens.
+
+    Wildcard-DNS services answer with whatever address the name spells:
+    ``127-0-0-1.sslip.io`` and ``app-10-0-0-1.nip.io`` resolve to exactly what
+    they say, forever, for anyone. The dotted spelling (``127.0.0.1.nip.io``)
+    was always refused by the numeric-label rule; the hyphenated one read as an
+    ordinary public name until GHSA-q3h8-73xx-gwqx.
+
+    Only a non-global address counts. ``ec2-54-12-34-56.compute-1.amazonaws.com``
+    spells a public address and is a perfectly real hostname.
+
+    Hex spellings (``7f000001``) are not decoded. Eight hex digits is also what
+    a CDN's hash label looks like, and about one random label in thirteen
+    decodes to a non-global range - a media host refused for its hash. That
+    spelling, and every name whose owner simply points it at 127.0.0.1, is
+    caught by resolving; :func:`is_private_host` says who has to.
+    """
+    parts = label.split("-")
+    # Every window of four, not just the whole label: the services accept a
+    # prefix (``customer1-app-10-0-0-1``), and a prefix of digits must not hide
+    # the address after it.
+    for start in range(len(parts) - 3):
+        window = parts[start : start + 4]
+        if not all(len(part) <= 3 and part.isascii() and part.isdigit() for part in window):
+            continue
+        # int() drops leading zeros, which ipaddress refuses outright. Reading
+        # ``010`` as ten can only err toward refusing, never toward letting in.
+        try:
+            address = ipaddress.IPv4Address(".".join(str(int(part)) for part in window))
+        except ValueError:
+            continue
+        if not address.is_global:
+            return True
+
+    # sslip.io spells IPv6 with a hyphen per colon: ``--1`` is ``::1``.
+    if "-" in label:
+        try:
+            v6 = ipaddress.IPv6Address(label.replace("-", ":"))
+        except ValueError:
+            return False
+        return not v6.is_global
+    return False
 
 
 def _registrable_domain(host: str) -> str | None:

tests/unit/test_request_proxy.py

@@ -8,13 +8,23 @@ inside the instance's own network and have it dialled.
 
 from __future__ import annotations
 
+import asyncio
+from collections.abc import Sequence
+
 import pytest
 
-from dtk.api.request_proxy import MAX_LENGTH, RequestProxyMode, normalize
+from dtk.api import request_proxy
+from dtk.api.request_proxy import MAX_LENGTH, RequestProxyMode, Resolver, normalize, vet
 from dtk.core.errors import InvalidParam
 
 PUBLIC = RequestProxyMode.PUBLIC
 
+#: A globally routable address to stand in for "a real proxy's A record".
+#: Not one of the documentation ranges: those are not global, and the check
+#: under test would refuse them.
+PUBLIC_V4 = "93.184.216.34"
+PUBLIC_V6 = "2606:4700:4700::1111"
+
 
 def _reason(excinfo: pytest.ExceptionInfo[Exception]) -> str:
     """The stable machine-readable half of the refusal."""
@@ -144,3 +154,188 @@ def test_the_rejected_value_is_never_echoed_back() -> None:
     rendered = str(excinfo.value) + repr(excinfo.value.details)
     assert "hunter2" not in rendered
     assert secret not in rendered
+
+
+# --------------------------------------------------------------------------
+# Names that point inside the perimeter (GHSA-q3h8-73xx-gwqx)
+#
+# A name is the caller's to point anywhere, so judging the text of one proves
+# nothing about where it leads. `127-0-0-1.sslip.io` is the public spelling of
+# that, and an attacker's own domain with an A record of 127.0.0.1 is the one
+# no string check can ever catch.
+# --------------------------------------------------------------------------
+
+
+def _answers(*addresses: str) -> Resolver:
+    async def resolve(host: str) -> Sequence[str]:
+        return addresses
+
+    return resolve
+
+
+async def _must_not_resolve(host: str) -> Sequence[str]:
+    raise AssertionError(f"{host} should have been decided without a lookup")
+
+
+@pytest.mark.parametrize(
+    "target",
+    [
+        "http://127-0-0-1.sslip.io:8099",
+        "http://169-254-169-254.sslip.io:80",
+        "socks5://app-10-0-0-1.nip.io:1080",
+    ],
+)
+async def test_a_name_spelling_a_private_address_is_refused_without_a_lookup(target: str) -> None:
+    with pytest.raises(InvalidParam) as excinfo:
+        await vet(target, mode=PUBLIC, resolve=_must_not_resolve)
+    assert _reason(excinfo) == "host_not_public"
+
+
+@pytest.mark.parametrize(
+    "answers",
+    [
+        pytest.param(("127.0.0.1",), id="loopback"),
+        pytest.param(("169.254.169.254",), id="cloud-metadata"),
+        pytest.param(("10.0.0.5",), id="private-range"),
+        pytest.param(("::1",), id="v6-loopback"),
+        pytest.param(("::ffff:127.0.0.1",), id="v4-mapped"),
+        # One bad answer among good ones is enough: which of them gets dialled
+        # is up to the resolver's ordering, not ours.
+        pytest.param((PUBLIC_V4, "127.0.0.1"), id="mixed"),
+    ],
+)
+async def test_a_name_that_resolves_inside_is_refused(answers: tuple[str, ...]) -> None:
+    secret = "http://user:hunter2@innocent.example.com:8080"
+    with pytest.raises(InvalidParam) as excinfo:
+        await vet(secret, mode=PUBLIC, resolve=_answers(*answers))
+    assert _reason(excinfo) == "host_not_public"
+    rendered = str(excinfo.value) + repr(excinfo.value.details)
+    assert "hunter2" not in rendered
+
+
+@pytest.mark.parametrize(
+    ("target", "answers", "expected"),
+    [
+        pytest.param(
+            "http://user:secret@proxy.example.com:8080",
+            (PUBLIC_V4,),
+            f"http://user:secret@{PUBLIC_V4}:8080",
+            id="credentials-and-port-kept",
+        ),
+        pytest.param(
+            "http://proxy.example.com",
+            (PUBLIC_V4,),
+            f"http://{PUBLIC_V4}",
+            id="no-port-stays-no-port",
+        ),
+        pytest.param(
+            "socks5h://proxy.example.com:1080",
+            (PUBLIC_V6,),
+            f"socks5h://[{PUBLIC_V6}]:1080",
+            id="v6-bracketed",
+        ),
+        pytest.param(
+            "socks5://proxy.example.com:1080",
+            (PUBLIC_V6, PUBLIC_V4),
+            f"socks5://{PUBLIC_V4}:1080",
+            id="v4-preferred",
+        ),
+    ],
+)
+async def test_a_public_name_is_pinned_to_the_address_that_was_checked(
+    target: str, answers: tuple[str, ...], expected: str
+) -> None:
+    """Checking a name and then letting the client resolve it again is a race.
+
+    The dial goes to the address that passed, so a second answer - rebinding,
+    or a round-robin record with a private member - never gets asked for.
+    """
+    assert await vet(target, mode=PUBLIC, resolve=_answers(*answers)) == expected
+
+
+async def test_an_https_proxy_keeps_its_name_but_is_still_checked() -> None:
+    """Pinning would break the TLS handshake with the proxy, which is the defence.
+
+    Certificate verification is against the name: a rebind to an internal
+    address cannot produce a certificate for the caller's hostname.
+    """
+    target = "https://proxy.example.com:8443"
+    assert await vet(target, mode=PUBLIC, resolve=_answers(PUBLIC_V4)) == target
+    with pytest.raises(InvalidParam) as excinfo:
+        await vet(target, mode=PUBLIC, resolve=_answers("127.0.0.1"))
+    assert _reason(excinfo) == "host_not_public"
+
+
+async def test_a_name_that_does_not_resolve_is_refused() -> None:
+    async def fails(host: str) -> Sequence[str]:
+        raise OSError("Name or service not known")
+
+    with pytest.raises(InvalidParam) as excinfo:
+        await vet("http://nowhere.example.com:8080", mode=PUBLIC, resolve=fails)
+    assert _reason(excinfo) == "host_unresolvable"
+
+    with pytest.raises(InvalidParam) as excinfo:
+        await vet("http://nowhere.example.com:8080", mode=PUBLIC, resolve=_answers())
+    assert _reason(excinfo) == "host_unresolvable"
+
+
+async def test_a_lookup_that_hangs_is_given_up_on(monkeypatch: pytest.MonkeyPatch) -> None:
+    """It is holding an API request open, and no answer is not a yes."""
+    monkeypatch.setattr(request_proxy, "RESOLVE_TIMEOUT_SECONDS", 0.01)
+
+    async def hangs(host: str) -> Sequence[str]:
+        await asyncio.sleep(10)
+        return (PUBLIC_V4,)
+
+    with pytest.raises(InvalidParam) as excinfo:
+        await vet("http://slow.example.com:8080", mode=PUBLIC, resolve=hangs)
+    assert _reason(excinfo) == "host_unresolvable"
+
+
+async def test_a_public_literal_needs_no_lookup() -> None:
+    target = f"http://{PUBLIC_V4}:8080"
+    assert await vet(target, mode=PUBLIC, resolve=_must_not_resolve) == target
+
+
+async def test_any_mode_neither_resolves_nor_rewrites() -> None:
+    target = "http://127-0-0-1.sslip.io:8099"
+    assert await vet(target, mode=RequestProxyMode.ANY, resolve=_must_not_resolve) == target
+
+
+@pytest.mark.parametrize("mode", list(RequestProxyMode))
+async def test_vet_passes_no_proxy_through(mode: RequestProxyMode) -> None:
+    assert await vet(None, mode=mode, resolve=_must_not_resolve) is None
+
+
+async def test_vet_refuses_by_default_before_any_lookup() -> None:
+    with pytest.raises(InvalidParam) as excinfo:
+        await vet(
+            "http://proxy.example.com:8080", mode=RequestProxyMode.DENY, resolve=_must_not_resolve
+        )
+    assert _reason(excinfo) == "request_proxy_disabled"
+
+
+@pytest.mark.parametrize(
+    "target",
+    [
+        "http://proxy.example.com:8080\\@other.example.com",
+        "http://user\\name@proxy.example.com:8080",
+        "http://proxy.example.com :8080",
+        "http://proxy.exa^mple.com:8080",
+    ],
+)
+def test_an_authority_parsers_could_disagree_about_is_refused(target: str) -> None:
+    """The host checked here must be the host the HTTP client dials.
+
+    Python's parser and the WHATWG one wreq and the browser use do not agree
+    on every character, so anything outside RFC 3986's authority set is refused
+    before either of them is asked what the host is.
+    """
+    with pytest.raises(InvalidParam) as excinfo:
+        normalize(target, mode=PUBLIC)
+    assert _reason(excinfo) == "authority_invalid"
+
+
+def test_percent_encoded_credentials_are_still_fine() -> None:
+    url = "http://user:p%40ss@proxy.example.com:8080"
+    assert normalize(url, mode=PUBLIC) == url

tests/unit/test_urls.py

@@ -652,13 +652,36 @@ def test_allowed_targets(url: str) -> None:
         "printer",
         "db.internal",
         "foo.local",
+        # Wildcard DNS that answers with the address spelt in the name
+        # (GHSA-q3h8-73xx-gwqx). The dotted spelling was always caught by the
+        # numeric-label rule; the hyphenated one was not.
+        "127-0-0-1.sslip.io",
+        "169-254-169-254.sslip.io",
+        "app-10-0-0-1.nip.io",
+        "customer1-app-192-168-001-001.nip.io",
+        "--1.sslip.io",
+        "fe80--1.sslip.io",
     ],
 )
 def test_is_private_host(host: str) -> None:
     assert is_private_host(host) is True
 
 
-@pytest.mark.parametrize("host", ["www.douyin.com", "v.douyin.com", "www.tiktok.com", "8.8.8.8"])
+@pytest.mark.parametrize(
+    "host",
+    [
+        "www.douyin.com",
+        "v.douyin.com",
+        "www.tiktok.com",
+        "8.8.8.8",
+        # An address in the name is only a problem when the address is.
+        "8-8-8-8.sslip.io",
+        "ec2-54-12-34-56.compute-1.amazonaws.com",
+        "v26-web.douyinvod.com",
+        "p16-sign-va.tiktokcdn.com",
+        "xn--fiqs8s.cn",
+    ],
+)
 def test_is_private_host_allows_public_names(host: str) -> None:
     assert is_private_host(host) is False