Files
Evil0ctal ff838ce0b1 fix: explain a refused request, and stop the query contradicting the UA
Two things, both about a request describing itself honestly.

**A failure can now be explained.** Which is the case the feature exists for:
"it worked" rarely needs the request spelled out, and "it was refused" always
does. The explanation could not travel inside the error - errors are serialized
into the task row and rendered to every reader, while this contains a cookie
jar gated on identity:manage - so `fetch` writes it onto the context the caller
already owns, on the way past, before any raise. The worker stores it beside
the error and `_view_payload` emits it through the same `_visible_meta` gate it
uses on success. That gate is the part worth pinning: nothing else about a
failure is privileged, so a `result_meta` emitted beside an error without it
would read as entirely unremarkable. Verified live both ways - a read key sees
the failure and not the jar.

**The query string agreed with nothing.** Douyin and TikTok both echo the
screen, language, OS and browser version back in every query, right beside a
User-Agent carrying the same facts, and ours was a constant: every request
claimed Chrome 130 on a zh-CN Windows box while its own header said Chrome 146
and its Accept-Language said en-US. No browser produces that. `ClientProfile`
has documented itself as "derived from the identity's fingerprint by the
caller" since it was written; nobody derived it.

Now each adapter owns a `profile_for`, because the two platforms report the
same facts in different vocabularies - Douyin the marketing name and the real
version, TikTok the navigator product strings. Nothing is invented: the OS
table is lifted from Douyin's own chunk-43693 bundle, the browser version is
the substring its detector takes out of the User-Agent, and TikTok's
`5.0 (Windows)` / `os=windows` are reproduced exactly from a live capture. A
field the User-Agent does not state keeps its default - a value nobody can
vouch for is worth no more than one that is simply wrong.

The signer had the same bug one layer down: it was handed only a User-Agent, so
A-Bogus carried the fallback 1920x1080 Win32 geometry inside every signature
whatever the identity was - contradicting the screen_width the same request
sent in plain sight. For this pool the derived value is byte-identical, so
nothing changes today and it is correct for any identity minted later.

Measured after the change: Douyin returns the video, TikTok returns the
profile, and the signing self-check still reports douyin: match, tiktok: match
against the real browser.
2026-09-09 22:43:43 -07:00

143 lines
5.4 KiB
Python

"""Who may read the description of a request this instance made.
`?explain=true` hands back the call as it went out: the signed URL, the headers
and the identity's cookie jar. That is a credential, and it lands inside a task
result that an ordinary read key is otherwise entitled to fetch - reading a
result costs exactly the scope that created it, which is the rule that stops a
low-scope key becoming a way around every other scope.
So the explanation is stripped on the way out for anyone who could not have
revealed that jar directly. This file pins that, because the failure mode is
silent: the wrong answer here is a working feature that also hands a session
cookie to every key on the instance.
"""
from __future__ import annotations
import uuid
from datetime import UTC, datetime
import pytest
from dtk.api.deps import Principal
from dtk.api.routes.tasks import _PRIVILEGED_META, _view_payload, _visible_meta
from dtk.core.types import Language, Scope, TaskState, UserRole
from dtk.services.tasks import TaskView
META = {
"endpoint": "douyin.content_detail",
"cached": False,
"explain": {"cookie_header": "sessionid=not-a-real-value", "url": "https://..."},
}
def principal(*scopes: Scope, role: UserRole = UserRole.OPERATOR) -> Principal:
"""An API key bounded by `scopes` - i.e. one that is genuinely scoped.
`api_key_id` has to be set: a principal with none is a console session,
which `permits` treats as bounded by its role rather than by scopes, and
the whole point here is the key that IS bounded.
"""
return Principal(
user_id=uuid.uuid4(),
role=role,
scopes=frozenset(scopes),
api_key_id=uuid.uuid4(),
rate_limit_per_min=None,
)
def test_a_read_key_does_not_see_the_explanation() -> None:
"""The case this exists for: douyin:read created the task, and may read the
result, and must not be handed the jar that served it."""
visible = _visible_meta(META, principal(Scope.DOUYIN_READ))
assert "explain" not in visible
assert visible["endpoint"] == "douyin.content_detail", "the rest must survive"
def test_identity_manage_sees_it() -> None:
assert "explain" in _visible_meta(META, principal(Scope.IDENTITY_MANAGE))
def test_admin_scope_sees_it() -> None:
assert "explain" in _visible_meta(META, principal(Scope.ADMIN))
def test_no_principal_strips_it() -> None:
"""The default has to be the safe one.
Any caller that has not thought about the question gets nothing - the SSE
stream takes this path deliberately.
"""
assert "explain" not in _visible_meta(META, None)
def test_meta_without_an_explanation_is_untouched() -> None:
plain = {"endpoint": "x", "cached": True}
assert _visible_meta(plain, principal(Scope.DOUYIN_READ)) == plain
@pytest.mark.parametrize("key", sorted(_PRIVILEGED_META))
def test_every_privileged_key_is_gated_on_a_real_scope(key: str) -> None:
"""A typo in the table would silently disable the gate for that key."""
assert isinstance(_PRIVILEGED_META[key], Scope)
# --------------------------------------------------------------------------
# The same gate on a failed task
# --------------------------------------------------------------------------
#
# A refused request is the case `explain` exists for, so the explanation is
# stored on a failed task too. That added a second path to the same credential,
# and it is the easier one to get wrong: nothing else about a failure is
# privileged, so a `result_meta` emitted beside the error without going through
# `_visible_meta` would look entirely unremarkable in review.
def _failed(result: dict[str, object] | None) -> TaskView:
return TaskView(
id=uuid.uuid4(),
state=TaskState.FAILED,
endpoint="douyin.content_detail",
result=result,
error={"code": "NOT_FOUND", "message": "gone"},
created_at=datetime(2026, 9, 10, tzinfo=UTC),
finished_at=datetime(2026, 9, 10, tzinfo=UTC),
)
STORED = {"data": None, "meta": dict(META)}
def test_a_failed_task_carries_the_explanation_for_a_reader_who_may_see_it() -> None:
payload = _view_payload(
_failed(STORED), Language.EN, principal=principal(Scope.IDENTITY_MANAGE)
)
assert payload["error"]["code"] == "NOT_FOUND"
assert payload["result_meta"]["explain"]["cookie_header"]
def test_a_failed_task_hides_it_from_a_read_key() -> None:
"""The whole point: a refusal must not become the cheap way to a jar."""
payload = _view_payload(_failed(STORED), Language.EN, principal=principal(Scope.DOUYIN_READ))
assert payload["error"]["code"] == "NOT_FOUND"
assert "explain" not in payload.get("result_meta", {})
def test_a_failed_task_hides_it_from_a_caller_nobody_vouched_for() -> None:
"""None strips everything. The SSE stream deliberately passes None."""
payload = _view_payload(_failed(STORED), Language.EN)
assert "explain" not in payload.get("result_meta", {})
def test_a_failure_nobody_asked_to_explain_gains_no_metadata_key() -> None:
payload = _view_payload(_failed(None), Language.EN, principal=principal(Scope.ADMIN))
assert "result_meta" not in payload
assert payload["error"]["code"] == "NOT_FOUND"
def test_a_failed_task_still_has_no_data() -> None:
"""`data` stays absent so a reader cannot mistake a failure for an answer."""
payload = _view_payload(_failed(STORED), Language.EN, principal=principal(Scope.ADMIN))
assert "data" not in payload