# REST API guide

> **[Douyin_TikTok_Download_API](https://github.com/Evil0ctal/Douyin_TikTok_Download_API)** —
> a self-hosted Douyin and TikTok data API: REST, MCP and a web console, with
> an identity pool that maintains itself.
> [All docs](../README.md) · [中文](../zh/11-api.md)

This page explains how to *call* the HTTP API from a program: how to authenticate, how the uniform envelope is shaped, how the asynchronous task model works, how to page, and which errors your code has to handle. After reading it you should be able to write a client that submits work, collects results and backs off correctly when the instance pushes back.

It complements the generated reference rather than repeating it. The reference — `/swagger`, `/redoc` and `/openapi.json` — lists every endpoint, every parameter and every constraint, and it is generated from the same code that serves the requests, so it is never out of date. This page explains the parts a list of endpoints cannot: what a `202` means, when to retry, and what a parameter costs you.

## What this guide covers

| Where | What it gives you | Credentials |
| --- | --- | --- |
| `/swagger` | Swagger UI over the full endpoint list, with "Try it out" | None to view; a key to call |
| `/redoc` | The same document rendered for reading | None to view |
| `/openapi.json` | The OpenAPI document itself, for code generation | None |
| `/docs` | The console's own API reference page, themed and inside the shell | None to view |
| This page | How to use all of the above from a program | — |

`/docs` needs no session either: signed out it renders inside the console shell in its signed-out state and reads the same public `/openapi.json` everything else here reads.

All four honour `?lang=` (see [Response language](#response-language)), so `/swagger?lang=zh` renders the Chinese document.

## Base URL and versioning

The compose stack publishes the API on `127.0.0.1:8000` by default (`DTK_BIND_HOST` and `DTK_BIND_PORT` in `.env` change that — see [Installation and deployment](./02-installation.md)). If you have no instance and no key yet, [Quick start](./01-quickstart.md) produces both in about ten minutes. Every example below uses:

```bash
export DTK_BASE_URL=http://127.0.0.1:8000
export DTK_API_KEY=dtk_0a1b2c3d4e5f_REPLACE_WITH_YOUR_KEY
```

Path layout:

| Prefix | Contains |
| --- | --- |
| `/api/v1/…` | Everything a program calls: content, tasks, tools, archive, downloads, admin |
| `/api/setup/…` | First-run bootstrap only; anonymous by necessity and gated by the one-time setup token, and can never be added to `api.public_endpoints` |
| `/healthz`, `/readyz` | Process probes, outside the versioned surface and outside the envelope |
| `/mcp/` | The MCP endpoint — keep the trailing slash, because `/mcp` answers `307 Temporary Redirect` to it. See [MCP and AI agents](./12-mcp.md) |

The `v1` in the path is the version of this contract. Two things inside it are also treated as contract and are append-only, never renamed: the **error codes** and the **envelope keys**. New fields may appear in `data` and `meta` over time, so parse defensively — read the keys you need, ignore the ones you do not.

The running version is reported by `GET /api/v1/system/status`, which returns `version`, `commit`, `uptime_seconds`, `settings_version`, component health, the identity pool census and storage use. It requires a credential, because naming component versions and row counts is more than an anonymous probe needs.

## Authentication

Every endpoint requires a credential unless an operator has explicitly opened it. A handful of routes are anonymous by construction and cannot be closed — login, logout, the two `/api/setup` routes and `GET /api/v1/ios/shortcut`, which the Shortcut asks for before it has anywhere to put an API key. They are listed in [Routes that were never closed](./09-users-and-api-keys.md#routes-that-were-never-closed); `/healthz` and `/readyz` are unauthenticated too and are not in the API document at all. A program authenticates with an API key, in **either** of two header forms — they are equivalent, and the `Authorization` header is read first when both are present:

```bash
curl -sS "$DTK_BASE_URL/api/v1/auth/me" -H "Authorization: Bearer $DTK_API_KEY"
curl -sS "$DTK_BASE_URL/api/v1/auth/me" -H "X-API-Key: $DTK_API_KEY"
```

`GET /api/v1/auth/me` is the cheapest way to check that a key is live and see what it may do: it returns the account, the role, the scopes the credential carries, `via` (`api_key` or `session`) and the key's own `rate_limit_per_min`.

A third form exists and is not for programs: the console signs in with `POST /api/v1/auth/login` and gets a `dtk_session` cookie. Use a key instead — a key carries scopes, can be revoked on its own, and does not expire with a browser session.

### Keys

A key looks like `dtk_<12 hex characters>_<random>`. It is generated by `POST /api/v1/admin/api-keys` and shown **exactly once**, in that response; only a prefix (for display) and a SHA-256 digest (for verification) are stored, so nobody — administrator included — can read it back. Revocation takes effect on the next request, since authentication reads the row every time. See [Users and API keys](./09-users-and-api-keys.md).

### Scopes

| Scope | Reaches |
| --- | --- |
| `douyin:read` | Douyin content endpoints, and `/parse` for a Douyin link |
| `tiktok:read` | TikTok content endpoints, and `/parse` for a TikTok link |
| `archive:read` | `GET /api/v1/archive…` — what this instance has already stored |
| `archive:export` | `GET /api/v1/archive/export` — the whole collection in one call |
| `media:read` | Download records and stored files |
| `media:write` | Starting, pinning and cancelling a download |
| `identity:manage` | The identity pool, and the `identity` and `explain` request parameters |
| `admin` | Everything |

Two rules are worth knowing because they surprise people:

- **A key is bounded by its scopes even when its owner is an administrator.** On a self-hosted instance almost every key belongs to the admin account; a plain `douyin:read` key still cannot reach identity management or `archive:export`.
- **Reading a task's result costs the same scope that creating it did.** `GET /api/v1/tasks/{task_id}` checks the scope for the endpoint the task was submitted to, so a low-scope key cannot read a high-scope result out of a task id it was handed.

Some operations additionally require a **role** (`demo` < `viewer` < `operator` < `admin`) on top of a scope — `identity` and `explain` both need at least `operator`. Roles apply to the account the credential belongs to. `?proxy=` is not one of them: it carries no scope or role check at all and is gated solely by the `security.request_proxy` setting, as [`?proxy=<url>`](#proxyurl) below describes.

### Opened endpoints

An operator can list specific endpoints in `api.public_endpoints` (written as `GET /api/v1/{platform}/video`, exactly as the API document spells the path) and those are served without a credential. Admin, auth and setup paths can never be opened, whatever the setting says. An anonymous caller runs with `douyin:read` and `tiktok:read` only and is metered by client address rather than by key.

Sending a **bad** credential is not the same as sending none: a key we rejected gets `UNAUTHENTICATED` even on an opened endpoint, rather than being quietly downgraded to anonymous. That turns "your key expired" into an error you can see instead of "your key works but sees less".

## The response envelope

Every JSON response — success or failure — has the same four top-level keys.

```json
{
  "success": true,
  "data": {},
  "error": null,
  "meta": { "request_id": "b6f0f2c6-4a5f-4a0e-9a1f-2b7c2b6f1f21" }
}
```

| Key | Meaning |
| --- | --- |
| `success` | `true` or `false`. Branch on this, not on the HTTP status alone |
| `data` | The endpoint's payload. Always `null` when `success` is `false` |
| `error` | `null` on success. Otherwise `{code, message}` plus optional `retry_after` and `details` |
| `meta` | Always carries `request_id`. May carry `cached`, `duration_ms`, `cursor` and endpoint-specific extras |

**Branch on `error.code`, never on `error.message`.** The code is a stable enum that is never translated; the message is rendered in the caller's language and is written for a human.

### A worked success

Submitting a link returns `202` with a task id:

```bash
curl -sS -X POST "$DTK_BASE_URL/api/v1/parse" \
  -H "X-API-Key: $DTK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://www.douyin.com/video/7123456789012345678"}'
```

```json
{
  "success": true,
  "data": { "task_id": "0f2f1b7c-3f9e-4b8a-9a11-0a6d2b0e51c3", "state": "queued" },
  "error": null,
  "meta": { "request_id": "b6f0f2c6-4a5f-4a0e-9a1f-2b7c2b6f1f21" }
}
```

### A worked failure

Send a URL the allowlist does not admit and the same envelope carries the error, with HTTP `400`:

```bash
curl -sS -X POST "$DTK_BASE_URL/api/v1/parse" \
  -H "X-API-Key: $DTK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://evil.example/video/1"}'
```

```json
{
  "success": false,
  "data": null,
  "error": {
    "code": "INVALID_URL",
    "message": "The URL was not recognized as a supported Douyin or TikTok link.",
    "details": { "reason": "host_not_allowed" }
  },
  "meta": { "request_id": "1d5b1a7e-90c7-4f2e-9b3c-6b2f0f9a8d44" }
}
```

(The message is English here because the command names no language and the instance default is `en`; add `?lang=zh` and it renders in Chinese.)

`details` is keyed by name and its shape varies by code — the field that failed, the endpoint that refused, the id that was not found. It never contains a credential.

### Correlation

Every response carries two headers you can log:

| Header | Meaning |
| --- | --- |
| `X-Request-ID` | The correlation id, also written into the request log |
| `X-Response-Time-Ms` | How long this process spent on the request |

Quote `X-Request-ID` in a bug report; the operator can find the matching row on the console's Logs page. `meta.request_id` normally carries the same value — with one exception worth knowing: a result rendered from a finished task (a `?wait=` call that completed) carries the **worker's** fetch request id in `meta.request_id`, because the worker's own metadata is merged in there. The header is always the HTTP correlation id.

### Content type

POST bodies must be sent as `application/json`. Plain `curl -d '{…}'` sends `application/x-www-form-urlencoded` and the body will not parse — every example here passes `-H 'Content-Type: application/json'` for that reason. A body that fails to parse comes back as `INVALID_PARAM` with the field path and the reason in `details.fields`; the offending value is deliberately never quoted back to you, because for an unparseable body that value is the whole request.

A request body is capped at **1 MiB**. Over that you get HTTP `413` with code `INVALID_PARAM` and `details.limit_bytes`.

## The asynchronous model

Fetching from a platform costs a real upstream call on a real pooled identity, and it can take seconds. So **the data endpoints are asynchronous by default**: they validate, queue the work and answer `202` immediately with a task id. Nothing in the API container ever talks to a platform.

```
POST /api/v1/parse            -> 202 {"task_id": "...", "state": "queued"}
GET  /api/v1/tasks/{task_id}  -> 200 {"state": "done", "data": {...}}
```

There are three ways to get the result. They differ only in who does the waiting.

### 1. Poll the task

`GET /api/v1/tasks/{task_id}` until `state` is `done` or `failed`. It is safe to poll and safe to call twice: the answer does not change until the task does. The payload:

| Field | Present | Meaning |
| --- | --- | --- |
| `task_id` | always | The id you polled |
| `state` | always | `queued`, `running`, `done` or `failed` |
| `endpoint` | always | What was submitted, e.g. `parse` or `douyin.content_detail` |
| `created_at`, `finished_at` | always | ISO-8601; `finished_at` is `null` until it settles |
| `data` | when `done` | **The result payload, nested inside the envelope's `data`** |
| `result_meta` | when finished | `cached`, `duration_ms`, `endpoint`, `platform`, `cursor`, and gated extras |
| `error` | when `failed` | `{code, message, retryable}` plus optional `retry_after` and `details` |

Note the nesting: on a finished task the result is at `body.data.data`, not `body.data`. And note that **a failed task is still HTTP 200 with `success: true`** — the HTTP call to read the task succeeded; the failure is inside it, in `data.state` and `data.error`. This is the opposite of the `?wait=` path below, and the difference trips people up.

A task's stored `error` object carries an explicit `retryable` boolean, so an agent reading a result does not need to carry the non-retryable table around.

**Results expire.** The payload is kept for `retention.task_result_hours` (default **24**). After that the row survives with its state and timestamps but the payload is blanked, and the lookup answers `TASK_NOT_FOUND`. That means submit the work again — not keep polling.

### 2. Let the server wait — `?wait=`

Add `?wait=10` to any submitting endpoint and the connection is held until the task settles, up to that many seconds. This is how you make a call synchronous, and it is meant for clients that cannot poll at all: an iOS Shortcut, a shell one-liner, a spreadsheet.

| Outcome | Status | Body |
| --- | --- | --- |
| Finished in time | `200` | The result in `data`, exactly as the task endpoint would have returned it |
| The task failed while you waited | its own status (e.g. `503`) | A normal failure envelope with that error's code |
| Not finished in time | `202` | `{"task_id": "...", "state": "running"}` |
| Above the instance ceiling | `400` | `INVALID_PARAM`, with `details.max` |
| Omitted, or `0` | `202` | Returns at once, same as no wait |
| Negative | `400` | `INVALID_PARAM` |

**A `202` after the wait elapses is not an error and nothing was lost.** The work is still running; the same task id fetches it a moment later. A client that treats the early `202` as a failure will resubmit work that was already queued, spend identities twice and get the same answer.

The ceiling is `api.max_wait_seconds`, default **30 seconds**. A value above it is *rejected* rather than silently shortened — deliberately, because a caller that asked to block for five minutes has to learn that it cannot, or it will read the early `202` as a failure. The live ceiling is published on the parameter itself in the OpenAPI document, as `maximum`.

Nothing changes internally: the work goes through the same queue either way. `?wait=` only decides who holds the connection. It is the wrong choice for a batch job or anything with its own event loop — you are holding an HTTP connection open to save yourself a loop.

### 3. Get called back — `callback_url`

`POST /api/v1/parse` and `POST /api/v1/tasks/batch` accept a `callback_url` in the body. When the task finishes the instance POSTs a notification there, so nothing polls and nothing blocks.

This is an outbound request to an address the caller chooses — SSRF in its plainest form — so it is fenced:

- It is refused unless an administrator has set `security.enable_task_webhook` (default **off**). Asking for one on an instance where it is off is `INVALID_PARAM` on `callback_url`.
- The URL must be `https`, and its host must not be loopback, private or link-local. Checked at submission **and again at delivery**, because the setting can be turned off and a queued task can outlive the moment it was accepted.
- Redirects are not followed and certificates are verified.

The notification says what happened, not what was found — the result can be megabytes, and posting it to a third party is a data-flow decision nobody makes by typing a URL into a field:

```json
{
  "event": "task.completed",
  "task_id": "0f2f1b7c-3f9e-4b8a-9a11-0a6d2b0e51c3",
  "endpoint": "parse",
  "state": "done",
  "sent_at": "2026-09-10T04:15:12.883921+00:00"
}
```

A failure sends `"event": "task.failed"` and adds `error` with the code and a truncated message. Collect the actual result with `GET /api/v1/tasks/{task_id}`.

| Property | Value |
| --- | --- |
| `X-Dtk-Event` header | `task.completed` or `task.failed` |
| `X-Dtk-Signature` header | `sha256=<hex>`, HMAC-SHA256 over the exact bytes sent, when `security.webhook_secret` is set |
| Timeout | 10 seconds per attempt |
| Attempts | 3, with 2s then 8s backoff |
| Gives up early on | any `4xx` except `429` |

Verify the signature over the raw body bytes, not over a re-serialization of the parsed object — key order and separators would differ and an otherwise correct check would fail. Without a secret configured a receiver cannot tell a real notification from anyone who guessed the URL.

Delivery never affects the task. A webhook endpoint that is down, slow or hostile cannot turn a successful fetch into a failed task; every delivery failure is logged and swallowed.

### The event stream

`GET /api/v1/tasks/{task_id}/events` is a server-sent event stream, which the console uses instead of polling many rows at once.

| Event | Payload |
| --- | --- |
| `state` | The task payload without the result, sent whenever the state changes |
| `result` | The full task payload once it is `done` or `failed` |
| `end` | `{task_id}`, immediately after `result` |
| `timeout` | `{task_id, state}` when the deadline passes first |
| `error` | `{code: "TASK_NOT_FOUND", message}` if the row disappears |

`?timeout=` bounds the stream between 1 and **300** seconds and defaults to 300; a comment frame goes out every ~15 seconds so an intermediate proxy does not drop a quiet connection. Authorization happens before the stream opens. The privileged `explain` block is **not** sent on this stream — fetch it once over the ordinary `GET`, where the scope check sits beside the read.

### Batch submission

`POST /api/v1/tasks/batch` takes up to **50** items and always answers `202`. Submission is batched; tracking is not — each item gets its own task id and its own fate, so one bad link cannot smear the whole request into a single error:

```json
{
  "success": true,
  "data": {
    "items": [
      { "url": "https://v.douyin.com/abc123/", "task_id": "0f2f…", "state": "queued" },
      { "url": "https://evil.example/x", "task_id": null,
        "error": { "code": "INVALID_URL",
                   "message": "The URL was not recognized as a supported Douyin or TikTok link.",
                   "details": { "reason": "host_not_allowed" } } }
    ],
    "submitted": 1,
    "rejected": 1
  },
  "error": null,
  "meta": { "request_id": "…" }
}
```

Per-item errors are rendered in the language you asked for, like every other error this API returns. `/tasks/batch` takes no `?wait=`: poll each task id, or supply a `callback_url`.

### When the queue is full

Submission is refused rather than queued once the Redis queue reaches `sched.queue_max` (default **500**): `503` with code `QUEUE_FULL`, a `Retry-After` header, and `details.queued` naming the current depth. `retry_after` is `sched.max_wait_seconds` (default **10**). A caller left hanging is worse off than one that is told to come back.

Two exceptions, both deliberate: work that joins an *already queued* identical task is admitted (it adds nothing to the backlog), and operator-triggered maintenance jobs are exempt — refusing to run the self check because the queue is deep would withhold the tool exactly when it is wanted. Media downloads are **not** exempt, because one operator's bulk download would otherwise starve every read on the instance.

## Pagination

List endpoints page with an **opaque cursor**. You send back what the previous page gave you and nothing else.

| Parameter | Where | Default | Ceiling |
| --- | --- | --- | --- |
| `cursor` | Platform lists, archive list | first page | 512 characters |
| `count` | Platform lists | 20 | 50 |
| `limit` | `GET /api/v1/archive` | 50 | 200 |

A platform list's result carries the cursor in two places, because two readers want it in two shapes:

```json
{
  "success": true,
  "data": { "items": [], "cursor": "1757462400000", "has_more": true },
  "error": null,
  "meta": {
    "request_id": "…",
    "cached": false,
    "duration_ms": 812,
    "cursor": { "next": "1757462400000", "has_more": true },
    "task_id": "…",
    "endpoint": "douyin.author_posts",
    "platform": "douyin"
  }
}
```

When polling instead of waiting, the same values are at `data.data.cursor` and `data.result_meta.cursor.next`.

Rules:

- **The cursor is opaque.** Douyin pages by a millisecond `max_cursor` timestamp and TikTok by an offset; both are stringified into one field so a caller never has to know which. Do not parse it, do not increment it, do not construct one.
- **No cursor, or `has_more: false`, means the last page.** Stop there rather than sending the previous cursor again.
- A cursor belongs to the query that produced it. Changing `count` or any filter mid-walk and reusing an old cursor is not defined behaviour.

A page above the ceiling is refused by validation rather than silently trimmed for `count` (the OpenAPI schema declares `maximum: 50`). A caller that wants more should page; one that wants everything at once is a caller to slow down.

Not everything uses cursors. `GET /api/v1/downloads` and the admin listings use `limit` + `offset` and return a `total` (default 100, ceiling 500) because they read the local database; `GET /api/v1/admin/logs/requests` uses a bounded `minutes` window plus `limit`, so there is no combination of parameters that reaches the log table without a window.

## Response language

Two error codes are the same in every language; the sentence beside them is not. Resolution order for one request:

1. `?lang=` query parameter — `en` or `zh`
2. `Accept-Language` header — matched on the primary subtag, so `zh-CN`, `zh-Hans` and `zh-TW` all resolve to `zh`; `*` resolves to the configured default
3. `api.default_language` (default `en`)

```bash
curl -sS "$DTK_BASE_URL/api/v1/parse?lang=zh" -X POST \
  -H "X-API-Key: $DTK_API_KEY" -H 'Content-Type: application/json' \
  -d '{"url": "not a link"}'
```

An unsupported `?lang=` value is *ignored* rather than rejected, and negotiation continues with the header — so `?lang=fr` still honours `Accept-Language: zh`. Anything that resolves to neither language falls back to English rather than being half-served in a language this build does not have.

This applies to the OpenAPI document too (`/openapi.json?lang=zh`, `/swagger?lang=zh`), and to a task's stored error: the worker stores the code and its arguments, and the sentence is built when you read it, so one failed task can answer a Chinese console and an English agent correctly.

## Errors

Every failure is the same envelope with `success: false`. Branch on `error.code`.

| Code | HTTP | Retryable | What it means |
| --- | --- | --- | --- |
| `INVALID_URL` | 400 | no | Not a supported Douyin or TikTok link, or the host is off the allowlist |
| `UNSUPPORTED_CONTENT` | 400 | no | This platform does not serve that operation; `details.supported` names the ones that do |
| `INVALID_PARAM` | 400 | no | A parameter is wrong; `details` names the field. Also used for a body over the 1 MiB ceiling (with HTTP 413) |
| `UNAUTHENTICATED` | 401 | no | No credential, or one that was rejected |
| `FORBIDDEN_SCOPE` | 403 | no | The credential is not permitted; `details` names both halves - `required_scopes` or `required_roles`, and `have_role` / `have_scopes` / `via` |
| `CONTENT_PRIVATE` | 403 | no | Private, or removed by its author |
| `NOT_FOUND` | 404 | no | No such resource |
| `TASK_NOT_FOUND` | 404 | **yes** | No such task, or its result has expired. Retryable in the sense the flag means — but no retry brings an expired result back, so resubmit the work |
| `METHOD_NOT_ALLOWED` | 405 | no | Wrong method for this path; the `Allow` header lists the right ones |
| `SETUP_ALREADY_DONE` | 409 | no | Setup has already run on this instance |
| `CANCELLED` | 409 | no | Somebody cancelled the task deliberately |
| `UNSUPPORTED_MEDIA_TYPE` | 415 | no | The body arrived in a media type this endpoint does not read |
| `RATE_LIMITED` | 429 | **yes** | Too many requests; honour `retry_after` |
| `SETUP_TOKEN_INVALID` | 403 | no | The first-run token is wrong or spent |
| `NOT_CONFIGURED` | 501 | no | This deployment never had that optional component |
| `INTERNAL` | 500 | **yes** | Unexpected; quote `X-Request-ID` in a bug report |
| `UPSTREAM_RISK_CONTROL` | 502 | **yes** | The platform flagged the request; the identity is cooling down |
| `UPSTREAM_CHANGED` | 502 | no | The platform's response no longer matches the parser; `details.path` names the field. Report it |
| `SIGNING_FAILED` | 502 | **yes** | Signing failed; the algorithm may be out of date |
| `IDENTITY_POOL_EXHAUSTED` | 503 | **yes** | No identity free; `retry_after` estimates the recovery |
| `ENDPOINT_CIRCUIT_OPEN` | 503 | **yes** | This endpoint is paused after repeated failures |
| `QUEUE_FULL` | 503 | **yes** | The queue is at its ceiling |
| `DOWNLOADER_UNAVAILABLE` | 503 | **yes** | The media sidecar is running somewhere and did not answer |

`NOT_CONFIGURED` and `DOWNLOADER_UNAVAILABLE` look similar and are not: the first means this install never had the component, so no amount of waiting will help; the second means a service that exists is down.

### `retry_after`

Codes that clear on their own carry `retry_after` (seconds) inside `error`, **and** the same value in the standard `Retry-After` response header. Wait at least that long. `RATE_LIMITED`, `QUEUE_FULL`, `IDENTITY_POOL_EXHAUSTED`, `ENDPOINT_CIRCUIT_OPEN` and `UPSTREAM_RISK_CONTROL` are the ones you will actually see.

### `retryable`

The `retryable` boolean appears on a **task's stored error** (`GET /api/v1/tasks/{task_id}` → `data.error.retryable`) and in the OpenAPI error schema. A top-level envelope error carries the code, and the table above is what that code means. A client that branches on the code needs nothing else; an agent reading a task result gets the flag spelled out so it does not have to carry the table.

### The codes a client must handle

Anything else can reasonably be treated as "log it and stop", but these four decisions have to be in your code:

1. **Back off and retry**: `RATE_LIMITED`, `QUEUE_FULL`, `IDENTITY_POOL_EXHAUSTED`, `ENDPOINT_CIRCUIT_OPEN`, `UPSTREAM_RISK_CONTROL`, `SIGNING_FAILED`, `INTERNAL`, `DOWNLOADER_UNAVAILABLE`. Honour `retry_after` where present; use exponential backoff where it is not.
2. **Never retry the same request**: `INVALID_URL`, `UNSUPPORTED_CONTENT`, `INVALID_PARAM`, `NOT_FOUND`, `CONTENT_PRIVATE`, `METHOD_NOT_ALLOWED`, `UNSUPPORTED_MEDIA_TYPE`, `CANCELLED`, `NOT_CONFIGURED`, `UPSTREAM_CHANGED`. A loop on one of these burns identities to be told the same thing.
3. **Fix the credential**: `UNAUTHENTICATED`, `FORBIDDEN_SCOPE`. `details` carries both sides: `required_scopes` or `required_roles` for what the endpoint wants, and `have_role`, `have_scopes` and `via` for what you sent. `via` is the one to read first — it says whether this caller is judged by its scopes (`api_key`) or by its role (`session`), and an administrator's key is still bounded by its scopes.
4. **Resubmit the work**: `TASK_NOT_FOUND`. The result window has passed; the task id is dead.

## Rate limits

A fixed one-minute window, per credential. It is abuse protection, not metering and not billing: the only purpose is stopping one runaway script from draining the identity pool.

| Header | Meaning |
| --- | --- |
| `X-RateLimit-Limit` | Requests allowed in the current window |
| `X-RateLimit-Remaining` | How many are left |
| `X-RateLimit-Reset` | Unix timestamp (seconds) when the window rolls over |

The limit is the key's own `rate_limit` when one was set at creation (between 1 and 100,000), otherwise `api.default_rate_limit_per_min` (default **120**). If that instance default is itself set to 0 or less, the counter is off for every caller without a key-specific limit and the three headers are absent. Exceeding the limit is `429` with code `RATE_LIMITED`, `details.limit`, `error.retry_after` and a `Retry-After` header.

The bucket is keyed by API key when you send one, by user for a console session, and by client address for an anonymous caller on an opened endpoint. That last one has a caveat worth knowing before you rely on it: behind Docker's published-port userland proxy every request appears to come from the bridge gateway, so the anonymous bucket degrades to one shared bucket for the whole internet unless the operator declares a reverse proxy with `DTK_FORWARDED_ALLOW_IPS`. Degrading toward stricter is the right direction for an abuse counter, but it means one busy anonymous caller can lock others out. See [Security](./15-security.md).

Read the headers and slow down before you hit `429`, rather than probing for the ceiling.

## Repeat calls: coalescing, the cache and `?refresh=`

Two separate mechanisms make a repeated request cheap, and both are on by default.

**Coalescing.** Two identical requests arriving close together are joined onto one task rather than run twice — a hundred callers asking for the same video cost the pool one upstream request. The claim lives for 90 seconds. A task that has already *failed* is never joined: replaying a failure for the rest of the window would hide a retry that might well succeed.

**The response cache.** A shaped answer is cached for a TTL that depends on what was asked for:

| Setting | Default | Applies to |
| --- | --- | --- |
| `cache.content_ttl` | 1800s (30 min) | One post |
| `cache.author_ttl` | 900s (15 min) | An author profile |
| `cache.list_ttl` | 300s (5 min) | Anything paged |

`meta.cached` tells you which answer you got. Entries expire on their own, and Redis is capped below its container limit so the cache cannot grow without bound.

**`?refresh=true` turns off both.** It ignores any cached or in-flight answer and asks upstream again. The fresh answer is still written to the cache — "do not read the cache" and "do not keep this" are different requests, and only the first one was asked for. It costs an identity and a real upstream request, so it is for checking whether something changed, not for every call.

Both mechanisms are why two spellings of the same request coalesce: `?url=https://www.douyin.com/video/7123…` and `?aweme_id=7123…` extract to the same parameters and get the same task id.

## `?include_raw=`

Adds the platform's own untouched payload beside the normalized one. Off by default, and worth understanding before you turn it on:

- On a single post it is large — a single post's raw payload runs to hundreds of kilobytes.
- On a **page** it is per item, so it multiplies the response and everything that stores it.
- It is part of the cache key, so a raw and a non-raw request for the same thing are two cache entries and two upstream calls.

On `POST /api/v1/parse` and each item of `POST /api/v1/tasks/batch` it is a body field; on the `GET` endpoints it is a query parameter.

## `?identity=` and `?proxy=`

Both change *how* a request is made rather than what it asks for, and both are gated.

### `?identity=<uuid>`

Sends the request as one named identity and no other. The case it exists for is a cookie jar you imported from your own logged-in browser: the content is visible to that session and to no other, so substituting a different identity would not degrade the answer, it would change the question.

- Requires the `identity:manage` scope **and** at least the `operator` role.
- The id is checked at submission for existence, retirement and platform, so a mistyped uuid is an immediate `400` or `404` naming the field rather than a task that queues, runs and fails. A retired identity is refused explicitly — retirement wipes the ciphertext, so there is no jar left to sign with.
- A pinned request **neither reads nor writes the response cache** and is never joined to an unpinned task.
- It gets **one** transport attempt instead of three. Retrying is only worth anything because the next attempt lands on a different identity behind a different exit; pinned, three attempts would just drain one identity's token bucket.
- The identity id is echoed back in the result metadata as `identity_id` — only when you named it.

See [Identities and proxies](./06-identities-and-proxies.md).

### `?proxy=<url>`

Sends the upstream request through an egress **you** supply, replacing the identity's own exit.

- Refused unless an administrator sets `security.request_proxy` to `public` or `any`. The default is `deny`, and an unrecognised value is treated as `deny` — a typo in configuration must not be the thing that opens a network.
- `public` accepts only publicly routable destinations. A proxy given by name is resolved, every address it answers with must be public, and the request is dialled at the address that was checked rather than at the name — so `http://proxy.example.com:8080` is sent on as `http://<its address>:8080`. An `https` proxy keeps its name, because its certificate is verified against it. `any` accepts loopback and private ranges too and is only coherent when every API key holder is already trusted with the network the instance runs in.
- Schemes: `http`, `https`, `socks5`, `socks5h`. Maximum 512 characters. Give it as a full URL, e.g. `http://host:port`.
- A refusal is `INVALID_PARAM` with `details.reason` — one of `request_proxy_disabled`, `too_long`, `scheme_missing`, `scheme_not_supported`, `authority_invalid`, `host_missing`, `port_invalid`, `host_not_public`, `host_unresolvable`. The value you sent is never echoed back or logged, because a rejected proxy URL is exactly when someone has pasted a real credential.
- A disabled feature **refuses** rather than ignores. Silently dropping the parameter would send the request from the instance's own address while you believed it went through your proxy.

The cost is real and is not a free option: the identity's cookies were minted behind one address and would now be presented from another, which is an incoherence the platforms can see. Two callers asking for the same post through different proxies are not asking the same question and are never coalesced.

## `?explain=`

Returns the request as it actually went out — the signed URL, the headers and the identity's cookie jar — so it can be replayed outside this instance. It is the debugging tool for "the platform refused us and I need to see what we sent".

- Requires `identity:manage` **and** the `operator` role, because the answer contains a credential. A `douyin:read` key may ask this instance to *use* a jar; it may not ask to be handed one.
- Every use is written to the audit log — who asked, and for which endpoint. The line names no jar.
- It **implies `refresh`**: an explanation of a cached answer would describe a call this request did not make. It also disables coalescing, so an explained call is never joined to an unexplained one already in flight.
- The block appears in the result metadata under `explain`, with `method`, `url`, `headers`, `cookie_header`, `identity_id`, `signer`, `endpoint` and `proxy`.
- It is recorded on **failed** tasks too — that is the case it exists for.
- It is stripped from a stored task result for any reader without `identity:manage`, and it is never sent on the SSE stream.

See [Playground and tools](./07-playground-and-tools.md), which is the console's front end for the same thing.

## The endpoint families

`{platform}` is `douyin` or `tiktok` and must match the link you pass; a mismatch is `INVALID_URL` with `details.reason: "platform_mismatch"`.

### Content — asynchronous, spends an identity

| Method | Path | Scope | Notes |
| --- | --- | --- | --- |
| `POST` | `/api/v1/parse` | either read scope | Any supported link, or the whole share text around one |
| `POST` | `/api/v1/tasks/batch` | either read scope | Up to 50 links, one task each |
| `GET` | `/api/v1/{platform}/video` | `{platform}:read` | One post. `url` **or** `aweme_id` |
| `GET` | `/api/v1/{platform}/video/comments` | `{platform}:read` | Paged |
| `GET` | `/api/v1/{platform}/video/comments/replies` | `{platform}:read` | `comment_id` plus `url` **or** `aweme_id`. Paged |
| `GET` | `/api/v1/{platform}/user` | `{platform}:read` | Profile. `url` **or** `sec_user_id` |
| `GET` | `/api/v1/{platform}/user/posts` | `{platform}:read` | Paged |
| `GET` | `/api/v1/{platform}/user/likes` | `{platform}:read` | Paged. See the platform note below |
| `GET` | `/api/v1/{platform}/mix/posts` | `{platform}:read` | `mix_id` — Douyin `mix_info`, TikTok `playlistId`. An author's own series, not a bookmark folder. Paged |
| `GET` | `/api/v1/{platform}/user/collections` | `{platform}:read` | Bookmark folders. Takes an author on TikTok; takes none on Douyin. Paged |
| `GET` | `/api/v1/{platform}/collection` | `tiktok:read` | **TikTok only.** One folder by `collection_id`: name, cover, count, owner, visibility |
| `GET` | `/api/v1/{platform}/collection/posts` | `{platform}:read` | Posts in one folder, by `collection_id`. Paged |
| `GET` | `/api/v1/{platform}/user/bookmarks` | `tiktok:read` | **TikTok only.** Every saved post, across all folders. Owner only. Paged |
| `GET` | `/api/v1/{platform}/user/reposts` | `tiktok:read` | **TikTok only.** Posts an author reposted. Paged |
| `GET` | `/api/v1/{platform}/user/followers` | `tiktok:read` | **TikTok only** |
| `GET` | `/api/v1/{platform}/user/following` | `tiktok:read` | **TikTok only** |

The platforms are not symmetric past the core five, and the asymmetry is reported rather than papered over. Asking Douyin for `followers` or `following` returns `UNSUPPORTED_CONTENT` with `details.supported` naming the platforms that do serve it — a route that submitted the task anyway would answer with an empty page and let you conclude the author has no followers. `user/likes` is offered on both, but Douyin does not serve that list to a guest identity: it needs an imported logged-in identity, and on TikTok an empty page usually means the author keeps their likes private.

Bookmark folders are the sharpest case of that asymmetry. A folder is public or private **individually on both platforms**, and a public one is readable with an ordinary pooled identity — no login anywhere. But the folder *list* differs: TikTok's takes an author and will answer about a stranger's public folders, while Douyin's request carries no user id at all and answers only about the identity sending it. So on Douyin, `user/collections` needs `identity` pinned to an imported identity, and passing `url` or `sec_user_id` there is refused rather than ignored — ignoring it would hand back your own folders under somebody else's name. Every folder reports `is_public` and its owner, so you never have to guess which case you are in.

Identify a post by **either** `url` **or** `aweme_id`, never both-or-neither; an author by **either** `url` **or** `sec_user_id` (TikTok calls it `secUid`). A URL that already carries the id has it extracted at the edge, so both spellings coalesce onto one task. Text with a link buried in it is accepted — the clipboard content the platform apps produce can be sent unedited. Short links (`v.douyin.com`, `vm.tiktok.com`) are queued for expansion in the worker, because following one is a network call the API container does not make.

Ids the caller typed are validated before they cost anything: `aweme_id=not-an-id` and `aweme_id=7123` are both `INVALID_PARAM` with `details.field`, answered for free rather than sent upstream to be refused.

### Everything else

| Prefix | What it is | Covered by |
| --- | --- | --- |
| `/api/v1/tasks/…` | Poll, stream and cancel tasks | This page |
| `/api/v1/tools/…` | `parse-url`, `parse-batch`, `sign`, `decode`, `identity` — dry runs and signing, no identity spent by the first four | [Playground and tools](./07-playground-and-tools.md) |
| `/api/v1/archive/…` | What this instance has already stored: search, stats, collections, export, recheck, backfill | [Downloads, library and watchlist](./08-downloads-and-library.md) |
| `/api/v1/downloads/…` | Media stored on the operator's disk | [Downloads, library and watchlist](./08-downloads-and-library.md) |
| `/api/v1/admin/…` | Identities, proxies, keys, users, settings, logs, watchlist, backup | [Operations](./10-operations.md), [Users and API keys](./09-users-and-api-keys.md) |
| `/api/v1/auth/…` | Console sessions and password changes | [Users and API keys](./09-users-and-api-keys.md) |
| `/api/v1/ios/…` | iOS Shortcut release metadata — unauthenticated | This page |
| `/api/v1/system/status` | Version, health, pool census, storage | [Operations](./10-operations.md) |

Archive endpoints answer from local storage: no identity is spent, nothing is fetched, and a post that has since been deleted is still there with `availability` saying so. That makes `GET /api/v1/archive` the right call when you want what the instance already knows, and `/parse` the right call when you need it fresh.

## Responses that are not the envelope

Five endpoints deliberately answer with something else, in four shapes. Do not send them through your envelope parser.

| Endpoint | Content type | Why |
| --- | --- | --- |
| `GET /healthz`, `GET /readyz` | plain JSON object | Probes for a load balancer; `/healthz` touches no dependency, `/readyz` answers `503` when Postgres or Redis is down |
| `GET /api/v1/tasks/{task_id}/events` | `text/event-stream` | Server-sent events |
| `GET /api/v1/archive/export` | `application/x-ndjson` | One post per line, streamed a page at a time; capped at 50,000 rows; needs `archive:export` |
| `GET /api/v1/downloads/{download_id}/files/{name}` | the file's own type | The stored bytes, with a `Content-Disposition` that makes a browser save them |

## A complete worked example

Submit a link, poll the task, read the result. All three clients do the same thing.

### curl

The polling version, using `python3` to read one field out of the JSON:

```bash
#!/usr/bin/env bash
set -euo pipefail

BASE="${DTK_BASE_URL:-http://127.0.0.1:8000}"
KEY="$DTK_API_KEY"
LINK="https://www.douyin.com/video/7123456789012345678"

field() { python3 -c 'import json, sys
value = json.load(sys.stdin)
for key in sys.argv[1:]:
    value = value[key]
print(value)' "$@"; }

task=$(curl -sS -X POST "$BASE/api/v1/parse" \
  -H "X-API-Key: $KEY" \
  -H 'Content-Type: application/json' \
  -d "{\"url\": \"$LINK\"}" | field data task_id)

echo "task $task"

for _ in $(seq 1 60); do
  body=$(curl -sS "$BASE/api/v1/tasks/$task" -H "X-API-Key: $KEY")
  state=$(printf '%s' "$body" | field data state)
  case "$state" in
    done)   printf '%s' "$body" | python3 -m json.tool; exit 0 ;;
    failed) printf '%s' "$body" | python3 -m json.tool; exit 1 ;;
  esac
  sleep 2
done

echo "still running after two minutes; task $task is still valid" >&2
exit 1
```

The synchronous one-liner, for when you just want the answer in a terminal:

```bash
curl -sS -X POST "$DTK_BASE_URL/api/v1/parse?wait=20" \
  -H "X-API-Key: $DTK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://www.douyin.com/video/7123456789012345678"}' \
  | python3 -m json.tool
```

Remember that `?wait=20` can still come back `202` — that is the task still running, not a failure.

### Python (httpx)

```python
"""Submit a link, poll the task, print the result. Needs: pip install httpx"""

from __future__ import annotations

import os
import time

import httpx

BASE = os.environ.get("DTK_BASE_URL", "http://127.0.0.1:8000")
HEADERS = {"X-API-Key": os.environ["DTK_API_KEY"]}

RETRYABLE = {
    "RATE_LIMITED",
    "QUEUE_FULL",
    "IDENTITY_POOL_EXHAUSTED",
    "ENDPOINT_CIRCUIT_OPEN",
    "UPSTREAM_RISK_CONTROL",
    "SIGNING_FAILED",
    "DOWNLOADER_UNAVAILABLE",
    "INTERNAL",
}


class DtkError(RuntimeError):
    def __init__(self, error: dict, request_id: str | None = None) -> None:
        super().__init__(f"{error['code']}: {error.get('message', '')}")
        self.code = error["code"]
        self.retry_after = error.get("retry_after")
        self.details = error.get("details")
        self.request_id = request_id


def unwrap(response: httpx.Response) -> dict:
    """Return `data`, or raise the envelope's error."""
    body = response.json()
    if not body.get("success"):
        raise DtkError(body["error"], response.headers.get("X-Request-ID"))
    return body["data"]


def submit(client: httpx.Client, url: str) -> str:
    """Queue a parse and return the task id, retrying while the instance pushes back."""
    for attempt in range(5):
        try:
            return unwrap(
                client.post("/api/v1/parse", json={"url": url})
            )["task_id"]
        except DtkError as exc:
            if exc.code not in RETRYABLE or attempt == 4:
                raise
            time.sleep(exc.retry_after or 2 ** attempt)
    raise RuntimeError("unreachable")


def collect(client: httpx.Client, task_id: str, deadline: float = 120.0) -> dict:
    """Poll until the task settles. A failed task is HTTP 200 with state 'failed'."""
    delay, end = 0.5, time.monotonic() + deadline
    while time.monotonic() < end:
        task = unwrap(client.get(f"/api/v1/tasks/{task_id}"))
        if task["state"] == "done":
            # The result is nested: envelope.data.data
            return task["data"]
        if task["state"] == "failed":
            raise DtkError(task["error"])
        time.sleep(delay)
        delay = min(delay * 2, 5.0)
    raise TimeoutError(f"task {task_id} still running; it remains valid, poll it again")


def main() -> None:
    with httpx.Client(base_url=BASE, headers=HEADERS, timeout=30.0) as client:
        task_id = submit(client, "https://www.douyin.com/video/7123456789012345678")
        print("task", task_id)
        result = collect(client, task_id)
        print(result.get("title") or result.get("description"))


if __name__ == "__main__":
    main()
```

Two things in that code are the point of it: `unwrap` branches on `success` and on `error.code`, never on the message; and `collect` treats a `failed` task as data rather than as an HTTP failure, because that is what it is.

### JavaScript (fetch)

```javascript
// Node 18+ or any modern browser (see "Calling from a browser" for CORS).
const BASE = process.env.DTK_BASE_URL ?? "http://127.0.0.1:8000";
const HEADERS = { "X-API-Key": process.env.DTK_API_KEY, "Content-Type": "application/json" };

const RETRYABLE = new Set([
  "RATE_LIMITED", "QUEUE_FULL", "IDENTITY_POOL_EXHAUSTED", "ENDPOINT_CIRCUIT_OPEN",
  "UPSTREAM_RISK_CONTROL", "SIGNING_FAILED", "DOWNLOADER_UNAVAILABLE", "INTERNAL",
]);

class DtkError extends Error {
  constructor(error, requestId) {
    super(`${error.code}: ${error.message ?? ""}`);
    this.code = error.code;
    this.retryAfter = error.retry_after ?? null;
    this.details = error.details ?? null;
    this.requestId = requestId ?? null;
  }
}

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function unwrap(response) {
  const body = await response.json();
  if (!body.success) throw new DtkError(body.error, response.headers.get("X-Request-ID"));
  return body.data;
}

async function submit(url) {
  for (let attempt = 0; attempt < 5; attempt += 1) {
    try {
      const data = await unwrap(await fetch(`${BASE}/api/v1/parse`, {
        method: "POST",
        headers: HEADERS,
        body: JSON.stringify({ url }),
      }));
      return data.task_id;
    } catch (err) {
      if (!(err instanceof DtkError) || !RETRYABLE.has(err.code) || attempt === 4) throw err;
      await sleep(1000 * (err.retryAfter ?? 2 ** attempt));
    }
  }
}

async function collect(taskId, deadlineMs = 120_000) {
  let delay = 500;
  const end = Date.now() + deadlineMs;
  while (Date.now() < end) {
    const task = await unwrap(await fetch(`${BASE}/api/v1/tasks/${taskId}`, { headers: HEADERS }));
    if (task.state === "done") return task.data;   // envelope.data.data
    if (task.state === "failed") throw new DtkError(task.error);
    await sleep(delay);
    delay = Math.min(delay * 2, 5000);
  }
  throw new Error(`task ${taskId} still running; it remains valid, poll it again`);
}

const taskId = await submit("https://www.douyin.com/video/7123456789012345678");
console.log("task", taskId);
console.log(await collect(taskId));
```

## Calling from a browser

Cross-origin requests are refused by default: `security.cors_allow_origins` is empty, which means same-origin only, and a preflight simply finds no `OPTIONS` route to answer it. An operator who wants browser access lists the exact origins in that setting.

- Methods answered cross-origin: `GET`, `POST`, `PUT`, `DELETE`, `OPTIONS`.
- Headers a browser caller can read: `X-Request-ID`, `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, `Retry-After`. Without these exposed, the correlation id and the rate-limit budget are invisible to `fetch()`, which is most of what they exist for.
- Setting the origin list to `*` forces credentials off. An operator who typed `*` asked for an open anonymous API, not an open authenticated one — and a browser would reject the pair anyway.

Do not ship an API key in a page you do not control: anyone who reads the page holds the key and can spend your identity pool. Put a small server of your own in front, or open exactly the endpoints you need with `api.public_endpoints` and accept the anonymous rate limit.

## Limits and defaults

Everything here is read from the code; the settings marked *(setting)* are editable at runtime on the console's Settings page — see [Configuration reference](./03-configuration.md).

| Thing | Value |
| --- | --- |
| `?wait=` ceiling *(setting `api.max_wait_seconds`)* | 30 seconds |
| Default rate limit *(setting `api.default_rate_limit_per_min`)* | 120 requests/minute |
| Queue ceiling *(setting `sched.queue_max`)* | 500 tasks |
| `QUEUE_FULL` retry hint *(setting `sched.max_wait_seconds`)* | 10 seconds |
| Task result retention *(setting `retention.task_result_hours`)* | 24 hours |
| Cache TTL, one post *(setting `cache.content_ttl`)* | 1800 seconds |
| Cache TTL, author profile *(setting `cache.author_ttl`)* | 900 seconds |
| Cache TTL, any paged list *(setting `cache.list_ttl`)* | 300 seconds |
| Coalescing claim lifetime | 90 seconds |
| Platform page size (`count`) | default 20, maximum 50 |
| Archive page size (`limit`) | default 50, maximum 200 |
| Downloads / admin listing (`limit`) | default 100, maximum 500 |
| Batch items per request | 50 |
| Request body ceiling | 1 MiB (1,048,576 bytes) |
| `url` parameter length | 4096 characters |
| `cursor` length | 512 characters |
| `aweme_id` / `mix_id` / `comment_id` length | 64 characters |
| `sec_user_id` length | 256 characters |
| `callback_url` length | 2048 characters |
| `proxy` length | 512 characters |
| SSE stream `timeout` | 1–300 seconds, default 300 |
| SSE heartbeat | ~15 seconds |
| Webhook attempts / timeout / backoff | 3 / 10s / 2s then 8s |
| Archive export cap | 50,000 rows |

One documentation wrinkle to know about: the API document's own front page (visible at `/swagger`) says a `callback_url` host "must be on the operator's `security.url_allowlist`". That list governs which hosts a *short link* may redirect through during expansion; it has nothing to do with callbacks. The actual gate on a callback is `security.enable_task_webhook` plus the https and non-private-address checks described above.

## Where to go next

- [Quick start](./01-quickstart.md) — if you have no running instance yet, this is where the base URL and the first key come from
- [Concepts](./04-concepts.md) — identities, the scheduler, the circuit breaker, and why a request costs what it costs
- [Configuration reference](./03-configuration.md) — every setting named on this page
- [Users and API keys](./09-users-and-api-keys.md) — creating and scoping the key these examples use
- [Playground and tools](./07-playground-and-tools.md) — the console front end for `explain`, signing and link identification
- [MCP and AI agents](./12-mcp.md) — the same capabilities as agent tools
- [CLI reference](./13-cli.md) — the same capabilities from a terminal
- [Troubleshooting](./14-troubleshooting.md) — what to do when a code above keeps coming back