GithubStar/Douyin_TikTok_Download_API
1
0
Fork 0
mirror of https://github.com/Evil0ctal/Douyin_TikTok_Download_API.git synced 2026-10-27 03:06:12 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
release
Douyin_TikTok_Download_API/docker/Dockerfile.browser
T
Evil0ctal 5d35ef10cb fix: make the browser container actually build, start and mint 
Reported: minting the first identity in the setup wizard answers
NOT_CONFIGURED. That error was correct - there was no browser - but four
separate things had to be wrong for it to be the outcome, and none of them had
ever been run.

THE WIZARD LED INTO IT. browser-rpc is optional by design: docker/compose.yml
keeps it behind a `browser` profile because a deployment that imports cookies by
hand does not need the heaviest image in the stack, and the file says manual
import is "the correct degraded mode rather than an error". The wizard did not
implement that mode. It offered Mint unconditionally, so the documented default
configuration answered the first thing a new operator does with a failure, and
its only other path was Skip - which leaves the pool empty and the smoke step
failing too. The step now reads the browser_rpc component from
/api/v1/system/status and, when it is absent, says so, gives the command that
starts it, and points at importing cookies instead.

THE IMAGE COULD NOT BE BUILT. Two lines were written from Playwright's shape and
never run against cloakbrowser:

  CLOAKBROWSER_INSTALL_CMD="python -m cloakbrowser install chromium"
      the real CLI takes no argument and rejects it outright
  test -d /opt/ms-playwright
      cloakbrowser installs to $HOME/.cloakbrowser, and reads no environment
      variable of its own - checked by grepping the package

So any build with a pinned commit failed, and every build without one produced
an image that starts and cannot mint. Verified against cloakbrowser 0.5.10:
`install` takes no arguments, the binary lands in
$HOME/.cloakbrowser/chromium-<version>/chrome, and `info` reports
"Installed: True", which is what the build asserts on now.

The install runs as root and the service runs as uid 10001 with a read-only
rootfs, so HOME differs between the two and a browser installed at build time
was unreachable at run time. An entrypoint links the versioned directory from
the image into the runtime HOME, which keeps the tool's small metadata writes on
the tmpfs where they can happen.

THE BACKEND WAS WRITTEN AGAINST THE WRONG LIBRARY. cloak.py looked for a module
exporting `async_playwright` and called `.chromium.launch_persistent_context()`.
cloakbrowser exports module-level `launch_*_async` coroutines and no
`async_playwright` at all, so the backend reported itself unavailable on an
image that had a working browser in it. It drives
`launch_persistent_context_async` now, with the parameter names checked against
the real signature - `timezone`, not Playwright's `timezone_id`, and no
`extra_http_headers`, which is set on the context instead.

AND THE CONTAINER STARVED IT. CHROMIUM_ARGS carried --disable-dev-shm-usage with
a comment saying the container also raises shm_size. Both were true, and they
cancel: the flag exists to move shared memory off a 64MB /dev/shm and onto the
filesystem, so with shm_size at 1GB it pushed every renderer into the 256MB /tmp
tmpfs, which then filled. Chromium died mid-navigation with "No space left on
device", which reaches the caller as "the page crashed" and reads exactly like a
platform block. The flag is gone and /tmp is 1G, which is a memory budget: two
warm contexts plus a mint in flight hold roughly 240MB of deleted-but-open
files.

Verified on the live stack: /rpc/health reports ok with chromium_major 146 and
two warm contexts; five consecutive mints returned 23-24 Douyin cookies and 7-8
TikTok cookies, four of five succeeding; and a minted identity issuing a real
signed request came back 200-with-empty-payload, classified risk_control /
payload.withheld - which is the classifier fix from earlier today reporting
correctly on real traffic rather than calling it a missing video.

CloakBrowser is pinned to f04c23da285b3b3d3cf10c8f9d282e7adc1d52ce (v0.5.10,
MIT). Its free tier downloads the browser without an account and allows one
concurrent session, which matches the one-at-a-time mint lock.
2026-09-07 21:11:10 -07:00

175 lines
7.4 KiB
Docker
Raw Permalink Blame History

# syntax=docker/dockerfile:1
#
# browser-rpc: the resident headless browser (docs/design/04-transport-signing.md).
#
# The backend is pinned, not tagged. At least three GitHub organizations publish
# repositories called cloakbrowser with identical descriptions, so "the latest
# cloakbrowser" does not identify any particular piece of software. This image
# installs one repository at one commit and stamps the pair into the image, where
# /rpc/health reports it.
#
# CLOAKBROWSER_COMMIT=<40-char sha> \
# docker compose -p dtk -f docker/compose.yml --profile browser build browser-rpc
#
# Built without that argument, the image contains the service and the fake
# backend but no browser. It then refuses to mint unless DTK_BROWSER_BACKEND=fake
# is set explicitly: falling back to synthetic identities on its own would fill
# the pool with cookies no platform accepts, and everything downstream would keep
# reporting success.
ARG PYTHON_VERSION=3.12.8
# Override with a digest-pinned image that already carries CloakBrowser, e.g.
# BROWSER_BASE_IMAGE=ghcr.io/<org>/cloakbrowser@sha256:<digest>
ARG BROWSER_BASE_IMAGE=python:${PYTHON_VERSION}-slim-bookworm
FROM ${BROWSER_BASE_IMAGE} AS runtime
ARG CLOAKBROWSER_REPO=https://github.com/CloakHQ/cloakbrowser
ARG CLOAKBROWSER_COMMIT=""
# The command that fetches the browser binaries after the Python package is
# installed. It is an argument because it belongs to the pinned revision, not to
# this file; verify it against the revision you pin (see docker/README.md).
# Verified against the real CLI at v0.5.10: `install` takes no arguments, and
# `install chromium` is rejected outright. The previous value was written from
# the shape Playwright uses and had never been run.
ARG CLOAKBROWSER_INSTALL_CMD="python -m cloakbrowser install"
# Where the browser is installed. CloakBrowser reads no environment variable of
# its own - checked by grepping the package - so the only lever is HOME, which
# is why the install and the runtime have to agree through the shim below.
ARG CLOAKBROWSER_HOME=/opt/cloakbrowser
ARG DTK_UID=10001
ARG DTK_GID=10001
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PYTHONPATH=/srv
# The browser binaries must not land under /tmp. Compose mounts a 256MB tmpfs
# over /tmp at runtime, which would hide a Chromium installed there at build
# time and could not hold it anyway - the container would come up, answer
# /rpc/health, and fail every mint with "executable doesn't exist".
#
# PLAYWRIGHT_BROWSERS_PATH is the documented override; HOME and XDG_CACHE_HOME
# are pointed at the same persistent place for the duration of the build in case
# the pinned revision derives its install directory from those instead. Both are
# moved back to /tmp below, which is where the *runtime* user needs to write.
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright \
HOME=/opt/browser-build \
XDG_CACHE_HOME=/opt/browser-build/.cache
# Chromium's shared-library and font dependencies. The CJK fonts are not
# cosmetic: a browser that cannot render the platform's own script measures text
# differently from every real visitor, and font metrics are a fingerprint.
RUN if [ -n "${CLOAKBROWSER_COMMIT}" ]; then \
apt-get update && apt-get install --no-install-recommends -y \
ca-certificates \
fonts-liberation \
fonts-noto-color-emoji \
fonts-noto-cjk \
libasound2 \
libatk-bridge2.0-0 \
libatk1.0-0 \
libatspi2.0-0 \
libcairo2 \
libcups2 \
libdbus-1-3 \
libdrm2 \
libexpat1 \
libgbm1 \
libglib2.0-0 \
libnspr4 \
libnss3 \
libpango-1.0-0 \
libx11-6 \
libxcb1 \
libxcomposite1 \
libxdamage1 \
libxext6 \
libxfixes3 \
libxkbcommon0 \
libxrandr2 \
xdg-utils \
git \
&& rm -rf /var/lib/apt/lists/*; \
fi
WORKDIR /srv
COPY docker/browser_rpc/requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir -r /tmp/requirements.txt && rm /tmp/requirements.txt
# The browser backend, pinned. Two separate steps so a failure names which one
# went wrong: fetching the package, or fetching the browser it drives.
RUN if [ -n "${CLOAKBROWSER_COMMIT}" ]; then \
pip install --no-cache-dir "cloakbrowser @ git+${CLOAKBROWSER_REPO}@${CLOAKBROWSER_COMMIT}" \
&& mkdir -p "${CLOAKBROWSER_HOME}" \
&& HOME="${CLOAKBROWSER_HOME}" sh -c "${CLOAKBROWSER_INSTALL_CMD}" \
&& HOME="${CLOAKBROWSER_HOME}" python -m cloakbrowser info \
| grep -qE "^Installed: *True"; \
else \
echo "browser-rpc: built without CLOAKBROWSER_COMMIT; only the fake backend is available" >&2; \
fi
# The install ran as root; the service runs as uid 10001 on a read-only root
# filesystem, so the binaries have to be world-readable and world-traversable.
# Asking the tool whether it is installed, rather than testing a path, is what
# turns "the pinned revision installs somewhere else" into a build failure
# instead of a container that starts and mints nothing.
RUN if [ -d "${CLOAKBROWSER_HOME}" ]; then chmod -R a+rX "${CLOAKBROWSER_HOME}"; fi
# Back to /tmp for the runtime: that tmpfs is the only writable path the
# container has. PLAYWRIGHT_BROWSERS_PATH deliberately stays where the browser
# actually is.
ENV HOME=/tmp \
XDG_CACHE_HOME=/tmp/.cache \
CLOAKBROWSER_HOME=${CLOAKBROWSER_HOME}
COPY docker/browser_rpc /srv/browser_rpc
COPY --chmod=0555 docker/browser-entrypoint.sh /usr/local/bin/browser-entrypoint
RUN groupadd --system --gid "${DTK_GID}" dtk \
&& useradd --system --uid "${DTK_UID}" --gid "${DTK_GID}" \
--home-dir /srv --shell /usr/sbin/nologin dtk \
&& chown -R root:root /srv
# Profile parent, owned by the runtime user. Compose mounts a tmpfs over it so
# single-use mint profiles never reach a disk; creating it here keeps a plain
# `docker run` of this image working too.
RUN mkdir -p /profiles \
&& chown "${DTK_UID}:${DTK_GID}" /profiles \
&& chmod 0700 /profiles
# Reported by /rpc/health, so "which browser is this" is answerable from the
# running system rather than from build history.
ENV DTK_BROWSER_BACKEND_PIN="${CLOAKBROWSER_REPO}@${CLOAKBROWSER_COMMIT}" \
DTK_BROWSER_BIND_HOST=0.0.0.0 \
DTK_BROWSER_BIND_PORT=9000 \
DTK_BROWSER_PROFILE_ROOT=/profiles
USER dtk
EXPOSE 9000
# Internal service, so the probe is loopback only. Compose repeats it because a
# compose-level check is what gates `up --wait`.
#
# The body is read, not just the status: /rpc/health answers 200 even when the
# browser backend failed to start - that is deliberate, so the operator gets a
# reason in the log instead of a restart loop - and a probe that stopped at the
# status code would call an image built without CLOAKBROWSER_COMMIT healthy.
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
CMD python -c "import json, sys, urllib.request; \
r = urllib.request.urlopen('http://127.0.0.1:9000/rpc/health', timeout=5); \
sys.exit(0 if r.status == 200 and json.load(r).get('status') == 'ok' else 1)"
ENTRYPOINT ["/usr/local/bin/browser-entrypoint"]
LABEL org.opencontainers.image.title="dtk browser-rpc" \
org.opencontainers.image.description="Headless browser RPC for dtk: identity minting and signature fallback" \
org.opencontainers.image.licenses="Apache-2.0" \
org.opencontainers.image.source="https://github.com/Evil0ctal/Douyin_TikTok_Download_API"
Reference in New Issue View Git Blame Copy Permalink