API, SDK and scheduled runs have hung since socket auth was added.
`authenticateSocket` is applied to every namespace via `io.on('new_namespace')`,
and it requires a token in `handshake.auth` or a `token` cookie. Browser clients
carry the httpOnly cookie, so the UI is unaffected. But the backend also joins
its own `/<browserId>` namespace as a socket.io *client* when a run is started
by something other than a browser, and that connection has neither a cookie nor
an auth token. It is refused.
The refusal is not what makes this hard to spot. `connect_error` is logged and
the socket cleaned up, but the run is never failed - so the browser eventually
falls back to a dummy socket, `ready-for-run` never reaches the caller, and the
run sits there. It reads as "slow", not as "broken".
Both server-opened connections are affected and they are the three paths people
report: `api/record.ts` (API and SDK) and `workflow-management/scheduler`
(scheduled runs).
Fixed by minting a short-lived token with the same `id` claim the HTTP login
issues, so the gate needs no special case and `socketOwns` keeps working: the
connection authenticates as the user the run already belongs to, not as a
privileged internal caller. A 60-second expiry because it is used once,
immediately - a login token is deliberately long-lived, which is right for a
session and wrong for a handshake.
Verified by A/B on one stack, same database, only the image changing:
before run hangs, 120s deadline exceeded
"Socket connection error for API run ...: Unauthorized"
"proceeding with dummy socket" -> "[SDK] Error executing robot: Run not found"
after run completes in 18.6s
0 Unauthorized socket errors, 0 dummy socket fallbacks
And the gate itself, against `/queued-run`: no auth is refused, a minted token
connects, a token signed with the wrong secret is refused.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Called downscaleIfOversized immediately after writing the temp file.
Also switched the resized dimensions from Math.round to Math.floor so the integer result cannot exceed MAX_OCR_PIXELS.
Added an escapeHtml helper and apply it to paragraph text and both table
cell paths. Paragraph text is escaped before newlines become <br>, so
the inserted breaks survive. Table structure, page sections, and the
DOCX sourceHtml passthrough are unchanged, and toMarkdown is untouched
since markdown output is not HTML.
Note this also affects the PDF path: ocrPDF calls preprocessPageImage per rasterized page, so a page that cannot be measured or downscaled now fails that page's OCR instead of silently continuing.
This fix addresses the edge case of an image with a large pixel count that is still under the 10 MB upload limit. For example, a 10000x10000 pixel JPEG compresses well under 10 MB but needs >2 GB across the per-pixel buffers in preprocessPageImage. Typed arrays are backed by external memory rather than V8's old space, so --max-old-space-size does not cap them and the process can be OOM-killed instead of failing gracefully.
Blank images are the most egregious case: PaddleOCR bounds its own input via maxSideLength so an image only reaches this code through the Tesseract fallback which is triggered by empty PaddleOCR output.
Read dimensions from the header with sharp (no full decode) and downscale above 25 MP before any allocation. Downscaling rather than rejecting keeps high-DPI scans working, since OCR gains nothing beyond roughly 300 DPI. This also covers the PDF path, where ocrPDF rasterizes pages at 4x and a large-format page could hit the same ceiling.
Add DOCX, JPEG (plus the image/jpg alternate form), and PNG to the SDK's document upload allow-list, so the SDK endpoints accept the same formats as the document robots themselves.
Keeps the explicit MIME-type list rather than delegating to normalizeDocumentMimeType, so the SDK continues to match on the client-supplied MIME type only, with no filename-extension fallback.
Added a image mime type check with isImageMimeType that will point images to parseImage.
Added parseImage function that uses Paddle to OCR parse images.
Added ocrImageWithTesseract function to use Tesseract as a fallback.
Reject non-positive/fractional limit values in getLLMDecisionWithVision's
Anthropic branch, matching the positive-safe-integer rule already enforced
in parseSearchIntent. Validate and sanitize generateFieldLabelsBatch's
Anthropic-branch label mapping before returning it: reject non-string,
blank, and __proto__/constructor/prototype values, since callers use each
value as an object key downstream (renamedFields[semanticLabel] = ...).
Requested by maintainer review on PR #1189. The forced tool_choice call in
isMultiSitePrompt's Anthropic branch left too little headroom at 20 tokens,
risking truncation before the tool_use block completes.
Reject __proto__/constructor/prototype field names and use a null-prototype
target in filterFieldsByIntent's Anthropic branch to prevent prototype
pollution from untrusted tool output. Add explicit type validation for
first/second/reason/limit in getLLMDecisionWithVision's Anthropic branch,
since input_schema isn't runtime-enforced without strict mode. Reject
non-positive and non-safe-integer limit values in both parseSearchIntent
validation blocks (Anthropic branch and shared ollama/openai tail).
All 3 verified against current code and the actual failure modes before
fixing, not applied blindly from the bot's suggestion text.
1. getLLMDecisionWithVision: the Anthropic image block declared
media_type: 'image/png' while the screenshot is captured as JPEG
(page.screenshot({ type: 'jpeg' })). This is the exact mismatch already
disclosed (by me) in the PR's "Note to Reviewers" before CodeRabbit
caught it independently, confirmed against Anthropic's own docs that a
media_type mismatch can 400 before the forced tool call ever runs.
Fixed to 'image/jpeg'. The identical bug also exists in the untouched
openai branch's `data:image/png;base64,...` URL - left as-is and
flagged, not silently fixed, since that branch is explicitly out of
this PR's scope.
2. parseSearchIntent: the validation `!intent.searchQuery ||
!intent.extractionGoal` is a truthiness check, not a type check. Since
callAnthropicWithTool returns raw tool_use.input with no strict-mode
validation, a non-string value (e.g. an object) passes this guard, and
downstream `encodeURIComponent(query)` (workflowEnricher.ts, both
performDuckDuckGoSearch and performDuckDuckGoMultiSearch) silently
turns it into the literal string "[object Object]" as a search query.
Verified this exact call chain before fixing. Replaced with a guard
that requires searchQuery/extractionGoal to be non-blank strings and
limit (when present) to be an integer. Applied to both the anthropic
branch's copy of this check AND the shared ollama/openai JSON-parsing
branch's identical copy, per CodeRabbit's own comment text ("Apply the
same validation to the JSON parsing branch") - not my own scope
expansion, the finding explicitly asked for both.
3. selectMultipleUrlsFromResults: the per-index collection loop had no
cap on `result.length`, so a tool response returning more valid,
unique-domain indices than the caller's `maxSites`/`cap` would collect
all of them instead of stopping at cap - e.g. 5 valid indices with
cap=4 would produce 5 downstream workflows instead of 4. Added
`if (result.length >= cap) break;` at the top of the loop, matching
CodeRabbit's exact suggested diff. This identical bug also exists,
unchanged, in the original shared-tail copy of this same loop (which
predates this PR and serves ollama/openai) - CodeRabbit's finding did
not ask for that copy to be fixed too, so it's flagged, not touched.
Verified: full project tsc --noEmit clean. A targeted test reproduced each
finding's exact described failure mode against the pre-fix code path
conceptually and confirmed the fix resolves it: searchQuery as an object,
as an array, as an empty/whitespace string, limit as a fraction, and limit
as a numeric string are all now correctly rejected (falling back to the
existing regex heuristic, not silently proceeding); valid inputs including
an omitted limit still succeed unchanged; 5 valid unique-domain indices
against cap=4 now correctly returns exactly 4, and a below-cap selection
is unaffected. Re-ran the full 9-site fallback regression afterward to
confirm the shared-code changes (finding #2) didn't affect any of the
other 8 sites.
All 9 anthropic branches now go through callAnthropicWithTool; the raw
`new Anthropic(...)` client construction that used to happen per-call-site
is gone as of the site #9 migration (372ded1d). The top-level
`import Anthropic from '@anthropic-ai/sdk'` had zero remaining references
- confirmed via `grep -n '\bAnthropic\b'` before removing it.
mrhard9090