A page over 20 segments put the rest of the page in the last one, which
the prompt then cut, so a large CSV or sheet silently lost its tail
(a 6,000-row CSV came back with 2,302 rows). Pages are now split into as
many segments as they need, as multi-page documents already are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hskm8VdJhy1krbeJZMXCcS
- buildChunks counts each unit's label and separator against the budget;
the prompt's chunk cap is now a safety net above it, so packed chunks
no longer lose their last rows.
- Retry halves of a cut-off chunk run one at a time, keeping requests
within EXTRACTION_CONCURRENCY.
- salvageTruncatedJson cuts only between array elements or top-level
fields, so a record cut mid-way is dropped instead of kept with
missing fields.
- extractBalancedJson skips brackets in prose ("[see notes]") and only
treats an unclosed JSON-looking value as cut-off output.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hskm8VdJhy1krbeJZMXCcS
Completes the port of the document extraction fixes: the helpers were
added but not used yet.
- Chunking: split oversized pages (CSV/DOCX are one page), repeat table
headers across chunks, label sheets, size record chunks from the
document's own table rows.
- Providers: report hitting the output limit (Ollama done_reason, OpenAI
finish_reason, Anthropic stop_reason) and salvage the complete JSON.
- Extraction: chunks run 3 at a time, unwrap envelopes, re-extract
cut-off chunks in halves (up to 3 levels), tables only as context for
single-chunk documents, part N of M prompts with the JSON shape.
- PDF: per-page tables from pdf-parse laid out in place.
- XLSX/CSV: one Markdown table per sheet. DOCX: Markdown with tables.
- Markdown/HTML output rendered from page blocks.
- Schema prompt models repeated records as an array of objects.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hskm8VdJhy1krbeJZMXCcS
These keys are called with t() but are missing from public/locales/en.json, which is the fallback language, so the UI shows the key path instead of text.
Karishma Shukla