Commit Graph
2071 Commits
Author SHA1 Message Date
Rohit Rajan afb841b28e fix: add summary format field doc 2026-08-07 23:44:43 +05:30
Rohit Rajan 9262631d7b fix: null byte corruption 2026-08-03 17:29:45 +05:30
Rohit d91ecf0a54 Merge branch 'develop' into llm-fixes 2026-08-03 16:59:42 +05:30
Karishma Shukla fb4c6b5b0e Merge pull request #1149 from RRRG3/feat/doc-parse-summary
feat: add summary output format to document parsing (#1106)
2026-08-03 16:49:44 +05:30
Karishma Shukla c5e849eac2 Merge pull request #1157 from getmaxun/llm-process
fix: enhance document llm post processing
2026-08-03 16:34:59 +05:30
Karishma Shukla 650bcad7b4 Merge pull request #1162 from audreydng/fix/serialize-navigation-rrweb-race
Fix: Server and client crash when multiple recordings are discarded in rapid succession
2026-08-03 16:34:49 +05:30
Rohit 3555e74f34 Merge branch 'develop' into llm-process 2026-08-03 16:32:44 +05:30
Rohit Rajan 54606d38c0 fix: revert multi-auth routes 2026-08-03 13:59:59 +05:30
Rohit Rajan 53ac553ccf fix: follow file naming convention 2026-07-31 14:13:53 +05:30
Rohit Rajan 062dcfc9fc fix: validate llm config at robot creation, centralize model defaults 2026-07-31 14:05:19 +05:30
Karishma Shukla b8900dc5b3 Merge pull request #1150 from vinaypokharkar/feature/api-schedule-robot
Add API routes to schedule robot runs
2026-07-30 14:27:51 +05:30
Vinaywho 1be4861cb3 fix(schedule): correct weekly/monthly cron generation and enforce time window
- MONTHS: drop the appended day-of-week field that produced a 6-field cron
  (node-cron reads 6-field crons seconds-first, shifting the schedule)
- MONTHS: validate dayOfMonth (1-31) and emit the parsed integer
- MINUTES/HOURS: constrain the schedule to the atTimeStart/atTimeEnd window
  instead of ignoring atTimeEnd; require end after start, and an hour-spanning
  window for hourly intervals
- WEEKS: reject runEvery > 1 since cron cannot express multi-week intervals
2026-07-30 01:25:22 +05:30
Thao Dang 2814e0d1b0 Scope navigation coordination per page 2026-07-29 14:25:39 -05:00
Thao Dang 96915d1f8e Move duplicate detection inside the serialized task 2026-07-29 14:22:43 -05:00
Thao Dang 64819f2113 Treat rrweb already recording as idempotent 2026-07-29 13:27:25 -05:00
Thao Dang 0739ab5023 Fix rrweb navigation race 2026-07-29 12:30:47 -05:00
RRRG3 f3f26391b2 Merge branch 'develop' into feat/doc-parse-summary 2026-07-29 09:39:07 -07:00
Rohit Rajan ae65ae837b fix: strip thinking blocks summarization 2026-07-29 13:12:11 +05:30
Rohit Rajan 71e10667e5 fix: add llm post processing 2026-07-27 11:47:40 +05:30
Wootark-Kim e7928d9a82 renamed extractTextFromPDF() to extractText() 2026-07-24 12:55:50 -07:00
Wootark-Kim b3b0380e24 expanded on edge cases for parse methods 2026-07-24 10:26:13 -07:00
joey chen 021288e685 minor fix in extractData() 2026-07-23 14:10:50 -07:00
joey chen 2df5614780 minor fix to extractTextFromPDF() 2026-07-23 13:59:39 -07:00
joey chen a46726f883 extend extract feature to CSV and XLSX, fix date serialization, handle cell type edge cases 2026-07-23 13:46:52 -07:00
Batkhuu Byambaa fcb442e490 fix: format CSV date cells as readable dates 2026-07-23 12:28:02 -07:00
Batkhuu Byambaa 29d123a1da fix: format CSV date values consistently 2026-07-23 11:56:07 -07:00
Batkhuu Byambaa c871484f13 fix: format XLSX date cells as readable dates 2026-07-23 11:46:15 -07:00
Vinaywho 1278f9f43b feat: add API routes to schedule robot runs
Add POST/GET/DELETE /api/robots/:id/schedule to the Website-to-API
feature so robots can be scheduled through the API, using the same
fields as the dashboard schedule form. Cron building and validation
are pulled into a shared helper so the dashboard and API routes stay
in sync.
2026-07-22 01:55:01 +05:30
RRRG3 f86778ce04 feat: add summary output format to document parsing (#1106)
Add 'summary' as a selectable output format for the doc-parse robot type,
matching the existing summary capability in crawl and search robots.

- Add 'summary' to DOC_PARSE_FORMAT_OPTIONS (frontend + backend constants)
- Extend DocumentInterpreter.parse() to generate summaries via the shared
  summarizeMarkdown() utility (same logic crawl/search already use)
- Wire LLM config (provider, model, API key) through the doc-parse creation
  endpoint, robot record, and run execution path
- Validate LLM API keys when summary is requested with a non-Ollama provider
- Serialize summary output in executeDocumentParseRun for run results
- Display summary in the frontend run results UI with copy/download actions

Closes #1106
2026-07-20 09:11:58 -07:00
Wootark-Kim c3b6cc0df9 changed name of pdfUpload to uploadFile 2026-07-18 14:16:33 -07:00
Wootark-Kim 0cbca0a082 Merge remote-tracking branch 'upstream/develop' into contingency-plan 2026-07-18 13:53:17 -07:00
Wootark-Kim ab5a5fad65 contingency plan in place 2026-07-18 13:28:13 -07:00
Wootark-Kim e228d9309c modification of parse() function to define the fileType logic 2026-07-18 02:18:08 -07:00
Wootark-Kim 3233b40e77 proof check of parseCSV() 2026-07-18 02:09:12 -07:00
Wootark-Kim 179b2d1d99 added onto async parse() method - if statement for xlsx 2026-07-17 13:33:09 -07:00
joey chen 32e81d1144 merge DocumentInterpreter changes 2026-07-17 13:26:21 -07:00
Wootark-Kim 7042139a9e test commit2 2026-07-17 12:23:27 -07:00
Wootark-Kim dcb221585a test commit 2026-07-17 11:52:04 -07:00
Wootark-Kim 212cc4c4d1 update progress: draft v.2 2026-07-15 13:27:22 -07:00
amhsirak e8807096c3 fix: add sdk api gen URL validation 2026-07-16 00:29:30 +05:30
Karishma Shukla ae73f1f676 Merge pull request #1132 from getmaxun/wait-logic
fix: add wait for ready logic scrape
2026-07-15 23:28:08 +05:30
Wootark-Kim 2982382952 progress update: wrote basic structure for parseXLSX 2026-07-15 10:44:34 -07:00
Karishma Shukla 79b1e995c8 Merge pull request #1130 from getmaxun/strip-hidden
fix: strip hidden base64-encoded data blobs
2026-07-15 23:06:42 +05:30
Karishma Shukla a5ea29ca29 Merge pull request #1128 from getmaxun/llm-config
fix: cover llm config setup for all robot types
2026-07-15 22:53:29 +05:30
Rohit Rajan 5ae84cdb7e fix: check hex 64bit key 2026-07-15 21:35:51 +05:30
Karishma Shukla a078b9fea7 Merge pull request #1138 from hmhngx/feat/incremental-run-persistence
fix: persist crawl/search results incrementally instead of only at the end of the run
2026-07-15 13:58:36 +05:30
hmhngx 0c0033216f chore: drop jest test infra to keep PR scoped to persistence fix
Per maintainer review (RohitR311 on #1138): the repo has no existing
test infra or CI to run tests, so introducing jest here is out of
scope. Removes jest configs, test files, tsconfig.test.json, the
test:server script, and jest/ts-jest/@types/jest/jest-environment-jsdom
dependencies from both maxun-core and server.

output-validation.ts stays - it's a real production module (not test
code) extracted to fix the duplicated isPartial bug CodeRabbit flagged;
only its test file is dropped.
2026-07-13 21:16:32 +07:00
Karishma Shukla 4731cce37c Merge pull request #1134 from sebastiondev/fix/cwe20-inputhandlers-onchangeur-522c
fix(browser-management): restrict handleChangeUrl to http(s) schemes
2026-07-10 18:42:38 +05:30
hmhngx 5b19898341 test(server): cover mergeSearchDelta empty-state and no-mutation cases
mergeCrawlDelta already had tests for an undefined starting state and
for not mutating its inputs; mergeSearchDelta was missing the same
coverage.
2026-07-10 14:36:06 +07:00
hmhngx 84c96baa24 fix(server): roll back crawl/search persist cursor on dropped writes
computeCrawlSearchPersistDelta advances lastPersistedResultCount as
soon as a delta is computed, before the write is confirmed. If
flushPersistenceBuffer exhausts its retries and drops the batch, that
data was never actually written, but the cursor had already moved
past it - so it was silently lost forever instead of being resent
with the next page's delta.

Roll the cursor back by the dropped delta's size when a batch is
dropped, so the next page recovers the lost items instead of skipping
them. Covered by two new tests exercising the real retry-exhaustion
path via a rejecting sequelize.transaction mock.
2026-07-10 14:16:38 +07:00