12 Commits

Author SHA1 Message Date
xming521 7daa270736 Add encrypted chat storage and shared-password workflows
Integrate suffixed encrypted storage across extraction, review, training, and inference. Add shared-password web authentication with two-hour sessions, private request logging, and an isolated profile-building pipeline.

Fix encrypted dataset discovery at training entrypoints and duplicate CSV imports. Add synthetic storage, authentication, and log checks; update documentation and frontend assets.
2026-10-02 23:18:07 +08:00
pre-commit-ci[bot] 86b7651f79 🎈 auto fixes by pre-commit hooks 2026-09-27 08:34:55 +00:00
xming521 8e337d7eea refactor: unify inference kernel and migrate main callers 2026-09-13 13:58:39 +08:00
xming521 3e7892fc76 feat(infer): add retry mechanism to API calls
Adds retry logic for OpenAI and image description API calls
to improve resilience against transient errors and rate
limiting. Implements exponential backoff with configurable
parameters.
2025-07-10 19:29:23 +08:00
xming521 38113a2524 翻译注释日志为英文 2025-07-01 20:00:34 +08:00
BAIKEMARK f3f0b61ad9 更新setting模板 2025-06-12 20:34:01 +08:00
BAIKEMARK 9f4a8f60a0 feat(data): 支持图像识别并优化数据清洗流程
- 新增 VisionApiConfig 类用于配置视觉 API
- 在数据处理中集成图像识别功能,支持并行处理
- 重构数据清洗策略,支持在线和离线两种方式- 优化数据清洗流程,提高可扩展性和可维护性
2025-06-12 13:13:57 +08:00
pre-commit-ci[bot] cf30258091 🎈 auto fixes by pre-commit hooks 2025-06-09 03:27:38 +00:00
xming521 11b0e3fbd3 Update .gitignore to ignore the dataset/images directory, modify the transformers version in pyproject.toml, adjust the clean_dataset configuration in settings.template.jsonc, add media_dir to the mllm template, optimize data processing logic in qa_generatorV2.py and utils.py, update the length_cdf function to support the media_dir parameter. 2025-05-31 10:24:51 +08:00
xming521 7ee6289ae4 修正check_image_file_exists函数中的文件匹配逻辑。 2025-05-30 15:49:20 +08:00
xming521 5c28bf084e 更新数据模型以支持ShareGPT格式,新增QaPairV2类和Message类,修改数据处理逻辑以支持图片消息,优化图片文件检查功能。 2025-05-25 23:23:58 +08:00
xming521 07662a3dc1 更新pyproject.toml以使用llamafactory的GitHub链接,修改settings.template.jsonc以更新模型路径和添加平台配置,增强数据处理逻辑以支持多语言切割类型,并在qa_generator.py中整合切割类型列表。 2025-05-24 23:24:26 +08:00