- Updated the guided decoding parameter logic to support dynamic imports, improving compatibility with different configurations.
- Simplified the creation of guided decoding parameters by introducing a helper function.
- Modified the retry mechanism in the OpenAI API call to retry on all exceptions, enhancing robustness in error handling.
- Replaced the previous parsing logic with enhanced error handling for JSON extraction from LLM outputs.
- Introduced detailed logging for failed requests and unexpected result types, improving traceability of issues during inference.
- Updated the import statement to reflect the new utility function for JSON extraction.
- Updated Python version requirement in `pyproject.toml` to 3.12.
- Adjusted dependency specifications for `llamafactory`, `torch`, `torchvision`, and `torchaudio` to include version constraints.
Enhance error messages when JSON parsing of LLM outputs fails in
offline inference. The system now dynamically extracts relevant
text snippets based on the response object type (e.g., RequestOutput,
ChatCompletion), making error logs more informative.
Additionally, suppress verbose debug/info logs from the OpenAI client
and httpx to reduce console noise during online inference.
Update the 'WC-exp' submodule and add 'models_final/*' to .gitignore
for repository hygiene.
Introduces a unified utility to parse and validate JSON-structured outputs
from both vLLM and OpenAI API inference results using Pydantic models.
This enables guided decoding for the OnlineLLM.chat method.
The existing guided decoding logic in vllm_infer is refactored to use
this new shared utility, improving consistency and error handling across
inference modes.
Introduce a ThreadPoolExecutor in the OnlineLLM class to enable
concurrent API calls, significantly boosting throughput for LLM operations.
Refactor the OlineLLMCleaningStrategy to leverage this new batching
capability, allowing multiple data points to be processed in parallel.
Increase the default `clean_batch_size` in settings and enhance
`n_process`/`batch_size` for PII detection to optimize for the new
concurrency. Simplify prompt management by removing a dedicated prompt
for online LLM cleaning. Add context manager support to OnlineLLM for
reliable resource cleanup. Ensure LLM chat responses explicitly request
JSON format.
Adds a call to empty the CUDA cache immediately after the LLM object is deleted, ensuring prompt release of GPU memory. This helps prevent out-of-memory issues and improves resource utilization for subsequent operations.
Allow `vllm_infer` to return `None` for failed JSON parsing results,
clarifying the sequence index for such failures. This enables
`LLMCleaningStrategy` to robustly align parsed LLM scores with original data,
correctly handling examples where JSON parsing failed by assigning a default
score of 0.
Adds an option to disable data cleaning during training setup.
Introduces a new 'enable_thinking' configuration for LLM interactions.
This allows dynamic control of thinking behavior in both offline inference
and the data cleaning process. When thinking is enabled for cleaning,
the maximum number of new tokens is increased to accommodate longer
reasoning outputs.
Also reduces the repetition penalty in the cleaning strategy for
improved response diversity and skips data entries containing images
during the LLM cleaning process to avoid irrelevant processing.
Implements batch processing for PII detection to improve
performance. Also, fixes a bug in the PII detection logic
and updates the installation instructions in the README.
Initializes the analyzer engines within the
ChinesePIIDetector class, ensuring proper
setup and custom recognizer loading during object
instantiation. This enhances the PII detection
process for Chinese-specific contexts.
Adds retry logic for OpenAI and image description API calls
to improve resilience against transient errors and rate
limiting. Implements exponential backoff with configurable
parameters.
Improves PII detection for Chinese language data by:
- Adding a Chinese-specific PII detector that inherits from
the base PII detector.
- Incorporating custom recognizers for common Chinese PII
patterns (e.g., phone numbers, ID cards).
- Filtering out irrelevant country-specific entities when
processing Chinese text.
- Adding unicode escape id recognizer.
These changes ensure more accurate and comprehensive
identification of sensitive information in Chinese datasets.
xming521