* FaceSwap 3 (#1515)
* Update extract pipeline
* Update requirements + setup for nvidia
* Remove allow-growth option
* tf.keras to keras updates
* lib.model.losses - Port + fix all loss functions for Keras3
* lib.model - port initializers, layers. normalization to Keras3
* lib.model.autoclip to Keras 3
* Update mixed precision layer storage
* model file to .keras format
* Restructure nn_blocks to initialize layers in __init__
* Tensorboard
- Trainer: Add Torch compatible Tensorboard callbacks
- GUI event reader remove TF dependency
* Loss logging
- Flush TB logs on save
- Replace TB live iterator for GUI
* Backup models on total loss drop rather than per side
* Update all models to Keras3 Compat
* Remove lib.model.session
* Update clip ViT to Keras 3
* plugins.extract.mask.unet-dfl - Fix for Keras3/Torch backend
* Port AdaBelief to Keras 3
* setup.py:
- Add --dev flag for dev tool install
* Fix Keras 3 syntax
* Fix LR Finder for Keras 3
* Fix mixed precision switching for Keras 3
* Add more optimizers + open up config setting
* train: Remove updating FS1 weights to FS2 models
* Alignments: Remove support for legacy .json files
* tools.model:
- Remove TF Saved Format saving
- Fix Backup/Restore + Nan-Scan
* Fix inference model creation for Keras 3
* Preview tool: Fix for Keras3
* setup.py: Configure keras backend
* train: Migration of FS2 models to FS3
* Training: Default coverage to 100%
* Remove DirectML backend
* Update setup for MacOS
* GUI: Force line reading to UTF-8
* Remove redundant Tensorflow references
* Remove redundant code
* Legacy model loading: Fix TFLamdaOp scalar ops and DepthwiseConv2D
* Add vertical offset option for training
* Github actions: Add more python versions
* Add python version to workflow names
* Github workflow: Exclude Python 3.12 for macOS
* Implement custom training loop
* Fs3 - Add RTX5xxx and ROCm 6.1-6.4 support (#1511)
* setup.py: Add Cuda/ROCm version select options
* bump minimum python version to 3.11
* Switch from setup.cgf to pyproject.toml
* Documentation: Update all docs to use automodapi
* Allow sysinfo to run with missing packages + correctly install tk under Linux
* Bugfix: dot naming convention in clip models
* lib.config: Centralise globally rather than passing as object
- Add torch DataParallel for multi-gpu training
- GUI: Group switches together when generating cli args
- CLI: Remove deprecated multi-character argparse args
- Refactor:
- Centralise tensorboard reading/writing + unit tests
- Create trainer plugin interfaces + add original + distributed
* Update installers
* tools.alignments - add export job
* plugins.extract: Update __repr__ for ExtractorBatch dataclass
* plugins.extract: Initial implementation of external import plugins
* plugins.extract: Disable lm masks on ROI alignment data import
* lib.align: Add `landmark_type` property to AlignedFace and return dummy data for ROI Landmarks pose estimate
* plugins.extract: Add centering config item for align import and fix filename mapping for images
* plugins.extract: Log warning on downstream plugins on limited alignment data
* tools: Fix plugins for 4 point ROI landmarks (alignments, sort, mask)
* tools.manual: Fix for 2D-4 ROI landmarks
* training: Fix for 4 point ROI landmarks
* lib.convert: Average color plugin. Avoid divide by zero errors
* extract - external:
- Default detector to 'external' when importing alignments
- Handle different frame origin co-ordinates
* alignments: Store video extension in alignments file
* plugins.extract.external: Handle video file keys
* plugins.extract.external: Output warning if missing data
* locales + docs
* plugins.extract.align.external: Roll the corner points to top-left for different origins
* Clean up
* linting fix
- Remove old face filter
- plugins.extract.pipeline: Expose plugins directly
- Change `is_aligned` from plugin level to ExtractMedia level
- Allow extract pipeline to take faceswap aligned images
- Add ability for recognition plugins to accept aligned faces as input
- Add face filter to recognition plugin
- Move extractor pipeline IO ops to own class
- Standardize image data reading and writing
- Optimize loading (just one pass required)
- Make all sort groups binnable (to greater or lesser results)
- Add sort by pitch
- Deprecate multiple options
- linting, docs + locales
- lib.detected_face
- Subclass Masks for Landmark based masks
- Add training mask propery + methods to DetectedFace
- lib.training_training
- subclass TrainingDataGenerator for training and preview data
- Split cache into own module
- Reduce thread count to 1 to prevent image corruption + data re-use
- Process on largest model input/output size rather than stored image size
- Size and crop masks during caching stage
- Implement ring buffer for data flow
- Fix preview reload bug
- augmentation
- typing
- switch color aug order
- better initialization
- Fix warp + landmark warp to correctly apply at different image scales
- Slightly improved warp caching
- Don't store whether image is_preview. Handle all data as training images implicitly
- plugins.trainer: Typing and fixes to work with trainingdata refactor
- lib.image Don't raise error if legacy non-png is found when reading header data
- plugins.train.trainer._base - Correctly pass legacy alignments through to DetectedFace
Documentation
- Update Usage.md, align.rst and image.rst
lib.image.py
- read_image - Remove hash return, add metadata return
- Remove read_image_hash functions
- Add read_image_meta functios
- Replace encode_image_with_hash with encode_image (to store metadata)
- Add png meta reading and writing functions
- Update Image Loaders/Savers to handle metadata rather than hashes
lib.training_data
- Naming updates to remove references to hashes
lib.align.Alignments
- Add versioning notes
- Increment alignments version to 2.1
- Deprecate hashing lookup functions
- Replace filter_hashes with filter_faces
lib.align.detected_face
- DetectedFace
- Remove hash property
- Add png header data serializing/deserializing functions
- Mask
- Add png header data serializing/deserializing functions
- add update_legacy_png_header function to update png meta data
lib.cli.args - Deprecate alignments files for training
- plugins.train.trainer
- Update alignments/mask code to read png header data
- scripts.convert
- Aligned images folder - read data from png headers
- scripts.extract
- Write png header information and no longer store hash of face
- tools.alignments
- remove leftover-faces, merge and update-hashes jobs
- Update jobs to use png meta data rather than hashes
- tools.manual
- Update extract code to output png meta data and don't store hashes
- Perform check on launch that tool is not pointing at a faces folder
tools.mask
- Update to use png meta data
tools.sort
- Update to use png meta data
- Manual Tool:
- Hide annotations for faces not meeting criteria
- Update landmarks on face add/del
- Clearer landmark annotations
- Handle non-numerics in frame number box
- Training
- Fix mis-aligned preview images
- Allows mixing legacy + new alignments for A and B
- Catch non-training images in training folder
- Catch inconsistently sized training images
- Standardize coverage ratio calculation
- lib.image - Add option to get image shape along with hash
Dfaker model:
- Add 256px mode
* Extract
- Implement aligner re-feeding
- Add extract type to pipeline.ExtractMedia
- Add pose annotation to debug
* Convert
- implement centering
- remove usage of feed and reference face properties
- Remove distributed option from convert
- Force update of alignments file on legacy receive
* Train
- Resize preview image to model output size
- Force legacy centering if centering does not exist in model's state file
- Enable training on legacy face sets
* Alignments Tool
- Update draw to include head/pose
- Remove DFL drop + linting
- Remove remove-frames job
- remove align-eyes option
- Update legacy masks to new extract type
- Exit if attempting to merge version 1.0 alignments files with version 2.0 alignments files
- Re-generate thumbnails on legacy upgrade
* Mask Tool
- Update for new extract + bugfix full frame
* Manual Tool
- Update to new extraction method
- Disable legacy alignments,
- extract box bugfix
- extract faces - size to 512 and center on head
* Preview Tool
- Display based on model centering
* Sort Tool
- Use alignments for sort by face
* lib.aligner
- Add Pose Class
- Add AlignedFace Class
- center _MEAN_FACE on x
- Add meta information with versioning to alignments file
- lib.aligner.get_align_matrix to use landmarks not face
- Refactor aligned faces in lib.faces_detect
* lib.logger
- larger file log padding
* lib.config
- Fix global changeable_items
* lib.face_filter
- Use new extracted face images
* lib.image
- bump thumbnail default size to 96px
torzdf