Model Card for CAT-Tryoff-Flux

CAT-Tryoff-Flux is an advanced tryoff model. It used the same method of (CATVTON-FLUX)[https://huggingface.co./xiaozaa/catvton-flux-alpha]. This model can extract and reconstruct the front view of clothing items from images of people wearing them.

Model Details

Model Description

Model Sources [optional]

Uses

The model is designed for virtual try-off applications, allowing users to visualize how different garments would look on a person. It can be used directly through command-line interface with the following parameters:

Input person image Person mask Garment image Random seed (optional)

How to Get Started with the Model

transformer = FluxTransformer2DModel.from_pretrained(
    "xiaozaa/cat-tryoff-flux", 
    torch_dtype=torch.bfloat16
)
pipe = FluxFillPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    transformer=transformer,
    torch_dtype=torch.bfloat16
).to("cuda")


Training Details

Training Data

VITON-HD dataset

Training Procedure

Finetuning Flux1-dev-fill

Evaluation

Summary

BibTeX:

@misc{chong2024catvtonconcatenationneedvirtual,
 title={CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models}, 
 author={Zheng Chong and Xiao Dong and Haoxiang Li and Shiyue Zhang and Wenqing Zhang and Xujie Zhang and Hanqing Zhao and Xiaodan Liang},
 year={2024},
 eprint={2407.15886},
 archivePrefix={arXiv},
 primaryClass={cs.CV},
 url={https://arxiv.org/abs/2407.15886}, 
}
@article{lhhuang2024iclora,
  title={In-Context LoRA for Diffusion Transformers},
  author={Huang, Lianghua and Wang, Wei and Wu, Zhi-Fan and Shi, Yupeng and Dou, Huanzhang and Liang, Chen and Feng, Yutong and Liu, Yu and Zhou, Jingren},
  journal={arXiv preprint arxiv:2410.23775},
  year={2024}
}
Downloads last month
197
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for xiaozaa/cat-tryoff-flux

Finetuned
(4)
this model

Spaces using xiaozaa/cat-tryoff-flux 3