File size: 2,993 Bytes
8f55a5d fc241ff 3d0d923 8f55a5d b6350c1 99200f4 8f55a5d b6350c1 8f55a5d b6350c1 8f55a5d b6350c1 8f55a5d f644555 8f55a5d b6350c1 8f55a5d 3d0d923 8f55a5d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
---
license: other
license_name: server-side-public-license
license_link: https://www.mongodb.com/legal/licensing/server-side-public-license
tags:
- diffusion
- virtual try-on
- virtual try-off
- image generation
- fashion
- e-commerce
base_model:
- CompVis/stable-diffusion-v1-4
pipeline_tag: image-to-image
library_name: diffusers
---
## TryOffDiff
The models proposed in the paper _"TryOffDiff: Virtual-Try-Off via High-Fidelity Garment Reconstruction using Diffusion Models"_
[[paper]][paper_arxiv] [[project page]][project_page]:
- `tryoffdiff.pth`: The pre-trained StableDiffusion-v1.4 fine-tuned on `VITON-HD-train` dataset.
- `.pth`: A U-Net trained from scratch on `VITON-HD-train` dataset.
- `.pth`:
## Usage
```python
from huggingface_hub import hf_hub_download
class TryOffDiff(nn.Module):
def __init__(self):
super().__init__()
self.unet = UNet2DConditionModel.from_pretrained("CompVis/stable-diffusion-v1-4", subfolder="unet")
self.transformer = torch.nn.TransformerEncoderLayer(d_model=768, nhead=8, batch_first=True)
self.proj = nn.Linear(1024, 77)
self.norm = nn.LayerNorm(768)
def adapt_embeddings(self, x):
x = self.transformer(x)
x = self.proj(x.permute(0, 2, 1)).permute(0, 2, 1)
return self.norm(x)
def forward(self, noisy_latents, t, cond_emb):
cond_emb = self.adapt_embeddings(cond_emb)
return self.unet(noisy_latents, t, encoder_hidden_states=cond_emb).sample
path_model = hf_hub_download(
repo_id="rizavelioglu/tryoffdiff",
filename="tryoffdiff.pth", # or one of ablations ["ldm-1", "ldm-2", "ldm-3", ...]
)
net = TryOffDiff()
net.load_state_dict(torch.load(path_model, weights_only=False))
net.eval().to(device)
```
> Check out the demo code on [HuggingFace Spaces][hf_spaces] for the full running example.
> Also, check out [GitHub repository][github] to get more information on
> training, inference, and evaluation.
### License
TL;DR: Not available for commercial use, unless the FULL source code is shared! \
This project is intended solely for academic research. No commercial benefits are derived from it.
Models are licensed under [Server Side Public License (SSPL)][license]
### Citation
If you find this repository useful in your research, please consider giving a star ⭐ and a citation:
```
@article{velioglu2024tryoffdiff,
title = {TryOffDiff: Virtual-Try-Off via High-Fidelity Garment Reconstruction using Diffusion Models},
author = {Velioglu, Riza and Bevandic, Petra and Chan, Robin and Hammer, Barbara},
journal = {arXiv},
year = {2024},
note = {\url{https://doi.org/nt3n}}
}
```
[hf_spaces]: https://huggingface.co./spaces/rizavelioglu/tryoffdiff/blob/main/app.py
[project_page]: https://rizavelioglu.github.io/tryoffdiff/
[paper_arxiv]: https://arxiv.org/abs/2411.18350
[github]: https://github.com/rizavelioglu/tryoffdiff
[license]: https://www.mongodb.com/legal/licensing/server-side-public-license
|