Total runs:
2.6M
Run Growth:
392.9K
Growth Rate:
15.24%
stabilityai / sd-vae-ft-mse-original
Model's Last Updated: December 19 2022
text-to-image
Introduction of sd-vae-ft-mse-original
Model Details of sd-vae-ft-mse-original
Improved Autoencoders
Utilizing
These weights are intended to be used with the original CompVis Stable Diffusion codebase . If you are looking for the model to use with the 🧨 diffusers library, come here .
Decoder Finetuning
We publish two kl-f8 autoencoder versions, finetuned from the original kl-f8 autoencoder on a 1:1 ratio of LAION-Aesthetics and LAION-Humans, an unreleased subset containing only SFW images of humans. The intent was to fine-tune on the Stable Diffusion training set (the autoencoder was originally trained on OpenImages) but also enrich the dataset with images of humans to improve the reconstruction of faces. The first, ft-EMA , was resumed from the original checkpoint, trained for 313198 steps and uses EMA weights. It uses the same loss configuration as the original checkpoint (L1 + LPIPS). The second, ft-MSE , was resumed from ft-EMA and uses EMA weights and was trained for another 280k steps using a different loss, with more emphasis on MSE reconstruction (MSE + 0.1 * LPIPS). It produces somewhat ``smoother'' outputs. The batch size for both versions was 192 (16 A100s, batch size 12 per GPU). To keep compatibility with existing models, only the decoder part was finetuned; the checkpoints can be used as a drop-in replacement for the existing autoencoder..
Original kl-f8 VAE vs f8-ft-EMA vs f8-ft-MSE
Evaluation
COCO 2017 (256x256, val, 5000 images)
| Model | train steps | rFID | PSNR | SSIM | PSIM | Link | Comments |
|---|---|---|---|---|---|---|---|
| original | 246803 | 4.99 | 23.4 +/- 3.8 | 0.69 +/- 0.14 | 1.01 +/- 0.28 | https://ommer-lab.com/files/latent-diffusion/kl-f8.zip | as used in SD |
| ft-EMA | 560001 | 4.42 | 23.8 +/- 3.9 | 0.69 +/- 0.13 | 0.96 +/- 0.27 | https://huggingface.co/stabilityai/sd-vae-ft-ema-original/resolve/main/vae-ft-ema-560000-ema-pruned.ckpt | slightly better overall, with EMA |
| ft-MSE | 840001 | 4.70 | 24.5 +/- 3.7 | 0.71 +/- 0.13 | 0.92 +/- 0.27 | https://huggingface.co/stabilityai/sd-vae-ft-mse-original/resolve/main/vae-ft-mse-840000-ema-pruned.ckpt | resumed with EMA from ft-EMA, emphasis on MSE (rec. loss = MSE + 0.1 * LPIPS), smoother outputs |
LAION-Aesthetics 5+ (256x256, subset, 10000 images)
| Model | train steps | rFID | PSNR | SSIM | PSIM | Link | Comments |
|---|---|---|---|---|---|---|---|
| original | 246803 | 2.61 | 26.0 +/- 4.4 | 0.81 +/- 0.12 | 0.75 +/- 0.36 | https://ommer-lab.com/files/latent-diffusion/kl-f8.zip | as used in SD |
| ft-EMA | 560001 | 1.77 | 26.7 +/- 4.8 | 0.82 +/- 0.12 | 0.67 +/- 0.34 | https://huggingface.co/stabilityai/sd-vae-ft-ema-original/resolve/main/vae-ft-ema-560000-ema-pruned.ckpt | slightly better overall, with EMA |
| ft-MSE | 840001 | 1.88 | 27.3 +/- 4.7 | 0.83 +/- 0.11 | 0.65 +/- 0.34 | https://huggingface.co/stabilityai/sd-vae-ft-mse-original/resolve/main/vae-ft-mse-840000-ema-pruned.ckpt | resumed with EMA from ft-EMA, emphasis on MSE (rec. loss = MSE + 0.1 * LPIPS), smoother outputs |
Visual
Visualization of reconstructions on 256x256 images from the COCO2017 validation dataset.
256x256: ft-EMA (left), ft-MSE (middle), original (right)
Runs of stabilityai sd-vae-ft-mse-original on huggingface.co
5
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
4
30-day runs
More Information About sd-vae-ft-mse-original huggingface.co Model
Url of sd-vae-ft-mse-original
sd-vae-ft-mse-original huggingface.co Url
Provider of sd-vae-ft-mse-original huggingface.co
Other API from stabilityai
Updated: October 31 2023
Total runs:
1.1M
Run Growth:
80.7K
Growth Rate:
7.54%
Total runs:
963.6K
Run Growth:
293.5K
Growth Rate:
30.46%
Total runs:
801.2K
Run Growth:
503.2K
Growth Rate:
62.80%
Total runs:
417.3K
Run Growth:
-16.6K
Growth Rate:
-3.99%
Total runs:
360.9K
Run Growth:
201.2K
Growth Rate:
55.76%
Total runs:
266.1K
Run Growth:
-266.9K
Growth Rate:
-100.30%
Total runs:
255.4K
Run Growth:
73.0K
Growth Rate:
28.58%
Total runs:
236.3K
Run Growth:
-77.1K
Growth Rate:
-43.16%
Total runs:
127.9K
Run Growth:
58.5K
Growth Rate:
45.73%
Total runs:
125.6K
Run Growth:
3.4K
Growth Rate:
3.56%
Total runs:
122.0K
Run Growth:
-102.1K
Growth Rate:
-83.70%
Total runs:
98.8K
Run Growth:
-8.5K
Growth Rate:
-8.63%
Total runs:
58.6K
Run Growth:
37.7K
Growth Rate:
35.58%
Total runs:
43.1K
Run Growth:
-10.1K
Growth Rate:
-23.46%
Total runs:
35.5K
Run Growth:
2.7K
Growth Rate:
7.51%
Total runs:
31.3K
Run Growth:
12.1K
Growth Rate:
38.74%
Total runs:
27.2K
Run Growth:
4.0K
Growth Rate:
14.64%
Total runs:
26.8K
Run Growth:
-13.4K
Growth Rate:
-50.04%
Total runs:
24.9K
Run Growth:
2.0K
Growth Rate:
8.00%
Total runs:
18.5K
Run Growth:
-1.9K
Growth Rate:
-10.24%
Total runs:
17.7K
Run Growth:
10.4K
Growth Rate:
58.51%
Total runs:
17.0K
Run Growth:
10.5K
Growth Rate:
61.42%
Total runs:
16.6K
Run Growth:
5.7K
Growth Rate:
34.49%
Total runs:
10.9K
Run Growth:
2.4K
Growth Rate:
22.30%
Total runs:
10.1K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
8.5K
Run Growth:
1.9K
Growth Rate:
22.81%
Total runs:
8.3K
Run Growth:
-864
Growth Rate:
-10.42%
Total runs:
8.3K
Run Growth:
-733
Growth Rate:
-8.87%
Total runs:
6.5K
Run Growth:
1.2K
Growth Rate:
19.07%
Total runs:
5.9K
Run Growth:
893
Growth Rate:
15.08%
Total runs:
5.4K
Run Growth:
-2.3K
Growth Rate:
-41.58%
Total runs:
5.3K
Run Growth:
644
Growth Rate:
12.19%
Total runs:
4.7K
Run Growth:
-1.0K
Growth Rate:
-21.39%
Total runs:
4.6K
Run Growth:
-1.4K
Growth Rate:
-29.80%
Total runs:
3.6K
Run Growth:
1.3K
Growth Rate:
34.94%
Total runs:
2.3K
Run Growth:
-677
Growth Rate:
-29.94%
Total runs:
2.0K
Run Growth:
-6.4K
Growth Rate:
-323.45%
Total runs:
1.9K
Run Growth:
135
Growth Rate:
7.03%
Total runs:
1.9K
Run Growth:
475
Growth Rate:
24.82%
Total runs:
1.6K
Run Growth:
111
Growth Rate:
6.94%
Total runs:
1.6K
Run Growth:
163
Growth Rate:
10.35%
Total runs:
1.6K
Run Growth:
-184
Growth Rate:
-11.86%
Total runs:
1.5K
Run Growth:
-427
Growth Rate:
-29.05%
Total runs:
1.3K
Run Growth:
406
Growth Rate:
31.30%
Total runs:
1.2K
Run Growth:
352
Growth Rate:
28.66%
Total runs:
1.1K
Run Growth:
202
Growth Rate:
17.81%
Total runs:
1.1K
Run Growth:
343
Growth Rate:
30.79%
Total runs:
1.1K
Run Growth:
-38
Growth Rate:
-3.47%
Total runs:
1.1K
Run Growth:
509
Growth Rate:
46.65%
Total runs:
1.0K
Run Growth:
-53
Growth Rate:
-5.07%
Total runs:
824
Run Growth:
138
Growth Rate:
16.75%
Total runs:
729
Run Growth:
0
Growth Rate:
0.00%
Total runs:
698
Run Growth:
312
Growth Rate:
44.70%
Total runs:
690
Run Growth:
92
Growth Rate:
100.00%
Total runs:
605
Run Growth:
384
Growth Rate:
63.47%
Total runs:
600
Run Growth:
0
Growth Rate:
0.00%
Total runs:
569
Run Growth:
0
Growth Rate:
0.00%
Total runs:
547
Run Growth:
439
Growth Rate:
80.26%
Total runs:
545
Run Growth:
43
Growth Rate:
7.89%
Total runs:
432
Run Growth:
271
Growth Rate:
100.00%
Total runs:
376
Run Growth:
201
Growth Rate:
53.46%
Total runs:
352
Run Growth:
71
Growth Rate:
20.17%
Total runs:
345
Run Growth:
342
Growth Rate:
93.44%
Total runs:
323
Run Growth:
410
Growth Rate:
92.76%
Total runs:
323
Run Growth:
-18
Growth Rate:
-5.57%
Total runs:
306
Run Growth:
274
Growth Rate:
100.00%
Total runs:
257
Run Growth:
-5.2K
Growth Rate:
-2031.91%
Total runs:
209
Run Growth:
-106
Growth Rate:
-50.72%
Total runs:
203
Run Growth:
-189
Growth Rate:
-93.10%
Total runs:
186
Run Growth:
-279
Growth Rate:
-150.00%
Total runs:
169
Run Growth:
48
Growth Rate:
100.00%
Total runs:
141
Run Growth:
30
Growth Rate:
21.28%
Total runs:
126
Run Growth:
-265
Growth Rate:
-210.32%
Total runs:
117
Run Growth:
-330
Growth Rate:
-282.05%
Total runs:
112
Run Growth:
79
Growth Rate:
70.54%
Total runs:
94
Run Growth:
24
Growth Rate:
25.53%
Total runs:
73
Run Growth:
25
Growth Rate:
100.00%
Total runs:
66
Run Growth:
-30
Growth Rate:
-45.45%
Total runs:
39
Run Growth:
-28
Growth Rate:
-71.79%
Total runs:
20
Run Growth:
-21
Growth Rate:
-105.00%
Total runs:
7
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5
Run Growth:
-14
Growth Rate:
-280.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%