BAAI / EVA

Table of Contents

• Image Classification
  • Summary of EVA's image classification performance
• Video Classification
• Object Detection & Instance Segmentation
  • COCO 2017 (single-scale evaluation on val set)
  • LVIS v1.0 (single-scale evaluation on val set)
• Semantic Segmentation
  • COCO-Stuff-164K
  • ADE20K
• EVA-CLIP
• Citation
• License
• Contact

Image Classification

We provide all pre-trained & fine-tuned EVAs for the community. The following table summarizes the basic statistics of MIM pre-trained EVA and image classification EVA.

• eva_psz14to16 model interpolates the kernel size of patch_embed from 14x14 to 16x16 . This is useful for object detection, instance segmentation & semantic segmentation, etc . See interpolate_patch_14to16.py for implementation details.
• For MIM pre-trained EVA and EVA-CLIP, we use deepspeed fp16 format. IN-1K fine-tuned EVA weights are larger ( 4GB v.s. 2GB ) because ema updates models with fp32 format. The weights of other downstream tasks are also with fp32 format.

Summary of EVA's image classification performance

Video Classification

Object Detection & Instance Segmentation

COCO 2017 (single-scale evaluation on val set)

LVIS v1.0 (single-scale evaluation on val set)

Semantic Segmentation

COCO-Stuff-164K

ADE20K

EVA-CLIP

The ImageNet-1K zero-shot classification performance is higher than our paper ( 78.5 v.s. 78.2 ) because of longer training.

We choose to train a 1.3B CLIP model, not because it is easy, but because it is hard. Please refer to this note for a glance of the challenges in training very large CLIP.

To our knowledge, EVA-CLIP is the largest performant open-sourced CLIP model evaluated via zero-shot classification performance. We will updates the results in our paper soon. For more details of EVA-CLIP, please refer to Section 2.3.5 of our paper .

We hope open-sourcing EVA-CLIP can facilitate future research in multi-modal learning, representation leaning, AIGC, etc .

Citation

If you find our work helpful, please star this repo and cite the related articles. Thanks for your support!

@article{EVA,
  title={EVA: Exploring the Limits of Masked Visual Representation Learning at Scale},
  author={Fang, Yuxin and Wang, Wen and Xie, Binhui and Sun, Quan and Wu, Ledell and Wang, Xinggang and Huang, Tiejun and Wang, Xinlong and Cao, Yue},
  journal={arXiv preprint arXiv:2211.07636},
  year={2022}
}

License

The content of this project itself is licensed under the MIT License.

Contact

For help or issues using EVA, please open a GitHub issue .

We are hiring at all levels at BAAI Vision Team, including full-time researchers, engineers and interns. If you are interested in working with us on foundation model, self-supervised learning and multimodal learning , please contact Yue Cao ( [email protected] ) and Xinlong Wang ( [email protected] ).