Total runs:
5.9M
Run Growth:
628.0K
Growth Rate:
10.69%
BAAI / Bunny-v1_0-3B
Model's Last Updated: November 28 2024
text-generation
Introduction of Bunny-v1_0-3B
Model Details of Bunny-v1_0-3B
Model Card
📖 Technical report | 🏠 Code | 🐰 Demo
This is the merged weights of bunny-phi-2-siglip-lora .
Bunny is a family of lightweight but powerful multimodal models. It offers multiple plug-and-play vision encoders, like EVA-CLIP, SigLIP and language backbones, including Llama-3-8B, Phi-1.5, StableLM-2, Qwen1.5, MiniCPM and Phi-2. To compensate for the decrease in model size, we construct more informative training data by curated selection from a broader data source. Remarkably, our Bunny-v1.0-3B model built upon SigLIP and Phi-2 outperforms the state-of-the-art MLLMs, not only in comparison with models of similar size but also against larger MLLM frameworks (7B), and even achieves performance on par with 13B models.
The model is pretrained on LAION-2M and finetuned on Bunny-695K. More details about this model can be found in GitHub .
Quickstart
Here we show a code snippet to show you how to use the model with transformers.
Before running the snippet, you need to install the following dependencies:
pip install torch transformers accelerate pillow
If the CUDA memory is enough, it would be faster to execute this snippet by setting
CUDA_VISIBLE_DEVICES=0
.
Users especially those in Chinese mainland may want to refer to a HuggingFace mirror site .
import torch
import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
from PIL import Image
import warnings
# disable some warnings
transformers.logging.set_verbosity_error()
transformers.logging.disable_progress_bar()
warnings.filterwarnings('ignore')
# set device
device = 'cuda' # or cpu
torch.set_default_device(device)
# create model
model = AutoModelForCausalLM.from_pretrained(
'BAAI/Bunny-v1_0-3B',
torch_dtype=torch.float16, # float32 for cpu
device_map='auto',
trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(
'BAAI/Bunny-v1_0-3B',
trust_remote_code=True)
# text prompt
prompt = 'Why is the image funny?'
text = f"A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: <image>\n{prompt} ASSISTANT:"
text_chunks = [tokenizer(chunk).input_ids for chunk in text.split('<image>')]
input_ids = torch.tensor(text_chunks[0] + [-200] + text_chunks[1], dtype=torch.long).unsqueeze(0).to(device)
# image, sample images can be found in images folder
image = Image.open('example_2.png')
image_tensor = model.process_images([image], model.config).to(dtype=model.dtype, device=device)
# generate
output_ids = model.generate(
input_ids,
images=image_tensor,
max_new_tokens=100,
use_cache=True
repetition_penalty=1.0 # increase this to avoid chattering
)[0]
print(tokenizer.decode(output_ids[input_ids.shape[1]:], skip_special_tokens=True).strip())
License
This project utilizes certain datasets and checkpoints that are subject to their respective original licenses. Users must comply with all terms and conditions of these original licenses. The content of this project itself is licensed under the Apache license 2.0.
Runs of BAAI Bunny-v1_0-3B on huggingface.co
2.0K
Total runs
-33
24-hour runs
-78
3-day runs
133
7-day runs
-188
30-day runs
More Information About Bunny-v1_0-3B huggingface.co Model
Url of Bunny-v1_0-3B
Bunny-v1_0-3B huggingface.co Url
Provider of Bunny-v1_0-3B huggingface.co
Other API from BAAI
Updated: February 22 2024
Total runs:
2.3M
Run Growth:
-1.8M
Growth Rate:
-75.18%
Total runs:
1.6M
Run Growth:
-512.9K
Growth Rate:
-31.42%
Total runs:
1.1M
Run Growth:
-47.0K
Growth Rate:
-4.33%
Total runs:
983.7K
Run Growth:
160.9K
Growth Rate:
16.95%
Total runs:
798.7K
Run Growth:
153.9K
Growth Rate:
19.41%
Total runs:
766.3K
Run Growth:
-41.5K
Growth Rate:
-5.55%
Total runs:
197.8K
Run Growth:
21.7K
Growth Rate:
10.99%
Total runs:
138.8K
Run Growth:
74.2K
Growth Rate:
53.10%
Total runs:
123.8K
Run Growth:
-12.9K
Growth Rate:
-10.77%
Total runs:
25.9K
Run Growth:
-742
Growth Rate:
-2.86%
Total runs:
14.8K
Run Growth:
4.1K
Growth Rate:
27.88%
Total runs:
8.3K
Run Growth:
-11.1K
Growth Rate:
-84.72%
Total runs:
8.2K
Run Growth:
-1.4K
Growth Rate:
-17.96%
Total runs:
7.5K
Run Growth:
-1.5K
Growth Rate:
-20.32%
Total runs:
6.1K
Run Growth:
-1.5K
Growth Rate:
-23.92%
Total runs:
5.4K
Run Growth:
-1.6K
Growth Rate:
-29.82%
Total runs:
5.3K
Run Growth:
-1.8K
Growth Rate:
-33.31%
Total runs:
5.3K
Run Growth:
-1.8K
Growth Rate:
-33.28%
Total runs:
4.5K
Run Growth:
1.9K
Growth Rate:
42.59%
Total runs:
4.0K
Run Growth:
-12.2K
Growth Rate:
-313.51%
Total runs:
3.8K
Run Growth:
471
Growth Rate:
12.76%
Total runs:
3.6K
Run Growth:
-1.5K
Growth Rate:
-42.37%
Total runs:
3.5K
Run Growth:
-1.7K
Growth Rate:
-47.49%
Total runs:
2.5K
Run Growth:
196
Growth Rate:
7.14%
Total runs:
1.8K
Run Growth:
-36
Growth Rate:
-2.12%
Total runs:
1.2K
Run Growth:
917
Growth Rate:
78.51%
Total runs:
892
Run Growth:
25
Growth Rate:
86.21%
Total runs:
803
Run Growth:
-633
Growth Rate:
-29.46%
Total runs:
664
Run Growth:
0
Growth Rate:
0.00%
Total runs:
644
Run Growth:
0
Growth Rate:
0.00%
Total runs:
427
Run Growth:
81
Growth Rate:
18.84%
Total runs:
412
Run Growth:
-170
Growth Rate:
-41.98%
Total runs:
272
Run Growth:
-339
Growth Rate:
-130.89%
Total runs:
189
Run Growth:
37
Growth Rate:
19.47%
Total runs:
145
Run Growth:
-270
Growth Rate:
-163.64%
Total runs:
111
Run Growth:
-95
Growth Rate:
-150.79%
Total runs:
108
Run Growth:
82
Growth Rate:
75.93%
Total runs:
70
Run Growth:
11
Growth Rate:
17.74%
Total runs:
47
Run Growth:
-10.2K
Growth Rate:
-21691.49%
Total runs:
40
Run Growth:
3
Growth Rate:
100.00%
Total runs:
38
Run Growth:
3
Growth Rate:
100.00%
Total runs:
29
Run Growth:
-440
Growth Rate:
-427.18%
Total runs:
25
Run Growth:
-34
Growth Rate:
-113.33%
Total runs:
21
Run Growth:
1
Growth Rate:
100.00%
Total runs:
16
Run Growth:
-22
Growth Rate:
-115.79%