Total runs:
2.8M
Run Growth:
716.7K
Growth Rate:
25.99%
stabilityai / japanese-stablelm-base-beta-70b
Model's Last Updated: December 19 2023
text-generation
Introduction of japanese-stablelm-base-beta-70b
Model Details of japanese-stablelm-base-beta-70b
Japanese-StableLM-Base-Beta-70B
A cute robot wearing a kimono writes calligraphy with one single brush — Stable Diffusion XL
Model Description
japanese-stablelm-base-beta-70b
is a 70B-parameter decoder-only language model based on
Llama-2-70b
that has been fine-tuned on a diverse collection of Japanese data, with the intent of maximizing downstream performance on Japanese language tasks.
For an instruction-following model, check Japanese-StableLM-Instruct-Beta-70B . The base and instruct models are also available in smaller 7b sizes. For a model that has faster inference times, see Japanese-StableLM-Base-JA_Vocab-Beta-7B , or the instruction-following version .
Usage
First install additional dependencies in requirements.txt :
pip install -r requirements.txt
Then start generating text with
japanese-stablelm-base-beta-70b
by using the following code snippet:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "stabilityai/japanese-stablelm-base-beta-70b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
# The next line may need to be modified depending on the environment
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, low_cpu_mem_usage=True, device_map="auto")
prompt = """
AI で科学研究を加速するには、
""".strip()
input_ids = tokenizer.encode(
prompt,
add_special_tokens=True,
return_tensors="pt"
)
# this is for reproducibility.
# feel free to change to get different result
seed = 23
torch.manual_seed(seed)
tokens = model.generate(
input_ids.to(device=model.device),
max_new_tokens=128,
temperature=0.99,
top_p=0.95,
do_sample=True,
)
out = tokenizer.decode(tokens[0], skip_special_tokens=True)
print(out)
We suggest playing with different generation config (
top_p
,
repetition_penalty
etc) to find the best setup for your tasks. For example, use higher temperature for roleplay task, lower temperature for reasoning.
Model Details
-
Model type
:
japanese-stablelm-base-beta-70bmodel is an auto-regressive language model based on the Llama2 transformer architecture. - Language(s) : Japanese
- License : Llama2 Community License .
- Contact : For questions and comments about the model, please join Stable Community Japan . For future announcements / information about Stability AI models, research, and events, please follow https://twitter.com/StabilityAI_JP .
Training Dataset
Roughly 100B tokens from a mixture of the following corpora were used for continued pre-training.
- Japanese/English Wikipedia
- Japanese mc4
- Japanese CC-100
- Japanese OSCAR
- SlimPajama (excluding the Books3 subset)
Use and Limitations
Intended Use
The model is intended to be used by all individuals as a foundation for application-specific fine-tuning without strict limitations on commercial use.
Limitations and bias
The pre-training dataset may have contained offensive or inappropriate content even after applying data cleansing filters which can be reflected in the model generated text. We recommend users exercise reasonable caution when using these models in production systems. Do not use the model for any applications that may cause harm or distress to individuals or groups.
Authors
This model was developed by the Research & Development team at Stability AI Japan, and the development was co-led by Takuya Akiba and Meng Lee . The members of the team are as follows:
Acknowledgements
We thank Meta Research for releasing Llama 2 under an open license for others to build on.
We are grateful for the contributions of the EleutherAI Polyglot-JA team in helping us to collect a large amount of pre-training data in Japanese. Polyglot-JA members includes Hyunwoong Ko (Project Lead), Fujiki Nakamura (originally started this project when he commited to the Polyglot team), Yunho Mo, Minji Jung, KeunSeok Im, and Su-Kyeong Jang.
We are also appreciative of AI Novelist/Sta (Bit192, Inc.) and the numerous contributors from Stable Community Japan for assisting us in gathering a large amount of high-quality Japanese textual data for model training.
How to cite
@misc{JapaneseStableLMBaseBeta70B,
url={[https://huggingface.co/stabilityai/japanese-stablelm-base-beta-70b](https://huggingface.co/stabilityai/japanese-stablelm-base-beta-70b)},
title={Japanese StableLM Base Beta 70B},
author={Lee, Meng and Nakamura, Fujiki and Shing, Makoto and McCann, Paul and Akiba, Takuya and Orii, Naoki}
}
Runs of stabilityai japanese-stablelm-base-beta-70b on huggingface.co
1.5K
Total runs
59
24-hour runs
172
3-day runs
379
7-day runs
814
30-day runs
More Information About japanese-stablelm-base-beta-70b huggingface.co Model
Url of japanese-stablelm-base-beta-70b
japanese-stablelm-base-beta-70b huggingface.co Url
Provider of japanese-stablelm-base-beta-70b huggingface.co
Other API from stabilityai
Updated: October 31 2023
Total runs:
1.2M
Run Growth:
539.6K
Growth Rate:
46.56%
Total runs:
1.0M
Run Growth:
63.6K
Growth Rate:
6.11%
Total runs:
873.3K
Run Growth:
663.2K
Growth Rate:
75.95%
Total runs:
476.8K
Run Growth:
167.9K
Growth Rate:
35.22%
Total runs:
394.6K
Run Growth:
233.9K
Growth Rate:
59.29%
Total runs:
277.2K
Run Growth:
-60.2K
Growth Rate:
-21.73%
Total runs:
257.7K
Run Growth:
76.4K
Growth Rate:
29.65%
Total runs:
251.5K
Run Growth:
-77.1K
Growth Rate:
-43.16%
Total runs:
181.2K
Run Growth:
57.9K
Growth Rate:
31.98%
Total runs:
136.9K
Run Growth:
69.8K
Growth Rate:
50.98%
Total runs:
136.8K
Run Growth:
3.4K
Growth Rate:
3.56%
Total runs:
123.5K
Run Growth:
-44.3K
Growth Rate:
-35.87%
Total runs:
121.5K
Run Growth:
-18.6K
Growth Rate:
-15.29%
Total runs:
108.2K
Run Growth:
91.7K
Growth Rate:
84.78%
Total runs:
88.1K
Run Growth:
-9.4K
Growth Rate:
-10.69%
Total runs:
48.0K
Run Growth:
-3.3K
Growth Rate:
-6.92%
Total runs:
42.9K
Run Growth:
37.7K
Growth Rate:
35.58%
Total runs:
34.9K
Run Growth:
4.1K
Growth Rate:
11.65%
Total runs:
29.7K
Run Growth:
1.7K
Growth Rate:
5.67%
Total runs:
27.6K
Run Growth:
5.6K
Growth Rate:
20.29%
Total runs:
24.2K
Run Growth:
2.0K
Growth Rate:
8.27%
Total runs:
20.2K
Run Growth:
959
Growth Rate:
4.75%
Total runs:
19.5K
Run Growth:
13.7K
Growth Rate:
70.25%
Total runs:
16.8K
Run Growth:
8.6K
Growth Rate:
51.11%
Total runs:
16.5K
Run Growth:
5.8K
Growth Rate:
35.20%
Total runs:
11.8K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11.4K
Run Growth:
3.1K
Growth Rate:
26.84%
Total runs:
9.3K
Run Growth:
602
Growth Rate:
6.46%
Total runs:
8.6K
Run Growth:
1.7K
Growth Rate:
19.55%
Total runs:
8.5K
Run Growth:
2.5K
Growth Rate:
29.49%
Total runs:
6.8K
Run Growth:
1.5K
Growth Rate:
22.00%
Total runs:
6.4K
Run Growth:
1.5K
Growth Rate:
23.90%
Total runs:
6.1K
Run Growth:
2.0K
Growth Rate:
32.55%
Total runs:
6.1K
Run Growth:
187
Growth Rate:
3.08%
Total runs:
4.6K
Run Growth:
-313
Growth Rate:
-6.82%
Total runs:
4.4K
Run Growth:
-962
Growth Rate:
-21.67%
Total runs:
4.2K
Run Growth:
1.9K
Growth Rate:
45.93%
Total runs:
2.5K
Run Growth:
1.3K
Growth Rate:
50.63%
Total runs:
2.5K
Run Growth:
1.0K
Growth Rate:
41.62%
Total runs:
2.3K
Run Growth:
-368
Growth Rate:
-16.28%
Total runs:
2.0K
Run Growth:
718
Growth Rate:
36.39%
Total runs:
2.0K
Run Growth:
830
Growth Rate:
42.48%
Total runs:
1.9K
Run Growth:
-6.7K
Growth Rate:
-349.79%
Total runs:
1.8K
Run Growth:
130
Growth Rate:
7.35%
Total runs:
1.7K
Run Growth:
961
Growth Rate:
56.26%
Total runs:
1.6K
Run Growth:
899
Growth Rate:
54.68%
Total runs:
1.6K
Run Growth:
-287
Growth Rate:
-17.95%
Total runs:
1.5K
Run Growth:
586
Growth Rate:
39.54%
Total runs:
1.5K
Run Growth:
698
Growth Rate:
47.55%
Total runs:
1.4K
Run Growth:
580
Growth Rate:
40.28%
Total runs:
1.2K
Run Growth:
602
Growth Rate:
50.76%
Total runs:
966
Run Growth:
384
Growth Rate:
39.75%
Total runs:
775
Run Growth:
0
Growth Rate:
0.00%
Total runs:
724
Run Growth:
584
Growth Rate:
80.66%
Total runs:
712
Run Growth:
92
Growth Rate:
100.00%
Total runs:
700
Run Growth:
425
Growth Rate:
60.71%
Total runs:
629
Run Growth:
89
Growth Rate:
14.15%
Total runs:
600
Run Growth:
0
Growth Rate:
0.00%
Total runs:
569
Run Growth:
0
Growth Rate:
0.00%
Total runs:
466
Run Growth:
271
Growth Rate:
100.00%
Total runs:
458
Run Growth:
316
Growth Rate:
69.00%
Total runs:
443
Run Growth:
152
Growth Rate:
34.31%
Total runs:
427
Run Growth:
410
Growth Rate:
92.76%
Total runs:
426
Run Growth:
91
Growth Rate:
21.36%
Total runs:
369
Run Growth:
-4.9K
Growth Rate:
-1332.25%
Total runs:
311
Run Growth:
-3
Growth Rate:
-0.96%
Total runs:
308
Run Growth:
342
Growth Rate:
93.44%
Total runs:
257
Run Growth:
-39
Growth Rate:
-15.18%
Total runs:
218
Run Growth:
-57
Growth Rate:
-26.15%
Total runs:
149
Run Growth:
58
Growth Rate:
38.93%
Total runs:
140
Run Growth:
-135
Growth Rate:
-96.43%
Total runs:
131
Run Growth:
274
Growth Rate:
100.00%
Total runs:
127
Run Growth:
98
Growth Rate:
77.17%
Total runs:
125
Run Growth:
-172
Growth Rate:
-137.60%
Total runs:
107
Run Growth:
58
Growth Rate:
54.21%
Total runs:
99
Run Growth:
-23
Growth Rate:
-23.23%
Total runs:
98
Run Growth:
48
Growth Rate:
100.00%
Total runs:
87
Run Growth:
25
Growth Rate:
100.00%
Total runs:
35
Run Growth:
8
Growth Rate:
22.86%
Total runs:
30
Run Growth:
7
Growth Rate:
23.33%
Total runs:
17
Run Growth:
3
Growth Rate:
17.65%
Total runs:
5
Run Growth:
0
Growth Rate:
0.00%
Total runs:
4
Run Growth:
1
Growth Rate:
25.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%