Model Details of japanese-stablelm-2-instruct-1_6b
Japanese Stable LM 2 Instruct 1.6B
A beautiful anime-like hummingbird flying with the text "Japanese Stable LM 2" below it, with a lofi anime landscape of Mount Fuji forming the outline of the text "Japanese Stable LM 2" —
Stable Diffusion 3
Japanese Stable LM 2 Instruct 1.6B
is a 1.6B-parameter decoder-only language model based on
Stable LM 2 1.6B
that has been fine-tuned on a diverse collection of Japanese data, with the intent of maximizing downstream performance on Japanese language tasks.
Usage
Japanese Stable LM 2 Instruct 1.6B uses the following instruction format:
This format is also available through the tokenizer's
apply_chat_template
method:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "stabilityai/japanese-stablelm-2-instruct-1_6b"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
# The next line may need to be modified depending on the environment
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
low_cpu_mem_usage=True,
device_map="auto",
trust_remote_code=True,
)
prompt = [
{"role": "system", "content": "あなたは役立つアシスタントです。"},
{"role": "user", "content": "「情けは人のためならず」ということわざの意味を小学生でも分かるように教えてください。"},
]
inputs = tokenizer.apply_chat_template(
prompt,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
# this is for reproducibility.# feel free to change to get different result
seed = 23
torch.manual_seed(seed)
tokens = model.generate(
inputs,
max_new_tokens=128,
temperature=0.99,
top_p=0.95,
do_sample=True,
)
out = tokenizer.decode(tokens[0], skip_special_tokens=False)
print(out)
We suggest playing with different generation config (
top_p
,
repetition_penalty
etc) to find the best setup for your tasks. For example, use higher temperature for roleplay task, lower temperature for reasoning.
Model Details
Model type
:
Japanese Stable LM 2 Instruct 1.6B
models are auto-regressive language models based on the transformer decoder architecture.
Contact
: For questions and comments about the model, please join
Stable Community Japan
. For future announcements / information about Stability AI models, research, and events, please follow
@StabilityAI_JP
.
Model Architecture
The model is a decoder-only transformer similar to the LLaMA (
Touvron et al., 2023
) architecture with the following modifications:
Parameters
Hidden Size
Layers
Heads
Sequence Length
1,644,417,024
2048
24
32
4096
Position Embeddings
: Rotary Position Embeddings (
Su et al., 2021
) applied to the first 25% of head embedding dimensions for improved throughput following
Black et al. (2022)
.
Biases
: We remove all bias terms from the feed-forward networks and multi-head self-attention layers, except for the biases of the query, key, and value projections (
Bai et al., 2023
).
Tokenizer
: We use Arcade100k, a BPE tokenizer extended from OpenAI's
tiktoken.cl100k_base
. We split digits into individual tokens following findings by
Liu & Low (2023)
.
Training Dataset
The following datasets were used for the instruction training.
The model is intended to be used by all individuals as a foundation for application-specific fine-tuning without strict limitations on commercial use. For commercial use, please refer to
https://stability.ai/license
.
Limitations and Bias
The pre-training dataset may have contained offensive or inappropriate content even after applying data cleansing filters which can be reflected in the model generated text. We recommend users exercise reasonable caution when using these models in production systems. Do not use the model for any applications that may cause harm or distress to individuals or groups.
Authors
This model was developed by the Research & Development team at Stability AI Japan, and the development was led by Meng Lee (@leemeng) and Naoki Orii (@mrorii). The members of the team are as follows:
@misc{JapaneseStableLM2Instruct1.6B,
url={[https://huggingface.co/stabilityai/japanese-stablelm-2-instruct-1_6b](https://huggingface.co/stabilityai/japanese-stablelm-instruct-2-1_6b)},
title={Japanese Stable LM 2 Instruct 1.6B},
author={Lee, Meng and Nakamura, Fujiki and McCann, Paul and Orii, Naoki and Shibui, Yusuke and Phung, Duy and Zhuravinskyi, Maksym and Mahan, Dakota and Chi, Jerry}
}
Runs of stabilityai japanese-stablelm-2-instruct-1_6b on huggingface.co
66
Total runs
0
24-hour runs
0
3-day runs
-51
7-day runs
-30
30-day runs
More Information About japanese-stablelm-2-instruct-1_6b huggingface.co Model
More japanese-stablelm-2-instruct-1_6b license Visit here:
japanese-stablelm-2-instruct-1_6b huggingface.co is an AI model on huggingface.co that provides japanese-stablelm-2-instruct-1_6b's model effect (), which can be used instantly with this stabilityai japanese-stablelm-2-instruct-1_6b model. huggingface.co supports a free trial of the japanese-stablelm-2-instruct-1_6b model, and also provides paid use of the japanese-stablelm-2-instruct-1_6b. Support call japanese-stablelm-2-instruct-1_6b model through api, including Node.js, Python, http.
japanese-stablelm-2-instruct-1_6b huggingface.co is an online trial and call api platform, which integrates japanese-stablelm-2-instruct-1_6b's modeling effects, including api services, and provides a free online trial of japanese-stablelm-2-instruct-1_6b, you can try japanese-stablelm-2-instruct-1_6b online for free by clicking the link below.
stabilityai japanese-stablelm-2-instruct-1_6b online free url in huggingface.co:
japanese-stablelm-2-instruct-1_6b is an open source model from GitHub that offers a free installation service, and any user can find japanese-stablelm-2-instruct-1_6b on GitHub to install. At the same time, huggingface.co provides the effect of japanese-stablelm-2-instruct-1_6b install, users can directly use japanese-stablelm-2-instruct-1_6b installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
japanese-stablelm-2-instruct-1_6b install url in huggingface.co: