stabilityai / StableBeluga1-Delta

huggingface.co
Total runs: 1.1K
24-hour runs: 0
7-day runs: 289
30-day runs: -53
Model's Last Updated: 7月 27 2023
text-generation

Introduction of StableBeluga1-Delta

Model Details of StableBeluga1-Delta

Stable Belgua 1

Model Description

Stable Beluga 1 is a Llama65B model fine-tuned on an Orca style Dataset

Usage
Apply Delta Weights

Stable Beluga 1 cannot be used from the stabilityai/StableBeluga1-Delta weights alone. To obtain the correct model, one must add back the difference between LLaMA 65B and stabilityai/StableBeluga1-Delta weights. We provide the apply_delta.py script to automate the conversion, which you can run as:

python3 apply_delta.py --base-model-path /path/to/model_weights/llama-65b --target-model-path StableBeluga1 --delta-path stabilityai/StableBeluga1-Delta

Start chatting with Stable Beluga 1 using the following code snippet:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("your_path_to_StableBeluga1", use_fast=False)
model = AutoModelForCausalLM.from_pretrained("your_path_to_StableBeluga1", torch_dtype=torch.float16, low_cpu_mem_usage=True, device_map="auto")

system_prompt = "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n"
system_prompt += "### Instruction:\nYou are Stable Beluga, an AI that follows instructions extremely well. Help as much as you can. Remember, be safe, and don't do anything illegal.\n\n"

message = "Write me a poem please"
prompt = f"{system_prompt}### Input: {message}\n\n### Response:\n"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, do_sample=True, top_p=0.95, top_k=0, max_new_tokens=256)

print(tokenizer.decode(output[0], skip_special_tokens=True))

Stable Beluga 1 should be used with prompts formatted similarly to Alpaca as below:

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

## Instruction:
This is a system prompt, please behave and help the user.

### Input:
Your prompt here

### Response:
The output of Stable Beluga 1
Model Details
  • Developed by : Stability AI
  • Model type : Stable Beluga 1 is an auto-regressive language model fine-tuned on LLaMA65B.
  • Language(s) : English
  • Library : HuggingFace Transformers
  • License : Fine-tuned checkpoints ( StableBeluga1 ) is licensed under the Non-Commercial Creative Commons license ( CC BY-NC-4.0 )
  • Contact : For questions and comments about the model, please email lm@stability.ai
Training Dataset

Stable Beluga 1 is trained on our internal Orca-style dataset

Training Procedure

Models are learned via supervised fine-tuning on the aforementioned datasets, trained in mixed-precision (BF16), and optimized with AdamW. We outline the following hyperparameters:

Dataset Batch Size Learning Rate Learning Rate Decay Warm-up Weight Decay Betas
Orca pt1 packed 512 3e-5 Cosine to 3e-6 100 1e-6 (0.9, 0.95)
Orca pt2 unpacked 512 3e-5 Cosine to 3e-6 100 1e-6 (0.9, 0.95)
Use and Limitations
Ethical Considerations and Limitations

Beluga is a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Beluga's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of Beluga, developers should perform safety testing and tuning tailored to their specific applications of the model.

Citations
@misc{touvron2023llama,
      title={LLaMA: Open and Efficient Foundation Language Models}, 
      author={Hugo Touvron and Thibaut Lavril and Gautier Izacard and Xavier Martinet and Marie-Anne Lachaux and Timothée Lacroix and Baptiste Rozière and Naman Goyal and Eric Hambro and Faisal Azhar and Aurelien Rodriguez and Armand Joulin and Edouard Grave and Guillaume Lample},
      year={2023},
      eprint={2302.13971},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
@misc{mukherjee2023orca,
      title={Orca: Progressive Learning from Complex Explanation Traces of GPT-4}, 
      author={Subhabrata Mukherjee and Arindam Mitra and Ganesh Jawahar and Sahaj Agarwal and Hamid Palangi and Ahmed Awadallah},
      year={2023},
      eprint={2306.02707},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
@misc{alpaca,
  author = {Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto },
  title = {Stanford Alpaca: An Instruction-following LLaMA model},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
}

Runs of stabilityai StableBeluga1-Delta on huggingface.co

1.1K
Total runs
0
24-hour runs
122
3-day runs
289
7-day runs
-53
30-day runs

More Information About StableBeluga1-Delta huggingface.co Model

More StableBeluga1-Delta license Visit here:

https://choosealicense.com/licenses/cc-by-nc-4.0

StableBeluga1-Delta huggingface.co

StableBeluga1-Delta huggingface.co is an AI model on huggingface.co that provides StableBeluga1-Delta's model effect (), which can be used instantly with this stabilityai StableBeluga1-Delta model. huggingface.co supports a free trial of the StableBeluga1-Delta model, and also provides paid use of the StableBeluga1-Delta. Support call StableBeluga1-Delta model through api, including Node.js, Python, http.

StableBeluga1-Delta huggingface.co Url

https://huggingface.co/stabilityai/StableBeluga1-Delta

stabilityai StableBeluga1-Delta online free

StableBeluga1-Delta huggingface.co is an online trial and call api platform, which integrates StableBeluga1-Delta's modeling effects, including api services, and provides a free online trial of StableBeluga1-Delta, you can try StableBeluga1-Delta online for free by clicking the link below.

stabilityai StableBeluga1-Delta online free url in huggingface.co:

https://huggingface.co/stabilityai/StableBeluga1-Delta

StableBeluga1-Delta install

StableBeluga1-Delta is an open source model from GitHub that offers a free installation service, and any user can find StableBeluga1-Delta on GitHub to install. At the same time, huggingface.co provides the effect of StableBeluga1-Delta install, users can directly use StableBeluga1-Delta installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

StableBeluga1-Delta install url in huggingface.co:

https://huggingface.co/stabilityai/StableBeluga1-Delta

Url of StableBeluga1-Delta

StableBeluga1-Delta huggingface.co Url

Provider of StableBeluga1-Delta huggingface.co

stabilityai
ORGANIZATIONS

Other API from stabilityai

huggingface.co

Total runs: 143.2K
Run Growth: 8.3K
Growth Rate: 5.81%
Updated: 8月 04 2023
huggingface.co

Total runs: 137.2K
Run Growth: 16.4K
Growth Rate: 11.93%
Updated: 7月 10 2024
huggingface.co

Total runs: 34.3K
Run Growth: 3.4K
Growth Rate: 10.04%
Updated: 8月 09 2024
huggingface.co

Total runs: 378
Run Growth: -97.9K
Growth Rate: -25898.41%
Updated: 8月 03 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: 7月 10 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: 4月 13 2024