bigcode / astraios-lora

huggingface.co
Total runs: 32
24-hour runs: -1
7-day runs: 9
30-day runs: 26
Model's Last Updated: 2024年1月1日

Introduction of astraios-lora

Model Details of astraios-lora

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models

Astraios

Table of Contents

  1. Model Summary
  2. Use
  3. Training
  4. Citation

Model Summary

Astraios-LoRA is an instruction tuned model with 15.5B parameters created by finetuning StarCoderBase on CommitPackFT & OASST as described in the Astraios paper.

  • Repository: bigcode-project/astraios
  • Paper: Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
  • Languages: 80+ Programming languages
  • ✨Astraios:
    Data CommitPackFT+OASST Filtered version of CommitPack and OASST for high-quality commit messages that resemble instructions
    Model Astraios-1B Collection of StarCoderBase-1B models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-3B Collection of StarCoderBase-3B (3B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-7B Collection of StarCoderBase-7B (7B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-16B Collection of StarCoderBase-16B (16B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Evaluation BigCloneBench Dataset for clone detection; We use 2,000 samples for evaluation
    Devign Dataset for defect detection; We use 2,000 samples for evaluation
    HumanEvalPack Extension of OpenAI's HumanEval to cover 3 scenarios across 6 languages
    ReCode Dataset for the robustness of code generation, covering 4 variants
    Asleep At The Keyboard Datasets for security of code generation; We use DoW for evaluation

Use

Intended use

The model follows instructions provided in the input. You should always preface your input with "Question: " and finish it with "Answer:", for example: "Question: Please write a function in Python that performs bubble sort.

Answer:"

Feel free to share your generations in the Community tab!

Generation
# pip install -q transformers
# pip install -e git+https://github.com/bigcode-project/astraios#subdirectory=peft
from peft import PeftModel 
from transformers import AutoModelForCausalLM, AutoTokenizer

peft_checkpoint = "bigcode/astraios-lora"
checkpoint = "bigcode/starcoderbase"
model = AutoModelForCausalLM.from_pretrained(checkpoint)
model = PeftModel.from_pretrained(model, peft_checkpoint)
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

inputs = tokenizer.encode("Question: Please write a function in Python that performs bubble sort.

Answer:", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

Training

Model
  • Architecture: GPT-2 model with multi-query attention and Fill-in-the-Middle objective
  • Steps: 250k pretraining & 200 instruction tuning
  • Precision: fp32
Hardware
  • Pretraining:
    • GPUs: 512 Tesla A100
    • Training time: 24 days
  • Instruction tuning:
    • GPUs: 8 Tesla A100
Software

Citation


Runs of bigcode astraios-lora on huggingface.co

32
Total runs
-1
24-hour runs
-2
3-day runs
9
7-day runs
26
30-day runs

More Information About astraios-lora huggingface.co Model

More astraios-lora license Visit here:

https://choosealicense.com/licenses/bigcode-openrail-m

astraios-lora huggingface.co

astraios-lora huggingface.co is an AI model on huggingface.co that provides astraios-lora's model effect (), which can be used instantly with this bigcode astraios-lora model. huggingface.co supports a free trial of the astraios-lora model, and also provides paid use of the astraios-lora. Support call astraios-lora model through api, including Node.js, Python, http.

astraios-lora huggingface.co Url

https://huggingface.co/bigcode/astraios-lora

bigcode astraios-lora online free

astraios-lora huggingface.co is an online trial and call api platform, which integrates astraios-lora's modeling effects, including api services, and provides a free online trial of astraios-lora, you can try astraios-lora online for free by clicking the link below.

bigcode astraios-lora online free url in huggingface.co:

https://huggingface.co/bigcode/astraios-lora

astraios-lora install

astraios-lora is an open source model from GitHub that offers a free installation service, and any user can find astraios-lora on GitHub to install. At the same time, huggingface.co provides the effect of astraios-lora install, users can directly use astraios-lora installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

astraios-lora install url in huggingface.co:

https://huggingface.co/bigcode/astraios-lora

Url of astraios-lora

astraios-lora huggingface.co Url

Provider of astraios-lora huggingface.co

bigcode
ORGANIZATIONS

Other API from bigcode

huggingface.co

Total runs: 1.1M
Run Growth: 717.0K
Growth Rate: 68.17%
Updated: 2024年3月4日
huggingface.co

Total runs: 35.3K
Run Growth: 26.7K
Growth Rate: 78.08%
Updated: 2024年6月11日
huggingface.co

Total runs: 26.6K
Run Growth: -1.9K
Growth Rate: -7.18%
Updated: 2024年6月5日
huggingface.co

Total runs: 18.6K
Run Growth: 1.0K
Growth Rate: 5.67%
Updated: 2024年10月8日
huggingface.co

Total runs: 7.0K
Run Growth: -48.3K
Growth Rate: -719.62%
Updated: 2023年10月12日
huggingface.co

Total runs: 2.1K
Run Growth: -472
Growth Rate: -23.95%
Updated: 2023年5月10日
huggingface.co

Total runs: 1.6K
Run Growth: -1.9K
Growth Rate: -128.21%
Updated: 2023年5月11日
huggingface.co

Total runs: 376
Run Growth: -287
Growth Rate: -101.06%
Updated: 2023年7月24日
huggingface.co

Total runs: 286
Run Growth: 93
Growth Rate: 32.52%
Updated: 2023年8月17日
huggingface.co

Total runs: 175
Run Growth: -16
Growth Rate: -9.04%
Updated: 2023年8月21日
huggingface.co

Total runs: 159
Run Growth: -16
Growth Rate: -10.06%
Updated: 2023年8月17日
huggingface.co

Total runs: 25
Run Growth: -145
Growth Rate: -580.00%
Updated: 2023年8月13日
huggingface.co

Total runs: 19
Run Growth: 15
Growth Rate: 78.95%
Updated: 2024年1月1日
huggingface.co

Total runs: 18
Run Growth: 1
Growth Rate: 5.56%
Updated: 2023年8月5日
huggingface.co

Total runs: 15
Run Growth: 0
Growth Rate: 0.00%
Updated: 2023年8月5日
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: 2024年2月28日
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: 2025年1月14日