bigcode / astraios-ia3

huggingface.co
Total runs: 19
24-hour runs: -1
7-day runs: 4
30-day runs: 15
Model's Last Updated: Enero 01 2024

Introduction of astraios-ia3

Model Details of astraios-ia3

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models

Astraios

Table of Contents

  1. Model Summary
  2. Use
  3. Training
  4. Citation

Model Summary

Astraios-IA3 is an instruction tuned model with 15.5B parameters created by finetuning StarCoderBase on CommitPackFT & OASST as described in the Astraios paper.

  • Repository: bigcode-project/astraios
  • Paper: Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
  • Languages: 80+ Programming languages
  • ✨Astraios:
    Data CommitPackFT+OASST Filtered version of CommitPack and OASST for high-quality commit messages that resemble instructions
    Model Astraios-1B Collection of StarCoderBase-1B models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-3B Collection of StarCoderBase-3B (3B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-7B Collection of StarCoderBase-7B (7B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-16B Collection of StarCoderBase-16B (16B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Evaluation BigCloneBench Dataset for clone detection; We use 2,000 samples for evaluation
    Devign Dataset for defect detection; We use 2,000 samples for evaluation
    HumanEvalPack Extension of OpenAI's HumanEval to cover 3 scenarios across 6 languages
    ReCode Dataset for the robustness of code generation, covering 4 variants
    Asleep At The Keyboard Datasets for security of code generation; We use DoW for evaluation

Use

Intended use

The model follows instructions provided in the input. You should always preface your input with "Question: " and finish it with "Answer:", for example: "Question: Please write a function in Python that performs bubble sort.

Answer:"

Feel free to share your generations in the Community tab!

Generation
# pip install -q transformers
# pip install -e git+https://github.com/bigcode-project/astraios#subdirectory=peft
from peft import PeftModel 
from transformers import AutoModelForCausalLM, AutoTokenizer

peft_checkpoint = "bigcode/astraios-ia3"
checkpoint = "bigcode/starcoderbase"
model = AutoModelForCausalLM.from_pretrained(checkpoint)
model = PeftModel.from_pretrained(model, peft_checkpoint)
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

inputs = tokenizer.encode("Question: Please write a function in Python that performs bubble sort.

Answer:", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

Training

Model
  • Architecture: GPT-2 model with multi-query attention and Fill-in-the-Middle objective
  • Steps: 250k pretraining & 200 instruction tuning
  • Precision: fp32
Hardware
  • Pretraining:
    • GPUs: 512 Tesla A100
    • Training time: 24 days
  • Instruction tuning:
    • GPUs: 8 Tesla A100
Software

Citation


Runs of bigcode astraios-ia3 on huggingface.co

19
Total runs
-1
24-hour runs
-2
3-day runs
4
7-day runs
15
30-day runs

More Information About astraios-ia3 huggingface.co Model

astraios-ia3 huggingface.co

astraios-ia3 huggingface.co is an AI model on huggingface.co that provides astraios-ia3's model effect (), which can be used instantly with this bigcode astraios-ia3 model. huggingface.co supports a free trial of the astraios-ia3 model, and also provides paid use of the astraios-ia3. Support call astraios-ia3 model through api, including Node.js, Python, http.

astraios-ia3 huggingface.co Url

https://huggingface.co/bigcode/astraios-ia3

bigcode astraios-ia3 online free

astraios-ia3 huggingface.co is an online trial and call api platform, which integrates astraios-ia3's modeling effects, including api services, and provides a free online trial of astraios-ia3, you can try astraios-ia3 online for free by clicking the link below.

bigcode astraios-ia3 online free url in huggingface.co:

https://huggingface.co/bigcode/astraios-ia3

astraios-ia3 install

astraios-ia3 is an open source model from GitHub that offers a free installation service, and any user can find astraios-ia3 on GitHub to install. At the same time, huggingface.co provides the effect of astraios-ia3 install, users can directly use astraios-ia3 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

astraios-ia3 install url in huggingface.co:

https://huggingface.co/bigcode/astraios-ia3

Url of astraios-ia3

astraios-ia3 huggingface.co Url

Provider of astraios-ia3 huggingface.co

bigcode
ORGANIZATIONS

Other API from bigcode

huggingface.co

Total runs: 1.1M
Run Growth: 717.0K
Growth Rate: 68.17%
Updated: Marzo 04 2024
huggingface.co

Total runs: 35.3K
Run Growth: 26.7K
Growth Rate: 78.08%
Updated: Junio 11 2024
huggingface.co

Total runs: 18.6K
Run Growth: 1.0K
Growth Rate: 5.67%
Updated: Octubre 08 2024
huggingface.co

Total runs: 7.0K
Run Growth: -48.3K
Growth Rate: -719.62%
Updated: Octubre 12 2023
huggingface.co

Total runs: 2.1K
Run Growth: -472
Growth Rate: -23.95%
Updated: Mayo 10 2023
huggingface.co

Total runs: 1.6K
Run Growth: -1.9K
Growth Rate: -128.21%
Updated: Mayo 11 2023
huggingface.co

Total runs: 376
Run Growth: -287
Growth Rate: -101.06%
Updated: Julio 24 2023
huggingface.co

Total runs: 286
Run Growth: 93
Growth Rate: 32.52%
Updated: Agosto 17 2023
huggingface.co

Total runs: 159
Run Growth: -16
Growth Rate: -10.06%
Updated: Agosto 17 2023
huggingface.co

Total runs: 25
Run Growth: -145
Growth Rate: -580.00%
Updated: Agosto 13 2023
huggingface.co

Total runs: 18
Run Growth: 1
Growth Rate: 5.56%
Updated: Agosto 05 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Febrero 28 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Enero 14 2025