bigcode / astraios-1b-adapterh

huggingface.co
Total runs: 10
24-hour runs: 0
7-day runs: 8
30-day runs: 6
Model's Last Updated: Januar 01 2024

Introduction of astraios-1b-adapterh

Model Details of astraios-1b-adapterh

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models

Astraios

Table of Contents

  1. Model Summary
  2. Use
  3. Training
  4. Citation

Model Summary

Astraios-1B-AdapterH is an instruction tuned model with 15.5B parameters created by finetuning StarCoderBase on CommitPackFT & OASST as described in the Astraios paper.

  • Repository: bigcode-project/astraios
  • Paper: Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
  • Languages: 80+ Programming languages
  • ✨Astraios:
    Data CommitPackFT+OASST Filtered version of CommitPack and OASST for high-quality commit messages that resemble instructions
    Model Astraios-1B Collection of StarCoderBase-1B models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-3B Collection of StarCoderBase-3B (3B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-7B Collection of StarCoderBase-7B (7B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-16B Collection of StarCoderBase-16B (16B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Evaluation BigCloneBench Dataset for clone detection; We use 2,000 samples for evaluation
    Devign Dataset for defect detection; We use 2,000 samples for evaluation
    HumanEvalPack Extension of OpenAI's HumanEval to cover 3 scenarios across 6 languages
    ReCode Dataset for the robustness of code generation, covering 4 variants
    Asleep At The Keyboard Datasets for security of code generation; We use DoW for evaluation

Use

Intended use

The model follows instructions provided in the input. You should always preface your input with "Question: " and finish it with "Answer:", for example: "Question: Please write a function in Python that performs bubble sort.

Answer:"

Feel free to share your generations in the Community tab!

Generation
# pip install -q transformers
# pip install -e git+https://github.com/bigcode-project/astraios#subdirectory=peft
from peft import PeftModel 
from transformers import AutoModelForCausalLM, AutoTokenizer

peft_checkpoint = "bigcode/astraios-1b-adapterh"
checkpoint = "bigcode/starcoderbase-1b"
model = AutoModelForCausalLM.from_pretrained(checkpoint)
model = PeftModel.from_pretrained(model, peft_checkpoint)
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

inputs = tokenizer.encode("Question: Please write a function in Python that performs bubble sort.

Answer:", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

Training

Model
  • Architecture: GPT-2 model with multi-query attention and Fill-in-the-Middle objective
  • Steps: 250k pretraining & 200 instruction tuning
  • Precision: fp32
Hardware
  • Pretraining:
    • GPUs: 512 Tesla A100
    • Training time: 24 days
  • Instruction tuning:
    • GPUs: 8 Tesla A100
Software

Citation


Runs of bigcode astraios-1b-adapterh on huggingface.co

10
Total runs
0
24-hour runs
2
3-day runs
8
7-day runs
6
30-day runs

More Information About astraios-1b-adapterh huggingface.co Model

More astraios-1b-adapterh license Visit here:

https://choosealicense.com/licenses/bigcode-openrail-m

astraios-1b-adapterh huggingface.co

astraios-1b-adapterh huggingface.co is an AI model on huggingface.co that provides astraios-1b-adapterh's model effect (), which can be used instantly with this bigcode astraios-1b-adapterh model. huggingface.co supports a free trial of the astraios-1b-adapterh model, and also provides paid use of the astraios-1b-adapterh. Support call astraios-1b-adapterh model through api, including Node.js, Python, http.

astraios-1b-adapterh huggingface.co Url

https://huggingface.co/bigcode/astraios-1b-adapterh

bigcode astraios-1b-adapterh online free

astraios-1b-adapterh huggingface.co is an online trial and call api platform, which integrates astraios-1b-adapterh's modeling effects, including api services, and provides a free online trial of astraios-1b-adapterh, you can try astraios-1b-adapterh online for free by clicking the link below.

bigcode astraios-1b-adapterh online free url in huggingface.co:

https://huggingface.co/bigcode/astraios-1b-adapterh

astraios-1b-adapterh install

astraios-1b-adapterh is an open source model from GitHub that offers a free installation service, and any user can find astraios-1b-adapterh on GitHub to install. At the same time, huggingface.co provides the effect of astraios-1b-adapterh install, users can directly use astraios-1b-adapterh installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

astraios-1b-adapterh install url in huggingface.co:

https://huggingface.co/bigcode/astraios-1b-adapterh

Url of astraios-1b-adapterh

astraios-1b-adapterh huggingface.co Url

Provider of astraios-1b-adapterh huggingface.co

bigcode
ORGANIZATIONS

Other API from bigcode

huggingface.co

Total runs: 1.1M
Run Growth: 717.0K
Growth Rate: 68.17%
Updated: März 04 2024
huggingface.co

Total runs: 35.3K
Run Growth: 26.7K
Growth Rate: 78.08%
Updated: Juni 11 2024
huggingface.co

Total runs: 18.6K
Run Growth: 1.0K
Growth Rate: 5.67%
Updated: Oktober 08 2024
huggingface.co

Total runs: 7.0K
Run Growth: -48.3K
Growth Rate: -719.62%
Updated: Oktober 12 2023
huggingface.co

Total runs: 2.1K
Run Growth: -472
Growth Rate: -23.95%
Updated: Mai 10 2023
huggingface.co

Total runs: 1.6K
Run Growth: -1.9K
Growth Rate: -128.21%
Updated: Mai 11 2023
huggingface.co

Total runs: 376
Run Growth: -287
Growth Rate: -101.06%
Updated: Juli 24 2023
huggingface.co

Total runs: 286
Run Growth: 93
Growth Rate: 32.52%
Updated: August 17 2023
huggingface.co

Total runs: 159
Run Growth: -16
Growth Rate: -10.06%
Updated: August 17 2023
huggingface.co

Total runs: 25
Run Growth: -145
Growth Rate: -580.00%
Updated: August 13 2023
huggingface.co

Total runs: 19
Run Growth: 15
Growth Rate: 78.95%
Updated: Januar 01 2024
huggingface.co

Total runs: 18
Run Growth: 1
Growth Rate: 5.56%
Updated: August 05 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Februar 28 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Januar 14 2025