bigcode / astraios-fft

huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: January 01 2024

Introduction of astraios-fft

Model Details of astraios-fft

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models

Astraios

Table of Contents

  1. Model Summary
  2. Use
  3. Training
  4. Citation

Model Summary

Astraios-FFT is an instruction tuned model with 15.5B parameters created by finetuning StarCoderBase on CommitPackFT & OASST as described in the Astraios paper.

  • Repository: bigcode-project/astraios
  • Paper: Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
  • Languages: 80+ Programming languages
  • ✨Astraios:
    Data CommitPackFT+OASST Filtered version of CommitPack and OASST for high-quality commit messages that resemble instructions
    Model Astraios-1B Collection of StarCoderBase-1B models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-3B Collection of StarCoderBase-3B (3B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-7B Collection of StarCoderBase-7B (7B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Astraios-16B Collection of StarCoderBase-16B (16B parameters) models instruction tuned on CommitPackFT + OASST with different tuning methods
    Evaluation BigCloneBench Dataset for clone detection; We use 2,000 samples for evaluation
    Devign Dataset for defect detection; We use 2,000 samples for evaluation
    HumanEvalPack Extension of OpenAI's HumanEval to cover 3 scenarios across 6 languages
    ReCode Dataset for the robustness of code generation, covering 4 variants
    Asleep At The Keyboard Datasets for security of code generation; We use DoW for evaluation

Use

Intended use

The model follows instructions provided in the input. You should always preface your input with "Question: " and finish it with "Answer:", for example: "Question: Please write a function in Python that performs bubble sort.

Answer:"

Feel free to share your generations in the Community tab!

Generation
# pip install -q transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "bigcode/astraios-fft"
model = AutoModelForCausalLM.from_pretrained(checkpoint)
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

inputs = tokenizer.encode("Question: Please write a function in Python that performs bubble sort.

Answer:", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

Training

Model
  • Architecture: GPT-2 model with multi-query attention and Fill-in-the-Middle objective
  • Steps: 250k pretraining & 200 instruction tuning
  • Precision: fp32
Hardware
  • Pretraining:
    • GPUs: 512 Tesla A100
    • Training time: 24 days
  • Instruction tuning:
    • GPUs: 8 Tesla A100
Software

Citation


Runs of bigcode astraios-fft on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About astraios-fft huggingface.co Model

astraios-fft huggingface.co

astraios-fft huggingface.co is an AI model on huggingface.co that provides astraios-fft's model effect (), which can be used instantly with this bigcode astraios-fft model. huggingface.co supports a free trial of the astraios-fft model, and also provides paid use of the astraios-fft. Support call astraios-fft model through api, including Node.js, Python, http.

astraios-fft huggingface.co Url

https://huggingface.co/bigcode/astraios-fft

bigcode astraios-fft online free

astraios-fft huggingface.co is an online trial and call api platform, which integrates astraios-fft's modeling effects, including api services, and provides a free online trial of astraios-fft, you can try astraios-fft online for free by clicking the link below.

bigcode astraios-fft online free url in huggingface.co:

https://huggingface.co/bigcode/astraios-fft

astraios-fft install

astraios-fft is an open source model from GitHub that offers a free installation service, and any user can find astraios-fft on GitHub to install. At the same time, huggingface.co provides the effect of astraios-fft install, users can directly use astraios-fft installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

astraios-fft install url in huggingface.co:

https://huggingface.co/bigcode/astraios-fft

Url of astraios-fft

astraios-fft huggingface.co Url

Provider of astraios-fft huggingface.co

bigcode
ORGANIZATIONS

Other API from bigcode

huggingface.co

Total runs: 1.1M
Run Growth: 717.0K
Growth Rate: 68.17%
Updated: March 04 2024
huggingface.co

Total runs: 35.3K
Run Growth: 26.7K
Growth Rate: 78.08%
Updated: June 11 2024
huggingface.co

Total runs: 18.6K
Run Growth: 1.0K
Growth Rate: 5.67%
Updated: October 08 2024
huggingface.co

Total runs: 7.0K
Run Growth: -48.3K
Growth Rate: -719.62%
Updated: October 12 2023
huggingface.co

Total runs: 2.1K
Run Growth: -472
Growth Rate: -23.95%
Updated: May 10 2023
huggingface.co

Total runs: 1.6K
Run Growth: -1.9K
Growth Rate: -128.21%
Updated: May 11 2023
huggingface.co

Total runs: 376
Run Growth: -287
Growth Rate: -101.06%
Updated: July 24 2023
huggingface.co

Total runs: 286
Run Growth: 93
Growth Rate: 32.52%
Updated: August 17 2023
huggingface.co

Total runs: 159
Run Growth: -16
Growth Rate: -10.06%
Updated: August 17 2023
huggingface.co

Total runs: 25
Run Growth: -145
Growth Rate: -580.00%
Updated: August 13 2023
huggingface.co

Total runs: 19
Run Growth: 15
Growth Rate: 78.95%
Updated: January 01 2024
huggingface.co

Total runs: 18
Run Growth: 1
Growth Rate: 5.56%
Updated: August 05 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: February 28 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: January 14 2025