bigcode / starencoder

huggingface.co
Total runs: 2.1K
24-hour runs: 0
7-day runs: -30
30-day runs: -472
Model's Last Updated: 5월 10 2023

Introduction of starencoder

Model Details of starencoder

StarEnCoder

Table of Contents
  1. Model Summary
  2. Training
  3. Use
  4. Limitations
  5. License
Model Summary

This is an encoder-only model (i.e., bi-directionally self-attentive Transformers) trained on The Stack dataset.

We leveraged the :

  • Masked Language Modelling (MLM) and Next Sentence Prediction (NSP) objectives from BERT .
  • Predicted masked-out tokens from an input sentence and whether a pair of sentences occur as neighbors in a document.
Training

We train for 100,000 steps with a global batch size of 4,096 sequences of a maximum length of 1,024 so that approximately 400B~tokens are observed. This takes roughly two days using 64 NVIDIA A100 GPUs. Details about the model architecture are reported in the table below.

Hyperparameter Value
Hidden size 768
Intermediate size 3072
Max. position embeddings 1024
Num. of attention heads 12
Num. of hidden layers 12
Attention Multi-head
Num. of parameters ≈125M
Use

This model is trained on 86 programming languages from GitHub code including GitHub issues and Git Commits, and can be efficiently fine-tuned for both code- and text-related tasks. We fine-tuned on a token classification task to detect PII and have released StaPII model.

Limitations

There are limitations to consider when using StarEncoder. It is an encoder-only model, which limits its flexibility in certain code generation or completion tasks, and it was trained on data containing PII, which could pose privacy concerns. Performance may vary across the 80+ supported programming languages, particularly for less common ones, and the model might struggle with understanding domains outside programming languages.

License

The model is licensed under the BigCode OpenRAIL-M v1 license agreement. You can find the full agreement here .

Runs of bigcode starencoder on huggingface.co

2.1K
Total runs
0
24-hour runs
195
3-day runs
-30
7-day runs
-472
30-day runs

More Information About starencoder huggingface.co Model

starencoder huggingface.co

starencoder huggingface.co is an AI model on huggingface.co that provides starencoder's model effect (), which can be used instantly with this bigcode starencoder model. huggingface.co supports a free trial of the starencoder model, and also provides paid use of the starencoder. Support call starencoder model through api, including Node.js, Python, http.

starencoder huggingface.co Url

https://huggingface.co/bigcode/starencoder

bigcode starencoder online free

starencoder huggingface.co is an online trial and call api platform, which integrates starencoder's modeling effects, including api services, and provides a free online trial of starencoder, you can try starencoder online for free by clicking the link below.

bigcode starencoder online free url in huggingface.co:

https://huggingface.co/bigcode/starencoder

starencoder install

starencoder is an open source model from GitHub that offers a free installation service, and any user can find starencoder on GitHub to install. At the same time, huggingface.co provides the effect of starencoder install, users can directly use starencoder installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

starencoder install url in huggingface.co:

https://huggingface.co/bigcode/starencoder

Url of starencoder

starencoder huggingface.co Url

Provider of starencoder huggingface.co

bigcode
ORGANIZATIONS

Other API from bigcode

huggingface.co

Total runs: 1.1M
Run Growth: 717.0K
Growth Rate: 68.17%
Updated: 3월 04 2024
huggingface.co

Total runs: 35.3K
Run Growth: 26.7K
Growth Rate: 78.08%
Updated: 6월 11 2024
huggingface.co

Total runs: 18.6K
Run Growth: 1.0K
Growth Rate: 5.67%
Updated: 10월 08 2024
huggingface.co

Total runs: 7.0K
Run Growth: -48.3K
Growth Rate: -719.62%
Updated: 10월 12 2023
huggingface.co

Total runs: 1.6K
Run Growth: -1.9K
Growth Rate: -128.21%
Updated: 5월 11 2023
huggingface.co

Total runs: 376
Run Growth: -287
Growth Rate: -101.06%
Updated: 7월 24 2023
huggingface.co

Total runs: 286
Run Growth: 93
Growth Rate: 32.52%
Updated: 8월 17 2023
huggingface.co

Total runs: 159
Run Growth: -16
Growth Rate: -10.06%
Updated: 8월 17 2023
huggingface.co

Total runs: 18
Run Growth: 1
Growth Rate: 5.56%
Updated: 8월 05 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: 2월 28 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: 1월 14 2025