castorini / afriberta_large

huggingface.co
Total runs: 1.4K
24-hour runs: -50
7-day runs: 213
30-day runs: 215
Model's Last Updated: Janvier 13 2023
fill-mask

Introduction of afriberta_large

Model Details of afriberta_large

afriberta_large

Model description

AfriBERTa large is a pretrained multilingual language model with around 126 million parameters. The model has 10 layers, 6 attention heads, 768 hidden units and 3072 feed forward size. The model was pretrained on 11 African languages namely - Afaan Oromoo (also called Oromo), Amharic, Gahuza (a mixed language containing Kinyarwanda and Kirundi), Hausa, Igbo, Nigerian Pidgin, Somali, Swahili, Tigrinya and Yorùbá. The model has been shown to obtain competitive downstream performances on text classification and Named Entity Recognition on several African languages, including those it was not pretrained on.

Intended uses & limitations
How to use

You can use this model with Transformers for any downstream task. For example, assuming we want to finetune this model on a token classification task, we do the following:

>>> from transformers import AutoTokenizer, AutoModelForTokenClassification
>>> model = AutoModelForTokenClassification.from_pretrained("castorini/afriberta_large")
>>> tokenizer = AutoTokenizer.from_pretrained("castorini/afriberta_large")
# we have to manually set the model max length because it is an imported sentencepiece model, which huggingface does not properly support right now
>>> tokenizer.model_max_length = 512 
Limitations and bias
  • This model is possibly limited by its training dataset which are majorly obtained from news articles from a specific span of time. Thus, it may not generalize well.
  • This model is trained on very little data (less than 1 GB), hence it may not have seen enough data to learn very complex linguistic relations.
Training data

The model was trained on an aggregation of datasets from the BBC news website and Common Crawl.

Training procedure

For information on training procedures, please refer to the AfriBERTa paper or repository

BibTeX entry and citation info
@inproceedings{ogueji-etal-2021-small,
    title = "Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages",
    author = "Ogueji, Kelechi  and
      Zhu, Yuxin  and
      Lin, Jimmy",
    booktitle = "Proceedings of the 1st Workshop on Multilingual Representation Learning",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.mrl-1.11",
    pages = "116--126",
}

Runs of castorini afriberta_large on huggingface.co

1.4K
Total runs
-50
24-hour runs
-42
3-day runs
213
7-day runs
215
30-day runs

More Information About afriberta_large huggingface.co Model

More afriberta_large license Visit here:

https://choosealicense.com/licenses/mit

afriberta_large huggingface.co

afriberta_large huggingface.co is an AI model on huggingface.co that provides afriberta_large's model effect (), which can be used instantly with this castorini afriberta_large model. huggingface.co supports a free trial of the afriberta_large model, and also provides paid use of the afriberta_large. Support call afriberta_large model through api, including Node.js, Python, http.

afriberta_large huggingface.co Url

https://huggingface.co/castorini/afriberta_large

castorini afriberta_large online free

afriberta_large huggingface.co is an online trial and call api platform, which integrates afriberta_large's modeling effects, including api services, and provides a free online trial of afriberta_large, you can try afriberta_large online for free by clicking the link below.

castorini afriberta_large online free url in huggingface.co:

https://huggingface.co/castorini/afriberta_large

afriberta_large install

afriberta_large is an open source model from GitHub that offers a free installation service, and any user can find afriberta_large on GitHub to install. At the same time, huggingface.co provides the effect of afriberta_large install, users can directly use afriberta_large installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

afriberta_large install url in huggingface.co:

https://huggingface.co/castorini/afriberta_large

Url of afriberta_large

afriberta_large huggingface.co Url

Provider of afriberta_large huggingface.co

castorini
ORGANIZATIONS

Other API from castorini