ai-forever / ruElectra-small

huggingface.co
Total runs: 287
24-hour runs: 142
7-day runs: 159
30-day runs: -201
Model's Last Updated: November 03 2023

Introduction of ruElectra-small

Model Details of ruElectra-small

RU-ELECTRA small model for Embeddings in Russian language.

The model architecture design, pretraining, and evaluation are documented in our preprint: A Family of Pretrained Transformer Language Models for Russian .

For better quality, use mean token embeddings.

Usage (HuggingFace Models Repository)

You can use the model directly from the model repository to compute sentence embeddings:

from transformers import AutoTokenizer, AutoModel
import torch
#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    sum_embeddings = torch.sum(token_embeddings * input_mask_expanded, 1)
    sum_mask = torch.clamp(input_mask_expanded.sum(1), min=1e-10)
    return sum_embeddings / sum_mask
#Sentences we want sentence embeddings for
sentences = ['Привет! Как твои дела?',
             'А правда, что 42 твое любимое число?']
#Load AutoModel from huggingface model repository
tokenizer = AutoTokenizer.from_pretrained("sberbank-ai/ruElectra-small")
model = AutoModel.from_pretrained("sberbank-ai/ruElectra-small")
#Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=24, return_tensors='pt')
#Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)
#Perform pooling. In this case, mean pooling
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

Authors

@misc{zmitrovich2023family,
      title={A Family of Pretrained Transformer Language Models for Russian}, 
      author={Dmitry Zmitrovich and Alexander Abramov and Andrey Kalmykov and Maria Tikhonova and Ekaterina Taktasheva and Danil Astafurov and Mark Baushenko and Artem Snegirev and Tatiana Shavrina and Sergey Markov and Vladislav Mikhailov and Alena Fenogenova},
      year={2023},
      eprint={2309.10931},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Runs of ai-forever ruElectra-small on huggingface.co

287
Total runs
142
24-hour runs
144
3-day runs
159
7-day runs
-201
30-day runs

More Information About ruElectra-small huggingface.co Model

More ruElectra-small license Visit here:

https://choosealicense.com/licenses/mit

ruElectra-small huggingface.co

ruElectra-small huggingface.co is an AI model on huggingface.co that provides ruElectra-small's model effect (), which can be used instantly with this ai-forever ruElectra-small model. huggingface.co supports a free trial of the ruElectra-small model, and also provides paid use of the ruElectra-small. Support call ruElectra-small model through api, including Node.js, Python, http.

ruElectra-small huggingface.co Url

https://huggingface.co/ai-forever/ruElectra-small

ai-forever ruElectra-small online free

ruElectra-small huggingface.co is an online trial and call api platform, which integrates ruElectra-small's modeling effects, including api services, and provides a free online trial of ruElectra-small, you can try ruElectra-small online for free by clicking the link below.

ai-forever ruElectra-small online free url in huggingface.co:

https://huggingface.co/ai-forever/ruElectra-small

ruElectra-small install

ruElectra-small is an open source model from GitHub that offers a free installation service, and any user can find ruElectra-small on GitHub to install. At the same time, huggingface.co provides the effect of ruElectra-small install, users can directly use ruElectra-small installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

ruElectra-small install url in huggingface.co:

https://huggingface.co/ai-forever/ruElectra-small

Url of ruElectra-small

ruElectra-small huggingface.co Url

Provider of ruElectra-small huggingface.co

ai-forever
ORGANIZATIONS

Other API from ai-forever

huggingface.co

Total runs: 525.5K
Run Growth: 507.4K
Growth Rate: 96.56%
Updated: November 03 2023
huggingface.co

Total runs: 10.6K
Run Growth: 1.5K
Growth Rate: 13.75%
Updated: December 05 2023
huggingface.co

Total runs: 8.2K
Run Growth: 5.1K
Growth Rate: 59.45%
Updated: December 29 2024
huggingface.co

Total runs: 5.9K
Run Growth: 3.5K
Growth Rate: 60.26%
Updated: December 11 2023
huggingface.co

Total runs: 2.3K
Run Growth: -360
Growth Rate: -15.50%
Updated: December 05 2023
huggingface.co

Total runs: 1.3K
Run Growth: -6.3K
Growth Rate: -493.39%
Updated: December 05 2023
huggingface.co

Total runs: 315
Run Growth: 165
Growth Rate: 52.05%
Updated: January 26 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: June 08 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: December 24 2021
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: September 21 2021