ai4bharat / MultiIndicSentenceSummarizationSS

huggingface.co
Total runs: 15
24-hour runs: 2
7-day runs: 0
30-day runs: 4
Model's Last Updated: April 30 2022
text2text-generation

Introduction of MultiIndicSentenceSummarizationSS

Model Details of MultiIndicSentenceSummarizationSS

MultiIndicSentenceSummarizationSS

This repository contains the IndicBARTSS checkpoint finetuned on the 11 languages of IndicSentenceSummarization dataset. For finetuning details, see the paper .

  • Supported languages: Assamese, Bengali, Gujarati, Hindi, Marathi, Odiya, Punjabi, Kannada, Malayalam, Tamil, and Telugu. Not all of these languages are supported by mBART50 and mT5.
  • The model is much smaller than the mBART and mT5(-base) models, so less computationally expensive for decoding.
  • Trained on large Indic language corpora (5.53 million sentences).
  • Unlike MultiIndicSentenceSummarization each language is written in its own script, so you do not need to perform any script mapping to/from Devanagari.
Using this model in transformers
from transformers import MBartForConditionalGeneration, AutoModelForSeq2SeqLM
from transformers import AlbertTokenizer, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("ai4bharat/MultiIndicSentenceSummarizationSS", do_lower_case=False, use_fast=False, keep_accents=True)
# Or use tokenizer = AlbertTokenizer.from_pretrained("ai4bharat/MultiIndicSentenceSummarizationSS", do_lower_case=False, use_fast=False, keep_accents=True)
model = AutoModelForSeq2SeqLM.from_pretrained("ai4bharat/MultiIndicSentenceSummarizationSS")
# Or use model = MBartForConditionalGeneration.from_pretrained("ai4bharat/MultiIndicSentenceSummarizationSS")

# Some initial mapping
bos_id = tokenizer._convert_token_to_id_with_added_voc("<s>")
eos_id = tokenizer._convert_token_to_id_with_added_voc("</s>")
pad_id = tokenizer._convert_token_to_id_with_added_voc("<pad>")

# To get lang_id use any of ['<2as>', '<2bn>', '<2en>', '<2gu>', '<2hi>', '<2kn>', '<2ml>', '<2mr>', '<2or>', '<2pa>', '<2ta>', '<2te>']
# First tokenize the input. The format below is how IndicBART was trained so the input should be "Sentence </s> <2xx>" where xx is the language code. Similarly, the output should be "<2yy> Sentence </s>".
inp = tokenizer("जम्मू एवं कश्मीर के अनंतनाग जिले में शनिवार को सुरक्षाबलों के साथ मुठभेड़ में दो आतंकवादियों को मार गिराया गया। </s> <2hi>", add_special_tokens=False, return_tensors="pt", padding=True).input_ids 

# For generation. Pardon the messiness. Note the decoder_start_token_id.

model_output=model.generate(inp, use_cache=True,no_repeat_ngram_size=3, num_beams=5, length_penalty=0.8, max_length=20, min_length=1, early_stopping=True, pad_token_id=pad_id, bos_token_id=bos_id, eos_token_id=eos_id, decoder_start_token_id=tokenizer._convert_token_to_id_with_added_voc("<2hi>"))

# Decode to get output strings
decoded_output=tokenizer.decode(model_output[0], skip_special_tokens=True, clean_up_tokenization_spaces=False)
print(decoded_output) # अनंतनाग में सुरक्षाबलों के साथ मुठभेड़ में दो आतंकवादी ढेर
Benchmarks

Scores on the IndicSentenceSummarization test sets are as follows:

Language Rouge-1 / Rouge-2 / Rouge-L
as 63.56 / 49.90 / 62.57
bn 52.52 / 36.15 / 50.60
gu 47.69 / 29.77 / 45.61
hi 50.43 / 28.13 / 45.15
kn 77.06 / 69.36 / 76.33
ml 65.00 / 51.99 / 63.76
mr 47.05 / 25.97 / 45.52
or 50.96 / 30.32 / 49.23
pa 54.95 / 36.26 / 51.26
ta 58.52 / 38.36 / 56.49
te 53.75 / 35.17 / 52.66
Citation

If you use this model, please cite the following paper:

@inproceedings{Kumar2022IndicNLGSM,
  title={IndicNLG Suite: Multilingual Datasets for Diverse NLG Tasks in Indic Languages},
  author={Aman Kumar and Himani Shrotriya and Prachi Sahu and Raj Dabre and Ratish Puduppully and Anoop Kunchukuttan and Amogh Mishra and Mitesh M. Khapra and Pratyush Kumar},
  year={2022},
  url = "https://arxiv.org/abs/2203.05437"
  }

Runs of ai4bharat MultiIndicSentenceSummarizationSS on huggingface.co

15
Total runs
2
24-hour runs
2
3-day runs
0
7-day runs
4
30-day runs

More Information About MultiIndicSentenceSummarizationSS huggingface.co Model

More MultiIndicSentenceSummarizationSS license Visit here:

https://choosealicense.com/licenses/mit

MultiIndicSentenceSummarizationSS huggingface.co

MultiIndicSentenceSummarizationSS huggingface.co is an AI model on huggingface.co that provides MultiIndicSentenceSummarizationSS's model effect (), which can be used instantly with this ai4bharat MultiIndicSentenceSummarizationSS model. huggingface.co supports a free trial of the MultiIndicSentenceSummarizationSS model, and also provides paid use of the MultiIndicSentenceSummarizationSS. Support call MultiIndicSentenceSummarizationSS model through api, including Node.js, Python, http.

MultiIndicSentenceSummarizationSS huggingface.co Url

https://huggingface.co/ai4bharat/MultiIndicSentenceSummarizationSS

ai4bharat MultiIndicSentenceSummarizationSS online free

MultiIndicSentenceSummarizationSS huggingface.co is an online trial and call api platform, which integrates MultiIndicSentenceSummarizationSS's modeling effects, including api services, and provides a free online trial of MultiIndicSentenceSummarizationSS, you can try MultiIndicSentenceSummarizationSS online for free by clicking the link below.

ai4bharat MultiIndicSentenceSummarizationSS online free url in huggingface.co:

https://huggingface.co/ai4bharat/MultiIndicSentenceSummarizationSS

MultiIndicSentenceSummarizationSS install

MultiIndicSentenceSummarizationSS is an open source model from GitHub that offers a free installation service, and any user can find MultiIndicSentenceSummarizationSS on GitHub to install. At the same time, huggingface.co provides the effect of MultiIndicSentenceSummarizationSS install, users can directly use MultiIndicSentenceSummarizationSS installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

MultiIndicSentenceSummarizationSS install url in huggingface.co:

https://huggingface.co/ai4bharat/MultiIndicSentenceSummarizationSS

Url of MultiIndicSentenceSummarizationSS

MultiIndicSentenceSummarizationSS huggingface.co Url

Provider of MultiIndicSentenceSummarizationSS huggingface.co

ai4bharat
ORGANIZATIONS

Other API from ai4bharat

huggingface.co

Total runs: 1.1M
Run Growth: 656.1K
Growth Rate: 59.59%
Updated: August 07 2022
huggingface.co

Total runs: 49.5K
Run Growth: 3.8K
Growth Rate: 7.58%
Updated: December 21 2022
huggingface.co

Total runs: 1.7K
Run Growth: 295
Growth Rate: 16.88%
Updated: August 07 2022
huggingface.co

Total runs: 1.1K
Run Growth: -3.1K
Growth Rate: -278.75%
Updated: March 11 2024
huggingface.co

Total runs: 223
Run Growth: -81
Growth Rate: -23.62%
Updated: October 18 2024
huggingface.co

Total runs: 24
Run Growth: 9
Growth Rate: 42.86%
Updated: October 18 2024
huggingface.co

Total runs: 18
Run Growth: 7
Growth Rate: 30.43%
Updated: October 18 2024
huggingface.co

Total runs: 8
Run Growth: -3
Growth Rate: -11.54%
Updated: October 18 2024
huggingface.co

Total runs: 3
Run Growth: 17
Growth Rate: 56.67%
Updated: October 18 2024
huggingface.co

Total runs: 3
Run Growth: 10
Growth Rate: 40.00%
Updated: October 18 2024