hfl / chinese-macbert-large

huggingface.co
Total runs: 853
24-hour runs: -16
7-day runs: -124
30-day runs: -391
Model's Last Updated: May 19 2021
fill-mask

Introduction of chinese-macbert-large

Model Details of chinese-macbert-large



GitHub

Please use 'Bert' related functions to load this model!

This repository contains the resources in our paper "Revisiting Pre-trained Models for Chinese Natural Language Processing" , which will be published in " Findings of EMNLP ". You can read our camera-ready paper through ACL Anthology or arXiv pre-print .

Revisiting Pre-trained Models for Chinese Natural Language Processing
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, Guoping Hu

You may also interested in,

More resources by HFL: https://github.com/ymcui/HFL-Anthology

Introduction

MacBERT is an improved BERT with novel M LM a s c orrection pre-training task, which mitigates the discrepancy of pre-training and fine-tuning.

Instead of masking with [MASK] token, which never appears in the fine-tuning stage, we propose to use similar words for the masking purpose . A similar word is obtained by using Synonyms toolkit (Wang and Hu, 2017) , which is based on word2vec (Mikolov et al., 2013) similarity calculations. If an N-gram is selected to mask, we will find similar words individually. In rare cases, when there is no similar word, we will degrade to use random word replacement.

Here is an example of our pre-training task.

Example
Original Sentence we use a language model to predict the probability of the next word.
MLM we use a language [M] to [M] ##di ##ct the pro [M] ##bility of the next word .
Whole word masking we use a language [M] to [M] [M] [M] the [M] [M] [M] of the next word .
N-gram masking we use a [M] [M] to [M] [M] [M] the [M] [M] [M] [M] [M] next word .
MLM as correction we use a text system to ca ##lc ##ulate the po ##si ##bility of the next word .

Except for the new pre-training task, we also incorporate the following techniques.

  • Whole Word Masking (WWM)
  • N-gram masking
  • Sentence-Order Prediction (SOP)

Note that our MacBERT can be directly replaced with the original BERT as there is no differences in the main neural architecture.

For more technical details, please check our paper: Revisiting Pre-trained Models for Chinese Natural Language Processing

Citation

If you find our resource or paper is useful, please consider including the following citation in your paper.

@inproceedings{cui-etal-2020-revisiting,
    title = "Revisiting Pre-Trained Models for {C}hinese Natural Language Processing",
    author = "Cui, Yiming  and
      Che, Wanxiang  and
      Liu, Ting  and
      Qin, Bing  and
      Wang, Shijin  and
      Hu, Guoping",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.findings-emnlp.58",
    pages = "657--668",
}

Runs of hfl chinese-macbert-large on huggingface.co

853
Total runs
-16
24-hour runs
-44
3-day runs
-124
7-day runs
-391
30-day runs

More Information About chinese-macbert-large huggingface.co Model

More chinese-macbert-large license Visit here:

https://choosealicense.com/licenses/apache-2.0

chinese-macbert-large huggingface.co

chinese-macbert-large huggingface.co is an AI model on huggingface.co that provides chinese-macbert-large's model effect (), which can be used instantly with this hfl chinese-macbert-large model. huggingface.co supports a free trial of the chinese-macbert-large model, and also provides paid use of the chinese-macbert-large. Support call chinese-macbert-large model through api, including Node.js, Python, http.

chinese-macbert-large huggingface.co Url

https://huggingface.co/hfl/chinese-macbert-large

hfl chinese-macbert-large online free

chinese-macbert-large huggingface.co is an online trial and call api platform, which integrates chinese-macbert-large's modeling effects, including api services, and provides a free online trial of chinese-macbert-large, you can try chinese-macbert-large online for free by clicking the link below.

hfl chinese-macbert-large online free url in huggingface.co:

https://huggingface.co/hfl/chinese-macbert-large

chinese-macbert-large install

chinese-macbert-large is an open source model from GitHub that offers a free installation service, and any user can find chinese-macbert-large on GitHub to install. At the same time, huggingface.co provides the effect of chinese-macbert-large install, users can directly use chinese-macbert-large installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

chinese-macbert-large install url in huggingface.co:

https://huggingface.co/hfl/chinese-macbert-large

Url of chinese-macbert-large

chinese-macbert-large huggingface.co Url

Provider of chinese-macbert-large huggingface.co

hfl
ORGANIZATIONS

Other API from hfl

huggingface.co

Total runs: 49.8K
Run Growth: 14.3K
Growth Rate: 28.92%
Updated: May 19 2021
huggingface.co

Total runs: 4.3K
Run Growth: 222
Growth Rate: 5.23%
Updated: May 19 2021
huggingface.co

Total runs: 1.0K
Run Growth: 125
Growth Rate: 12.11%
Updated: March 05 2024
huggingface.co

Total runs: 363
Run Growth: -403
Growth Rate: -111.02%
Updated: November 15 2022
huggingface.co

Total runs: 256
Run Growth: -133
Growth Rate: -52.36%
Updated: May 19 2021
huggingface.co

Total runs: 151
Run Growth: 32
Growth Rate: 21.19%
Updated: January 24 2022
huggingface.co

Total runs: 134
Run Growth: 80
Growth Rate: 59.70%
Updated: February 24 2022
huggingface.co

Total runs: 79
Run Growth: -67
Growth Rate: -84.81%
Updated: November 15 2022
huggingface.co

Total runs: 53
Run Growth: -42
Growth Rate: -79.25%
Updated: November 17 2022
huggingface.co

Total runs: 32
Run Growth: -306
Growth Rate: -518.64%
Updated: May 19 2021
huggingface.co

Total runs: 24
Run Growth: 7
Growth Rate: 28.00%
Updated: February 21 2022
huggingface.co

Total runs: 22
Run Growth: 12
Growth Rate: 54.55%
Updated: March 09 2023
huggingface.co

Total runs: 20
Run Growth: -10
Growth Rate: -50.00%
Updated: January 24 2022
huggingface.co

Total runs: 17
Run Growth: -13
Growth Rate: -76.47%
Updated: November 15 2022
huggingface.co

Total runs: 13
Run Growth: -39
Growth Rate: -300.00%
Updated: January 24 2022
huggingface.co

Total runs: 8
Run Growth: 2
Growth Rate: 25.00%
Updated: March 09 2023
huggingface.co

Total runs: 7
Run Growth: -10
Growth Rate: -142.86%
Updated: May 19 2021
huggingface.co

Total runs: 7
Run Growth: 1
Growth Rate: 14.29%
Updated: March 09 2023