Longformer
is a transformer model for long documents.
longformer-base-4096
is a BERT-like model started from the RoBERTa checkpoint and pretrained for MLM on long documents. It supports sequences of length up to 4,096.
Longformer uses a combination of a sliding window (local) attention and global attention. Global attention is user-configured based on the task to allow the model to learn task-specific representations.
Please refer to the examples in
modeling_longformer.py
and the paper for more details on how to set global attention.
@article{Beltagy2020Longformer,
title={Longformer: The Long-Document Transformer},
author={Iz Beltagy and Matthew E. Peters and Arman Cohan},
journal={arXiv:2004.05150},
year={2020},
}
Longformer
is an open-source project developed by
the Allen Institute for Artificial Intelligence (AI2)
.
AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.
Runs of allenai longformer-base-4096 on huggingface.co
5.6M
Total runs
98.3K
24-hour runs
404.1K
3-day runs
777.4K
7-day runs
2.7M
30-day runs
More Information About longformer-base-4096 huggingface.co Model
longformer-base-4096 huggingface.co is an AI model on huggingface.co that provides longformer-base-4096's model effect (), which can be used instantly with this allenai longformer-base-4096 model. huggingface.co supports a free trial of the longformer-base-4096 model, and also provides paid use of the longformer-base-4096. Support call longformer-base-4096 model through api, including Node.js, Python, http.
longformer-base-4096 huggingface.co is an online trial and call api platform, which integrates longformer-base-4096's modeling effects, including api services, and provides a free online trial of longformer-base-4096, you can try longformer-base-4096 online for free by clicking the link below.
allenai longformer-base-4096 online free url in huggingface.co:
longformer-base-4096 is an open source model from GitHub that offers a free installation service, and any user can find longformer-base-4096 on GitHub to install. At the same time, huggingface.co provides the effect of longformer-base-4096 install, users can directly use longformer-base-4096 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
longformer-base-4096 install url in huggingface.co: