Optimum Habana
is the interface between the Hugging Face Transformers and Diffusers libraries and Habana's Gaudi processor (HPU).
It provides a set of tools enabling easy and fast model loading, training and inference on single- and multi-HPU settings for different downstream tasks.
Learn more about how to take advantage of the power of Habana HPUs to train and deploy Transformers and Diffusers models at
hf.co/hardware/habana
.
DistilBERT Base model HPU configuration
This model only contains the
GaudiConfig
file for running the
distilbert-base-uncased
model on Habana's Gaudi processors (HPU).
This model contains no model weights, only a GaudiConfig.
This enables to specify:
use_fused_adam
: whether to use Habana's custom AdamW implementation
use_fused_clip_norm
: whether to use Habana's fused gradient norm clipping operator
use_torch_autocast
: whether to use Torch Autocast for managing mixed precision
Usage
The model is instantiated the same way as in the Transformers library.
The only difference is that there are a few new training arguments specific to HPUs.
It is strongly recommended to train this model doing bf16 mixed-precision training for optimal performance and accuracy.
Here
is a question-answering example script to fine-tune a model on SQuAD. You can run it with DistilBERT with the following command:
distilbert-base-uncased huggingface.co is an AI model on huggingface.co that provides distilbert-base-uncased's model effect (), which can be used instantly with this Habana distilbert-base-uncased model. huggingface.co supports a free trial of the distilbert-base-uncased model, and also provides paid use of the distilbert-base-uncased. Support call distilbert-base-uncased model through api, including Node.js, Python, http.
distilbert-base-uncased huggingface.co is an online trial and call api platform, which integrates distilbert-base-uncased's modeling effects, including api services, and provides a free online trial of distilbert-base-uncased, you can try distilbert-base-uncased online for free by clicking the link below.
Habana distilbert-base-uncased online free url in huggingface.co:
distilbert-base-uncased is an open source model from GitHub that offers a free installation service, and any user can find distilbert-base-uncased on GitHub to install. At the same time, huggingface.co provides the effect of distilbert-base-uncased install, users can directly use distilbert-base-uncased installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
distilbert-base-uncased install url in huggingface.co: