RU
ssian
D
ecoder
O
n
L
anguage
P
icture
H
yper-tasking (
RUDOLPH
)
350M
is a fast and light text-image-text transformer designed for a quick and easy fine-tuning for a range of tasks: from generating images by text description and image classification to visual question answering and more. This model demonstrates the power of Hyper-tasking Transformers.
Hyper-tasking model is a generalized multi-tasking model, i.e., the model that can solve almost all tasks within supported modalities, mandatory including mutual pairwise translations between modalities (two modalities in case of RUDOLPH: images and Russian texts).
Tasks:
text2image generation, self reranking, text ranking, image ranking, image2text generation, zero-shot image classification, text2text generation, and so on
Language:
Russian
Type:
decoder
Num Parameters:
350M
Training Data Volume:
141 million text-image pairs, 7.6 million text paragraphs
Details of architecture
The maximum sequence length that this model may be used with depends on the modality and stands for 64 - 256 - 64 for the left text tokens, image tokens, and right text tokens, respectively.
RUDOLPH 350M is a Transformer-based decoder model with the following parameters:
num_layers (24) — Number of hidden layers in the Transformer decoder.
hidden_size (1024) — Dimensionality of the hidden layers.
num_attention_heads (16) — Number of attention heads for each attention layer.
Sparse Attention Masks
The primary proposed method is to modify the sparse transformer's attention mask to better control modalities. It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with both image and left text condition.
More Information About RUDOLPH-350M huggingface.co Model
RUDOLPH-350M huggingface.co
RUDOLPH-350M huggingface.co is an AI model on huggingface.co that provides RUDOLPH-350M's model effect (), which can be used instantly with this ai-forever RUDOLPH-350M model. huggingface.co supports a free trial of the RUDOLPH-350M model, and also provides paid use of the RUDOLPH-350M. Support call RUDOLPH-350M model through api, including Node.js, Python, http.
RUDOLPH-350M huggingface.co is an online trial and call api platform, which integrates RUDOLPH-350M's modeling effects, including api services, and provides a free online trial of RUDOLPH-350M, you can try RUDOLPH-350M online for free by clicking the link below.
ai-forever RUDOLPH-350M online free url in huggingface.co:
RUDOLPH-350M is an open source model from GitHub that offers a free installation service, and any user can find RUDOLPH-350M on GitHub to install. At the same time, huggingface.co provides the effect of RUDOLPH-350M install, users can directly use RUDOLPH-350M installed effect in huggingface.co for debugging and trial. It also supports api for free installation.