ai-forever / fbc3_baseline

huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: Septiembre 28 2023

Introduction of fbc3_baseline

Model Details of fbc3_baseline

AI Journey 2023 Baseline solution

This solution is inspired by the methodologies of FROMAGe and Kosmos-1 . It primarily employs these approaches to fine-tune the linear mapping from visual and audio vector spaces into the language model-decoder's vector space. Subsequently, the response is generated exclusively using the intact language model.

As a modality encoder, we utilize ImageBind . This encoder has been trained specifically for understanding images, audio, and text and other data formats in a shared embedding space.

During the training phase, the weights of the encoder and the language model remain frozen. The exceptions to this are the additional embeddings for two tokens marking the beginning and end of the respective modalities in the language model: <SOI> , <EOI> and <SOA> , <EOA> (S, E — Start, End; I,A — Image, Audio).

Training is leveraged using four datasets: VisualDialogues , COCO Captions , Clotho v. 2.1 and Clotho-AQA . The core training objective is next token prediction with CrossEntropy loss. The general architecture is illustrated here:

Baseline model architecture

Training

To reproduce training, please run notebook after installing requirements:

pip install requirements.txt

Runs of ai-forever fbc3_baseline on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About fbc3_baseline huggingface.co Model

fbc3_baseline huggingface.co

fbc3_baseline huggingface.co is an AI model on huggingface.co that provides fbc3_baseline's model effect (), which can be used instantly with this ai-forever fbc3_baseline model. huggingface.co supports a free trial of the fbc3_baseline model, and also provides paid use of the fbc3_baseline. Support call fbc3_baseline model through api, including Node.js, Python, http.

ai-forever fbc3_baseline online free

fbc3_baseline huggingface.co is an online trial and call api platform, which integrates fbc3_baseline's modeling effects, including api services, and provides a free online trial of fbc3_baseline, you can try fbc3_baseline online for free by clicking the link below.

ai-forever fbc3_baseline online free url in huggingface.co:

https://huggingface.co/ai-forever/fbc3_baseline

fbc3_baseline install

fbc3_baseline is an open source model from GitHub that offers a free installation service, and any user can find fbc3_baseline on GitHub to install. At the same time, huggingface.co provides the effect of fbc3_baseline install, users can directly use fbc3_baseline installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

fbc3_baseline install url in huggingface.co:

https://huggingface.co/ai-forever/fbc3_baseline

Url of fbc3_baseline

fbc3_baseline huggingface.co Url

Provider of fbc3_baseline huggingface.co

ai-forever
ORGANIZATIONS

Other API from ai-forever

huggingface.co

Total runs: 525.5K
Run Growth: 507.4K
Growth Rate: 96.56%
Updated: Noviembre 03 2023
huggingface.co

Total runs: 10.6K
Run Growth: 1.5K
Growth Rate: 13.75%
Updated: Diciembre 05 2023
huggingface.co

Total runs: 8.2K
Run Growth: 5.1K
Growth Rate: 59.45%
Updated: Diciembre 29 2024
huggingface.co

Total runs: 5.9K
Run Growth: 3.5K
Growth Rate: 60.26%
Updated: Diciembre 11 2023
huggingface.co

Total runs: 2.3K
Run Growth: -408
Growth Rate: -17.86%
Updated: Diciembre 05 2023
huggingface.co

Total runs: 1.8K
Run Growth: 5
Growth Rate: 0.28%
Updated: Diciembre 28 2023
huggingface.co

Total runs: 1.3K
Run Growth: -6.3K
Growth Rate: -493.39%
Updated: Diciembre 05 2023
huggingface.co

Total runs: 315
Run Growth: 158
Growth Rate: 50.16%
Updated: Enero 26 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Diciembre 24 2021
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Junio 08 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Septiembre 21 2021