lucataco / llama-3-vision-alpha

Projection module trained to add vision capabilties to Llama 3 using SigLIP

replicate.com
Total runs: 5.5K
24-hour runs: 0
7-day runs: 0
30-day runs: 100
Github
Model's Last Updated: November 05 2024

Introduction of llama-3-vision-alpha

Model Details of llama-3-vision-alpha

Readme

llama3-vision-alpha

projection module trained to add vision capabilties to Llama 3 using SigLIP. built by @yeswondwerr and @qtnx_

examples

Image Examples
What is the title of this book? answer briefly
The title of the book is “The Little Book of Deep Learning”.

Where is the person standing? answer briefly
The person is standing on the balcony.

Describe the image
The image shows a person holding a book with a cityscape visible through the window behind them. The book has a cover with a title that reads “The Little Book of Deep Learning” in bold letters.
What type of food is the girl holding? answer briefly
A hamburger!

What color is the woman’s hair? answer briefly
It’s white!

Describe the image
The image is of a young girl with short, curly hair and a sweet smile, holding a giant hamburger in her hand. She’s sitting at a table with a festive dinner setting, surrounded by candles and a warm glow. Her eyes are shining with excitement and contentment as she takes a big bite of the burger.

acknowledgements

  • Liu et al. : LLaVA
  • Moon et al. : AnyMAL
  • vikhyatk : moondream, test images
                                       .x+=:.                                                             
                                      z`    ^%                                                  .uef^"    
               .u    .                   .   <k                           .u    .             :d88E       
    .u@u     .d88B :@8c       .u       .@8Ned8"      .u          u      .d88B :@8c        .   `888E       
 .zWF8888bx ="8888f8888r   ud8888.   .@^%8888"    ud8888.     us888u.  ="8888f8888r  .udR88N   888E .z8k  
.888  9888    4888>'88"  :888'8888. x88:  `)8b. :888'8888. .@88 "8888"   4888>'88"  <888'888k  888E~?888L 
I888  9888    4888> '    d888 '88%" 8888N=*8888 d888 '88%" 9888  9888    4888> '    9888 'Y"   888E  888E 
I888  9888    4888>      8888.+"     %8"    R88 8888.+"    9888  9888    4888>      9888       888E  888E 
I888  9888   .d888L .+   8888L        @8Wou 9%  8888L      9888  9888   .d888L .+   9888       888E  888E 
`888Nx?888   ^"8888*"    '8888c. .+ .888888P`   '8888c. .+ 9888  9888   ^"8888*"    ?8888u../  888E  888E 
 "88" '888      "Y"       "88888%   `   ^"F      "88888%   "888*""888"     "Y"       "8888P'  m888N= 888> 
       88E                  "YP'                   "YP'     ^Y"   ^Y'                  "P'     `Y"   888  
       98>                                                                                          J88"  
       '8                                                                                           @%    
        `                                                                                         :"      

Pricing of llama-3-vision-alpha replicate.com

Run time and cost

This model costs approximately $0.0031 to run on Replicate, or 322 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker .

This model runs on Nvidia A40 GPU hardware . Predictions typically complete within 6 seconds. The predict time for this model varies significantly based on the inputs.

Runs of lucataco llama-3-vision-alpha on replicate.com

5.5K
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
100
30-day runs

More Information About llama-3-vision-alpha replicate.com Model

llama-3-vision-alpha replicate.com

llama-3-vision-alpha replicate.com is an AI model on replicate.com that provides llama-3-vision-alpha's model effect (Projection module trained to add vision capabilties to Llama 3 using SigLIP), which can be used instantly with this lucataco llama-3-vision-alpha model. replicate.com supports a free trial of the llama-3-vision-alpha model, and also provides paid use of the llama-3-vision-alpha. Support call llama-3-vision-alpha model through api, including Node.js, Python, http.

llama-3-vision-alpha replicate.com Url

https://replicate.com/lucataco/llama-3-vision-alpha

lucataco llama-3-vision-alpha online free

llama-3-vision-alpha replicate.com is an online trial and call api platform, which integrates llama-3-vision-alpha's modeling effects, including api services, and provides a free online trial of llama-3-vision-alpha, you can try llama-3-vision-alpha online for free by clicking the link below.

lucataco llama-3-vision-alpha online free url in replicate.com:

https://replicate.com/lucataco/llama-3-vision-alpha

llama-3-vision-alpha install

llama-3-vision-alpha is an open source model from GitHub that offers a free installation service, and any user can find llama-3-vision-alpha on GitHub to install. At the same time, replicate.com provides the effect of llama-3-vision-alpha install, users can directly use llama-3-vision-alpha installed effect in replicate.com for debugging and trial. It also supports api for free installation.

llama-3-vision-alpha install url in replicate.com:

https://replicate.com/lucataco/llama-3-vision-alpha

llama-3-vision-alpha install url in github:

https://github.com/lucataco/cog-llama-3-vision-alpha

Url of llama-3-vision-alpha

llama-3-vision-alpha replicate.com Url

llama-3-vision-alpha Owner Github

Provider of llama-3-vision-alpha replicate.com

Other API from lucataco

replicate

Remove background from an image

Total runs: 5.3M
Run Growth: 200.0K
Growth Rate: 3.77%
Updated: September 15 2023
replicate

Falcons.ai Fine-Tuned Vision Transformer (ViT) for NSFW Image Classification

Total runs: 4.5M
Run Growth: 0
Growth Rate: 0.00%
Updated: November 21 2023
replicate

Implementation of Realistic Vision v5.1 with VAE

Total runs: 3.7M
Run Growth: 500.0K
Growth Rate: 13.51%
Updated: August 15 2023
replicate

FLUX.1-Dev LoRA Explorer

Total runs: 2.8M
Run Growth: 200.0K
Growth Rate: 7.14%
Updated: October 06 2024
replicate

SDXL ControlNet - Canny

Total runs: 2.1M
Run Growth: 100.0K
Growth Rate: 4.76%
Updated: October 04 2023
replicate

SDXL Inpainting by the HF Diffusers team

Total runs: 1.7M
Run Growth: 200.0K
Growth Rate: 11.76%
Updated: March 06 2024
replicate

Turn any image into a video

Total runs: 1.3M
Run Growth: 0
Growth Rate: 0.00%
Updated: September 03 2023
replicate

Segmind Stable Diffusion Model (SSD-1B) is a distilled 50% smaller version of SDXL, offering a 60% speedup while maintaining high-quality text-to-image generation capabilities

Total runs: 992.2K
Run Growth: 900
Growth Rate: 0.09%
Updated: November 09 2023
replicate

Hyper FLUX 8-step by ByteDance

Total runs: 926.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: August 28 2024
replicate

CLIP Interrogator for SDXL optimizes text prompts to match a given image

Total runs: 845.8K
Run Growth: 200
Growth Rate: 0.02%
Updated: May 17 2024
replicate

A multimodal LLM-based AI assistant, which is trained with alignment techniques. Qwen-VL-Chat supports more flexible interaction, such as multi-round question answering, and creative capabilities.

Total runs: 796.6K
Run Growth: 1.5K
Growth Rate: 0.19%
Updated: October 15 2023
replicate

FLUX.1-Dev Multi LoRA Explorer

Total runs: 769.3K
Run Growth: 126.2K
Growth Rate: 16.40%
Updated: October 06 2024
replicate

Robust face restoration algorithm for old photos/AI-generated faces

Total runs: 768.8K
Run Growth: 245.1K
Growth Rate: 31.88%
Updated: September 06 2023
replicate

FLUX.1-Schnell LoRA Explorer

Total runs: 654.5K
Run Growth: 85.1K
Growth Rate: 13.00%
Updated: September 07 2024
replicate

Coqui XTTS-v2: Multilingual Text To Speech Voice Cloning

Total runs: 644.5K
Run Growth: 64.8K
Growth Rate: 10.05%
Updated: November 28 2023
replicate

SDXL v1.0 - A text-to-image generative AI model that creates beautiful images

Total runs: 477.4K
Run Growth: 300
Growth Rate: 0.06%
Updated: November 02 2023
replicate

😊 Hotshot-XL is an AI text-to-GIF model trained to work alongside Stable Diffusion XL

Total runs: 464.2K
Run Growth: 60.6K
Growth Rate: 13.05%
Updated: October 23 2023
replicate

snowflake-arctic-embed is a suite of text embedding models that focuses on creating high-quality retrieval models optimized for performance

Total runs: 397.0K
Run Growth: 100
Growth Rate: 0.03%
Updated: April 20 2024
replicate

Latent Consistency Model (LCM): SDXL, distills the original model into a version that requires fewer steps (4 to 8 instead of the original 25 to 50)

Total runs: 394.2K
Run Growth: 0
Growth Rate: 0.00%
Updated: November 13 2023
replicate

Monster Labs QrCode ControlNet on top of SD Realistic Vision v5.1

Total runs: 387.3K
Run Growth: 2.3K
Growth Rate: 0.59%
Updated: September 24 2023
replicate

RealvisXL-v2.0 with LCM LoRA - requires fewer steps (4 to 8 instead of the original 40 to 50)

Total runs: 290.9K
Run Growth: 300
Growth Rate: 0.10%
Updated: November 16 2023
replicate

Implementation of SDXL RealVisXL_V2.0

Total runs: 283.6K
Run Growth: 2.3K
Growth Rate: 0.81%
Updated: November 09 2023
replicate

Animate Your Personalized Text-to-Image Diffusion Models

Total runs: 281.9K
Run Growth: 4.2K
Growth Rate: 1.49%
Updated: September 25 2023
replicate

moondream2 is a small vision language model designed to run efficiently on edge devices

Total runs: 263.8K
Run Growth: 31.5K
Growth Rate: 11.94%
Updated: July 29 2024
replicate

Practical face restoration algorithm for *old photos* or *AI-generated faces* (for larger images)

Total runs: 234.2K
Run Growth: 12.6K
Growth Rate: 5.38%
Updated: August 03 2023
replicate

DreamShaper is a general purpose SD model that aims at doing everything well, photos, art, anime, manga. It's designed to match Midjourney and DALL-E.

Total runs: 194.5K
Run Growth: 7.0K
Growth Rate: 3.60%
Updated: December 20 2023
replicate

Real-ESRGAN Video Upscaler

Total runs: 137.7K
Run Growth: 26.3K
Growth Rate: 19.10%
Updated: November 25 2023
replicate

A unique fusion that showcases exceptional prompt adherence and semantic understanding, it seems to be a step above base SDXL and a step closer to DALLE-3 in terms of prompt comprehension

Total runs: 124.3K
Run Growth: 900
Growth Rate: 0.72%
Updated: December 27 2023
replicate

CLIP Interrogator (for faster inference)

Total runs: 122.0K
Run Growth: 500
Growth Rate: 0.41%
Updated: September 12 2023
replicate

dreamshaper-xl-lightning is a Stable Diffusion model that has been fine-tuned on SDXL

Total runs: 105.1K
Run Growth: 6.7K
Growth Rate: 6.37%
Updated: February 27 2024
replicate

Phi-3-Mini-4K-Instruct is a 3.8B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets

Total runs: 81.7K
Run Growth: 0
Growth Rate: 0.00%
Updated: July 03 2024
replicate

SDXL_Niji_Special Edition

Total runs: 64.6K
Run Growth: 3.3K
Growth Rate: 5.11%
Updated: November 14 2023
replicate

PixArt-Alpha 1024px is a transformer-based text-to-image diffusion system trained on text embeddings from T5

Total runs: 64.0K
Run Growth: 11.8K
Growth Rate: 18.44%
Updated: December 04 2023
replicate

MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model

Total runs: 55.5K
Run Growth: 400
Growth Rate: 0.72%
Updated: December 05 2023
replicate

Dreamshaper-7 img2img with LCM LoRA for faster inference

Total runs: 55.1K
Run Growth: 0
Growth Rate: 0.00%
Updated: November 17 2023
replicate

AI-driven audio enhancement for your audio files, powered by Resemble AI

Total runs: 52.3K
Run Growth: 12.8K
Growth Rate: 24.47%
Updated: December 15 2023
replicate

Ostris AI-Toolkit for Flux LoRA Training (DEPRECATED. Please use: ostris/flux-dev-lora-trainer)

Total runs: 49.8K
Run Growth: 2.9K
Growth Rate: 5.82%
Updated: August 18 2024
replicate

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

Total runs: 48.6K
Run Growth: 3.0K
Growth Rate: 6.17%
Updated: June 26 2024
replicate

Implementation of SDXL RealVisXL_V1.0

Total runs: 44.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: September 13 2023
replicate

SDXL Image Blending

Total runs: 42.4K
Run Growth: 0
Growth Rate: 0.00%
Updated: December 12 2023
replicate

(Academic and Non-commercial use only) Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization

Total runs: 39.5K
Run Growth: 400
Growth Rate: 1.01%
Updated: January 08 2024
replicate

BakLLaVA-1 is a Mistral 7B base augmented with the LLaVA 1.5 architecture

Total runs: 39.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: October 24 2023
replicate

lmsys/vicuna-13b-v1.3

Total runs: 38.4K
Run Growth: 0
Growth Rate: 0.00%
Updated: June 30 2023
replicate

Mistral-7B-v0.1 fine tuned for chat with the Dolphin dataset (an open-source implementation of Microsoft's Orca)

Total runs: 35.0K
Run Growth: 500
Growth Rate: 1.43%
Updated: October 31 2023
replicate

Real-ESRGAN with optional face correction and adjustable upscale (for larger images)

Total runs: 34.3K
Run Growth: 300
Growth Rate: 0.87%
Updated: July 17 2023
replicate

Gemma2 2b by Google

Total runs: 33.1K
Run Growth: 0
Growth Rate: 0.00%
Updated: August 01 2024
replicate

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate SDXL images with an image prompt

Total runs: 31.1K
Run Growth: 400
Growth Rate: 1.29%
Updated: November 12 2023
replicate

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

Total runs: 30.9K
Run Growth: 1.6K
Growth Rate: 5.18%
Updated: June 26 2024
replicate

lmsys/vicuna-7b-v1.3

Total runs: 28.6K
Run Growth: 100
Growth Rate: 0.35%
Updated: June 30 2023
replicate

(Research only) IP-Adapter-FaceID can generate various style images conditioned on a face with only text prompts

Total runs: 28.3K
Run Growth: 0
Growth Rate: 0.00%
Updated: December 21 2023
replicate

Meta's Llama 2 7b Chat - GPTQ

Total runs: 20.3K
Run Growth: 100
Growth Rate: 0.49%
Updated: July 24 2023
replicate

sdxs-512-0.9 can generate high-resolution images in real-time based on prompt texts, trained using score distillation and feature matching

Total runs: 18.8K
Run Growth: 0
Growth Rate: 0.00%
Updated: March 28 2024
replicate

Stylized Audio-Driven Single Image Talking Face Animation

Total runs: 18.6K
Run Growth: 200
Growth Rate: 1.08%
Updated: October 08 2023
replicate

Meta's Llama 2 13b Chat - GPTQ

Total runs: 18.5K
Run Growth: 0
Growth Rate: 0.00%
Updated: July 25 2023
replicate

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

Total runs: 17.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: January 24 2024
replicate

ThinkDiffusionXL is a go-to model capable of amazing photorealism that's also versatile enough to generate high-quality images across a variety of styles and subjects without needing to be a prompting genius

Total runs: 15.5K
Run Growth: 100
Growth Rate: 0.65%
Updated: November 07 2023
replicate

This is wizard-vicuna-13b trained with a subset of the dataset - responses that contained alignment / moralizing were removed

Total runs: 15.1K
Run Growth: 0
Growth Rate: 0.00%
Updated: April 26 2024
replicate

Hyper FLUX 16-step by ByteDance

Total runs: 15.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: August 28 2024
replicate

Mistral-7B-v0.1 fine tuned for chat with the Dolphin dataset (an open-source implementation of Microsoft's Orca)

Total runs: 13.4K
Run Growth: 0
Growth Rate: 0.00%
Updated: October 31 2023
replicate

Image-to-video - SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction

Total runs: 12.0K
Run Growth: 1000
Growth Rate: 8.33%
Updated: November 23 2023
replicate

InterpAny-Clearer: Clearer anytime frame interpolation & Manipulated interpolation

Total runs: 11.4K
Run Growth: 0
Growth Rate: 0.00%
Updated: November 30 2023
replicate

Segments an audio recording based on who is speaking (on A100)

Total runs: 11.4K
Run Growth: 0
Growth Rate: 0.00%
Updated: July 22 2023
replicate

Latest model in the Qwen family for chatting with video and image models

Total runs: 11.1K
Run Growth: 0
Growth Rate: 0.00%
Updated: December 21 2024
replicate

(Research only) Moondream1 is a vision language model that performs on par with models twice its size

Total runs: 10.7K
Run Growth: 0
Growth Rate: 0.00%
Updated: January 25 2024
replicate

Image to Image enhancer using DemoFusion

Total runs: 10.4K
Run Growth: 100
Growth Rate: 0.96%
Updated: December 09 2023
replicate

Open diffusion model for high-quality video generation

Total runs: 10.3K
Run Growth: 100
Growth Rate: 0.97%
Updated: October 19 2023
replicate

Auto fuse a user's face onto the template image, with a similar appearance to the user

Total runs: 10.1K
Run Growth: 700
Growth Rate: 6.93%
Updated: November 16 2023
replicate

DemoFusion: Democratising High-Resolution Image Generation With No 💰

Total runs: 9.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: December 04 2023
replicate

Segment Anything 2 (SAM2) by Meta - Automatic mask generation

Total runs: 8.6K
Run Growth: 1.3K
Growth Rate: 15.12%
Updated: July 31 2024
replicate

Implementation of SDXL RealVisXL_V2.0 img2img

Total runs: 8.6K
Run Growth: 0
Growth Rate: 0.00%
Updated: November 07 2023
replicate

Phi-3-Mini-128K-Instruct is a 3.8 billion-parameter, lightweight, state-of-the-art open model trained using the Phi-3 datasets

Total runs: 8.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: April 26 2024
replicate

Cog wrapper for Ollama llama3:70b

Total runs: 6.6K
Run Growth: 6.5K
Growth Rate: 98.44%
Updated: July 09 2024
replicate

360 Panorama SDXL image with inpainted wrapping seam

Total runs: 6.3K
Run Growth: 100
Growth Rate: 1.59%
Updated: September 10 2023
replicate

Convert your videos to DensePose and use it with MagicAnimate

Total runs: 5.7K
Run Growth: 100
Growth Rate: 1.75%
Updated: December 06 2023
replicate

Fuyu-8B is a multi-modal text and image transformer trained by Adept AI

Total runs: 4.6K
Run Growth: 0
Growth Rate: 0.00%
Updated: October 20 2023
replicate

Qwen1.5 is the beta version of Qwen2, a transformer-based decoder-only language model pretrained on a large amount of data

Total runs: 4.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: February 07 2024
replicate

Controlnet v1.1 - Tile Version

Total runs: 4.0K
Run Growth: 0
Growth Rate: 0.00%
Updated: November 27 2023
replicate

SDXL using DeepCache

Total runs: 3.8K
Run Growth: 0
Growth Rate: 0.00%
Updated: January 08 2024
replicate

nomic-embed-text-v1 is 8192 context length text encoder that surpasses OpenAI text-embedding-ada-002 and text-embedding-3-small performance on short and long context tasks

Total runs: 3.6K
Run Growth: 100
Growth Rate: 2.78%
Updated: February 13 2024
replicate

Playground v2 is a diffusion-based text-to-image generative model trained from scratch. Try out all 3 models here

Total runs: 3.6K
Run Growth: 0
Growth Rate: 0.00%
Updated: December 08 2023
replicate

Segmind Stable Diffusion Model (SSD-1B) img2img

Total runs: 3.6K
Run Growth: 0
Growth Rate: 0.00%
Updated: November 03 2023
replicate

A combination of ip_adapter SDv1.5 and mediapipe-face to inpaint a face

Total runs: 3.6K
Run Growth: 200
Growth Rate: 5.56%
Updated: November 15 2023
replicate

Phi-2 by Microsoft

Total runs: 3.5K
Run Growth: 600
Growth Rate: 17.14%
Updated: January 31 2024
replicate

Implementation of SDXL RealVisXL_V1.0 img2img

Total runs: 3.4K
Run Growth: 0
Growth Rate: 0.00%
Updated: November 02 2023
replicate

A Flux LoRA trained on watercolor style photos

Total runs: 3.2K
Run Growth: 700
Growth Rate: 21.88%
Updated: August 15 2024