BAAI / LLARA-passage

huggingface.co
Total runs: 22
24-hour runs: 0
7-day runs: 3
30-day runs: -76
Model's Last Updated: May 13 2024
sentence-similarity

Introduction of LLARA-passage

Model Details of LLARA-passage

For more details please refer to our github repo: https://github.com/FlagOpen/FlagEmbedding

LLARA ( paper )

In this project, we introduce LLaRA:

  • EBAE: Embedding-Based Auto-Encoding.
  • EBAR: Embedding-Based Auto-Regression.
Usage
import torch
from transformers import AutoModel, AutoTokenizer, LlamaModel

def get_query_inputs(queries, tokenizer, max_length=512):
    prefix = '"'
    suffix = '", predict the following passage within eight words: <s9><s10><s11><s12><s13><s14><s15><s16>'
    prefix_ids = tokenizer(prefix, return_tensors=None)['input_ids']
    suffix_ids = tokenizer(suffix, return_tensors=None)['input_ids'][1:]
    queries_inputs = []
    for query in queries:
        inputs = tokenizer(query,
                           return_tensors=None,
                           max_length=max_length,
                           truncation=True,
                           add_special_tokens=False)
        inputs['input_ids'] = prefix_ids + inputs['input_ids'] + suffix_ids
        inputs['attention_mask'] = [1] * len(inputs['input_ids'])
        queries_inputs.append(inputs)
    return tokenizer.pad(
            queries_inputs,
            padding=True,
            max_length=max_length,
            pad_to_multiple_of=8,
            return_tensors='pt',
        )

def get_passage_inputs(passages, tokenizer, max_length=512):
    prefix = '"'
    suffix = '", summarize the above passage within eight words: <s1><s2><s3><s4><s5><s6><s7><s8>'
    prefix_ids = tokenizer(prefix, return_tensors=None)['input_ids']
    suffix_ids = tokenizer(suffix, return_tensors=None)['input_ids'][1:]
    passages_inputs = []
    for passage in passages:
        inputs = tokenizer(passage,
                           return_tensors=None,
                           max_length=max_length,
                           truncation=True,
                           add_special_tokens=False)
        inputs['input_ids'] = prefix_ids + inputs['input_ids'] + suffix_ids
        inputs['attention_mask'] = [1] * len(inputs['input_ids'])
        passages_inputs.append(inputs)
    return tokenizer.pad(
            passages_inputs,
            padding=True,
            max_length=max_length,
            pad_to_multiple_of=8,
            return_tensors='pt',
        )

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained('BAAI/LLARA-passage')
model = AutoModel.from_pretrained('BAAI/LLARA-passage')

# Define query and passage inputs
query = "What is llama?"
title = "Llama"
passage = "The llama is a domesticated South American camelid, widely used as a meat and pack animal by Andean cultures since the pre-Columbian era."
query_input = get_query_inputs([query], tokenizer)
passage_input = get_passage_inputs([passage], tokenizer)


with torch.no_grad():
    # compute query embedding
    query_outputs = model(**query_input, return_dict=True, output_hidden_states=True)
    query_embedding = query_outputs.hidden_states[-1][:, -8:, :]
    query_embedding = torch.mean(query_embedding, dim=1)
    query_embedding = torch.nn.functional.normalize(query_embedding, dim=-1)

    # compute passage embedding
    passage_outputs = model(**passage_input, return_dict=True, output_hidden_states=True)
    passage_embeddings = passage_outputs.hidden_states[-1][:, -8:, :]
    passage_embeddings = torch.mean(passage_embeddings, dim=1)
    passage_embeddings = torch.nn.functional.normalize(passage_embeddings, dim=-1)

    # compute similarity score
    score = query_embedding @ passage_embeddings.T
    print(score)
Acknowledgement

Thanks to the authors of open-sourced datasets, including MSMARCO, BEIR, etc. Thanks to the open-sourced libraries like Pyserini .

Citation

If you find this repository useful, please consider giving a star :star: and citation

@misc{li2023making,
      title={Making Large Language Models A Better Foundation For Dense Retrieval}, 
      author={Chaofan Li and Zheng Liu and Shitao Xiao and Yingxia Shao},
      year={2023},
      eprint={2312.15503},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Runs of BAAI LLARA-passage on huggingface.co

22
Total runs
0
24-hour runs
2
3-day runs
3
7-day runs
-76
30-day runs

More Information About LLARA-passage huggingface.co Model

More LLARA-passage license Visit here:

https://choosealicense.com/licenses/mit

LLARA-passage huggingface.co

LLARA-passage huggingface.co is an AI model on huggingface.co that provides LLARA-passage's model effect (), which can be used instantly with this BAAI LLARA-passage model. huggingface.co supports a free trial of the LLARA-passage model, and also provides paid use of the LLARA-passage. Support call LLARA-passage model through api, including Node.js, Python, http.

LLARA-passage huggingface.co Url

https://huggingface.co/BAAI/LLARA-passage

BAAI LLARA-passage online free

LLARA-passage huggingface.co is an online trial and call api platform, which integrates LLARA-passage's modeling effects, including api services, and provides a free online trial of LLARA-passage, you can try LLARA-passage online for free by clicking the link below.

BAAI LLARA-passage online free url in huggingface.co:

https://huggingface.co/BAAI/LLARA-passage

LLARA-passage install

LLARA-passage is an open source model from GitHub that offers a free installation service, and any user can find LLARA-passage on GitHub to install. At the same time, huggingface.co provides the effect of LLARA-passage install, users can directly use LLARA-passage installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

LLARA-passage install url in huggingface.co:

https://huggingface.co/BAAI/LLARA-passage

Url of LLARA-passage

LLARA-passage huggingface.co Url

Provider of LLARA-passage huggingface.co

BAAI
ORGANIZATIONS

Other API from BAAI

huggingface.co

Total runs: 4.9M
Run Growth: 238.0K
Growth Rate: 4.70%
Updated: February 22 2024
huggingface.co

Total runs: 2.2M
Run Growth: -1.6M
Growth Rate: -70.83%
Updated: February 21 2024
huggingface.co

Total runs: 2.2M
Run Growth: 34.0K
Growth Rate: 1.57%
Updated: July 03 2024
huggingface.co

Total runs: 1.6M
Run Growth: -95.3K
Growth Rate: -5.87%
Updated: February 21 2024
huggingface.co

Total runs: 922.5K
Run Growth: -114.0K
Growth Rate: -12.20%
Updated: June 24 2024
huggingface.co

Total runs: 810.2K
Run Growth: 362.4K
Growth Rate: 43.54%
Updated: April 02 2024
huggingface.co

Total runs: 702.1K
Run Growth: -7.5K
Growth Rate: -1.05%
Updated: December 13 2023
huggingface.co

Total runs: 579.5K
Run Growth: 156.4K
Growth Rate: 27.02%
Updated: October 12 2023
huggingface.co

Total runs: 137.4K
Run Growth: 15.6K
Growth Rate: 11.29%
Updated: October 12 2023
huggingface.co

Total runs: 122.8K
Run Growth: -63.0K
Growth Rate: -49.63%
Updated: November 14 2023
huggingface.co

Total runs: 77.4K
Run Growth: 21.6K
Growth Rate: 28.71%
Updated: April 17 2024
huggingface.co

Total runs: 22.1K
Run Growth: 18.0K
Growth Rate: 66.40%
Updated: October 12 2023
huggingface.co

Total runs: 21.0K
Run Growth: -1.5K
Growth Rate: -7.30%
Updated: January 15 2025
huggingface.co

Total runs: 11.7K
Run Growth: -17.9K
Growth Rate: -126.21%
Updated: October 12 2023
huggingface.co

Total runs: 6.3K
Run Growth: -530
Growth Rate: -8.38%
Updated: December 26 2022
huggingface.co

Total runs: 5.8K
Run Growth: 1.9K
Growth Rate: 32.81%
Updated: September 21 2023
huggingface.co

Total runs: 5.7K
Run Growth: 2.0K
Growth Rate: 30.79%
Updated: February 22 2024
huggingface.co

Total runs: 5.6K
Run Growth: 1.4K
Growth Rate: 25.86%
Updated: August 15 2024
huggingface.co

Total runs: 3.9K
Run Growth: 1.1K
Growth Rate: 27.68%
Updated: October 12 2023
huggingface.co

Total runs: 3.0K
Run Growth: 1.2K
Growth Rate: 40.86%
Updated: August 15 2024
huggingface.co

Total runs: 2.0K
Run Growth: -1.2K
Growth Rate: -55.97%
Updated: October 23 2024
huggingface.co

Total runs: 2.0K
Run Growth: 282
Growth Rate: 18.50%
Updated: October 23 2024
huggingface.co

Total runs: 1.9K
Run Growth: 220
Growth Rate: 10.80%
Updated: February 07 2024
huggingface.co

Total runs: 1.6K
Run Growth: -13
Growth Rate: -0.76%
Updated: September 18 2023
huggingface.co

Total runs: 1.3K
Run Growth: -650
Growth Rate: -48.51%
Updated: November 28 2024
huggingface.co

Total runs: 1.2K
Run Growth: 429
Growth Rate: 34.10%
Updated: October 24 2024
huggingface.co

Total runs: 974
Run Growth: 854
Growth Rate: 84.98%
Updated: April 02 2024
huggingface.co

Total runs: 766
Run Growth: 614
Growth Rate: 78.82%
Updated: January 15 2025
huggingface.co

Total runs: 690
Run Growth: 409
Growth Rate: 57.85%
Updated: October 27 2023
huggingface.co

Total runs: 645
Run Growth: 372
Growth Rate: 56.79%
Updated: March 07 2024
huggingface.co

Total runs: 626
Run Growth: 524
Growth Rate: 92.25%
Updated: April 19 2024
huggingface.co

Total runs: 483
Run Growth: -285
Growth Rate: -59.62%
Updated: June 07 2024
huggingface.co

Total runs: 402
Run Growth: 349
Growth Rate: 83.89%
Updated: January 14 2025
huggingface.co

Total runs: 343
Run Growth: 214
Growth Rate: 67.08%
Updated: June 24 2024
huggingface.co

Total runs: 289
Run Growth: 245
Growth Rate: 84.19%
Updated: January 01 2025
huggingface.co

Total runs: 273
Run Growth: 394
Growth Rate: 99.24%
Updated: April 18 2023
huggingface.co

Total runs: 248
Run Growth: 16
Growth Rate: 7.08%
Updated: August 15 2024
huggingface.co

Total runs: 204
Run Growth: 102
Growth Rate: 56.67%
Updated: December 21 2023
huggingface.co

Total runs: 146
Run Growth: -121
Growth Rate: -82.88%
Updated: June 24 2024
huggingface.co

Total runs: 122
Run Growth: -140
Growth Rate: -116.67%
Updated: October 28 2023
huggingface.co

Total runs: 116
Run Growth: 74
Growth Rate: 62.71%
Updated: August 23 2023
huggingface.co

Total runs: 103
Run Growth: 41
Growth Rate: 46.59%
Updated: August 23 2023
huggingface.co

Total runs: 102
Run Growth: -14
Growth Rate: -17.07%
Updated: August 15 2024
huggingface.co

Total runs: 94
Run Growth: 74
Growth Rate: 78.72%
Updated: December 31 2022
huggingface.co

Total runs: 71
Run Growth: -13
Growth Rate: -19.12%
Updated: December 21 2023
huggingface.co

Total runs: 55
Run Growth: -107
Growth Rate: -187.72%
Updated: June 21 2024
huggingface.co

Total runs: 42
Run Growth: 99
Growth Rate: 86.09%
Updated: January 20 2025
huggingface.co

Total runs: 41
Run Growth: -13
Growth Rate: -32.50%
Updated: August 15 2024
huggingface.co

Total runs: 32
Run Growth: -6
Growth Rate: -18.75%
Updated: August 28 2024
huggingface.co

Total runs: 30
Run Growth: 7
Growth Rate: 24.14%
Updated: July 24 2023
huggingface.co

Total runs: 29
Run Growth: 17
Growth Rate: 58.62%
Updated: July 05 2024
huggingface.co

Total runs: 29
Run Growth: -22
Growth Rate: -75.86%
Updated: February 07 2024
huggingface.co

Total runs: 26
Run Growth: 1
Growth Rate: 3.85%
Updated: December 25 2023
huggingface.co

Total runs: 26
Run Growth: 22
Growth Rate: 100.00%
Updated: February 11 2025