castorini / first_mistral

Model Card for first_mistral

first_mistral is a language model trained to act as a listwise reranker, decoding from the first-token logits only to improve efficiency while maintaining effectiveness. first_mistral is built on Mistral-7B-Instruct-v0.3, following FIRST 's strategy, trained using 40K GPT-4 labeled rerank instances from RankZephyr. More details can be found in the paper.

Model description

• Model type: A 7B parameter listwise reranker fine-tuned from Mistral-7B-Instruct-v0.3
• Language(s) (NLP): Primarily English
• License: MIT
• Fine-tuned from model: mistralai/Mistral-7B-Instruct-v0.3

Model Sources

• Repository: https://github.com/castorini/rank_llm
• Paper: https://arxiv.org/abs/2411.05508

Evaluation

At the time of release, first_mistral outperforms the original FIRST implementation on most subsets of the BEIR benchmark. More details that compare other LLM rerankers on more datasets can be found in the paper.

Citation

If you find first_mistral useful for your work, please consider citing:

@ARTICLE{chen2024firstrepro,
  title   = title={An Early FIRST Reproduction and Improvements to Single-Token Decoding for Fast Listwise Reranking},
  author  = {Zijian Chen and Ronak Pradeep and Jimmy Lin},
  year    = {2024},
  journal = {arXiv:2411.05508}
}

If you would like to cite the FIRST methodology, please consider citing:

@ARTICLE{reddy2024first,
  title   = {FIRST: Faster Improved Listwise Reranking with Single Token Decoding},
  author  = {Reddy, Revanth Gangi and Doo, JaeHyeok and Xu, Yifei and Sultan, Md Arafat and Swain, Deevya and Sil, Avirup and Ji, Heng},
  year    = {2024}
  journal = {arXiv:2406.15657},
}