Total runs:
1.9M
Run Growth:
253.3K
Growth Rate:
13.17%
deepset / gbert-base-germandpr-reranking
Model's Last Updated: September 26 2024
text-classification
Introduction of gbert-base-germandpr-reranking
Model Details of gbert-base-germandpr-reranking
Overview
Language model:
gbert-base-germandpr-reranking
Language:
German
Training data:
GermanDPR train set (~ 56MB)
Eval data:
GermanDPR test set (~ 6MB)
Infrastructure
: 1x V100 GPU
Published
: June 3rd, 2021
Details
- We trained a text pair classification model in FARM, which can be used for reranking in document retrieval tasks. To this end, the classifier calculates the similarity of the query and each retrieved top k document (e.g., k=10). The top k documents are then sorted by their similarity scores. The document most similar to the query is the best.
Hyperparameters
batch_size = 16
n_epochs = 2
max_seq_len = 512 tokens for question and passage concatenated
learning_rate = 2e-5
lr_schedule = LinearWarmup
embeds_dropout_prob = 0.1
Performance
We use the GermanDPR test dataset as ground truth labels and run two experiments to compare how a BM25 retriever performs with or without reranking with our model. The first experiment runs retrieval on the full German Wikipedia (more than 2 million passages) and second experiment runs retrieval on the GermanDPR dataset only (not more than 5000 passages). Both experiments use 1025 queries. Note that the second experiment is evaluating on a much simpler task because of the smaller dataset size, which explains strong BM25 retrieval performance.
Full German Wikipedia (more than 2 million passages):
BM25 Retriever without Reranking
- recall@3: 0.4088 (419 / 1025)
- mean_reciprocal_rank@3: 0.3322
BM25 Retriever with Reranking Top 10 Documents
- recall@3: 0.5200 (533 / 1025)
- mean_reciprocal_rank@3: 0.4800
GermanDPR Test Dataset only (not more than 5000 passages):
BM25 Retriever without Reranking
- recall@3: 0.9102 (933 / 1025)
- mean_reciprocal_rank@3: 0.8528
BM25 Retriever with Reranking Top 10 Documents
- recall@3: 0.9298 (953 / 1025)
- mean_reciprocal_rank@3: 0.8813
Usage
In haystack
You can load the model in haystack for reranking the documents returned by a Retriever:
...
retriever = ElasticsearchRetriever(document_store=document_store)
ranker = FARMRanker(model_name_or_path="deepset/gbert-base-germandpr-reranking")
...
p = Pipeline()
p.add_node(component=retriever, name="ESRetriever", inputs=["Query"])
p.add_node(component=ranker, name="Ranker", inputs=["ESRetriever"])
)
About us
Our focus: Industry specific language models & large scale QA systems.
Some of our work:
- German BERT (aka "bert-base-german-cased")
- GermanQuAD and GermanDPR datasets and models (aka "gelectra-base-germanquad", "gbert-base-germandpr")
- FARM
- Haystack
Get in touch: Twitter | LinkedIn | Website
By the way: we're hiring!
Runs of deepset gbert-base-germandpr-reranking on huggingface.co
221
Total runs
0
24-hour runs
-92
3-day runs
-19
7-day runs
79
30-day runs
More Information About gbert-base-germandpr-reranking huggingface.co Model
Url of gbert-base-germandpr-reranking
gbert-base-germandpr-reranking huggingface.co Url
Provider of gbert-base-germandpr-reranking huggingface.co
Other API from deepset
Updated: September 24 2024
Total runs:
223.9K
Run Growth:
22.4K
Growth Rate:
10.03%
Total runs:
103.4K
Run Growth:
20.0K
Growth Rate:
19.51%
Total runs:
51.8K
Run Growth:
38.8K
Growth Rate:
75.02%
Total runs:
41.1K
Run Growth:
29.7K
Growth Rate:
74.80%
Total runs:
37.9K
Run Growth:
9.6K
Growth Rate:
26.55%
Total runs:
32.1K
Run Growth:
-67.0K
Growth Rate:
-210.19%
Total runs:
27.7K
Run Growth:
3.2K
Growth Rate:
11.47%
Total runs:
13.4K
Run Growth:
-1.2M
Growth Rate:
-8767.40%
Total runs:
12.3K
Run Growth:
5.1K
Growth Rate:
49.25%
Total runs:
10.7K
Run Growth:
1.5K
Growth Rate:
14.77%
Total runs:
10.3K
Run Growth:
6.6K
Growth Rate:
72.72%
Total runs:
5.6K
Run Growth:
387
Growth Rate:
6.85%
Total runs:
3.9K
Run Growth:
2.1K
Growth Rate:
55.47%
Total runs:
3.4K
Run Growth:
-404
Growth Rate:
-12.10%
Total runs:
2.0K
Run Growth:
519
Growth Rate:
26.55%
Total runs:
1.2K
Run Growth:
468
Growth Rate:
40.80%
Total runs:
1.1K
Run Growth:
-36
Growth Rate:
-3.38%
Total runs:
1.0K
Run Growth:
230
Growth Rate:
22.79%
Total runs:
703
Run Growth:
-1.3K
Growth Rate:
-189.06%
Total runs:
655
Run Growth:
197
Growth Rate:
33.56%
Total runs:
544
Run Growth:
504
Growth Rate:
95.27%
Total runs:
469
Run Growth:
-300
Growth Rate:
-65.22%
Total runs:
457
Run Growth:
282
Growth Rate:
62.39%
Total runs:
299
Run Growth:
190
Growth Rate:
95.96%
Total runs:
292
Run Growth:
269
Growth Rate:
89.97%
Total runs:
269
Run Growth:
-53
Growth Rate:
-19.63%
Total runs:
247
Run Growth:
173
Growth Rate:
70.90%
Total runs:
219
Run Growth:
4
Growth Rate:
1.90%
Total runs:
203
Run Growth:
88
Growth Rate:
85.44%
Total runs:
202
Run Growth:
96
Growth Rate:
94.12%
Total runs:
162
Run Growth:
139
Growth Rate:
86.34%
Total runs:
161
Run Growth:
-38
Growth Rate:
-23.60%
Total runs:
144
Run Growth:
118
Growth Rate:
81.38%
Total runs:
137
Run Growth:
120
Growth Rate:
86.96%
Total runs:
128
Run Growth:
96
Growth Rate:
75.00%
Total runs:
112
Run Growth:
102
Growth Rate:
92.73%
Total runs:
105
Run Growth:
34
Growth Rate:
33.33%
Total runs:
104
Run Growth:
61
Growth Rate:
58.65%
Total runs:
67
Run Growth:
39
Growth Rate:
60.94%
Total runs:
60
Run Growth:
-72
Growth Rate:
-150.00%
Total runs:
60
Run Growth:
45
Growth Rate:
70.31%
Total runs:
24
Run Growth:
10
Growth Rate:
35.71%
Total runs:
20
Run Growth:
-170
Growth Rate:
-850.00%