Total runs:
1.8M
Run Growth:
383.5K
Growth Rate:
21.10%
Salesforce / xLAM-v0.1-r
Model's Last Updated: January 25 2025
text-generation
Introduction of xLAM-v0.1-r
Model Details of xLAM-v0.1-r
🎉 GitHub: https://github.com/SalesforceAIResearch/xLAM
🎉 Paper: https://arxiv.org/abs/2402.15506
License: cc-by-nc-4.0
If you already know Mixtral , xLAM-v0.1 is a significant upgrade and better at many things. For the same number of parameters, the model have been fine-tuned across a wide range of agent tasks and scenarios, all while preserving the capabilities of the original model.
xLAM-v0.1-r represents the version 0.1 of the Large Action Model series, with the "-r" indicating it's tagged for research. This model is compatible with VLLM and FastChat platforms.
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Salesforce/xLAM-v0.1-r")
model = AutoModelForCausalLM.from_pretrained("Salesforce/xLAM-v0.1-r", device_map="auto")
messages = [
{"role": "user", "content": "What is your favourite condiment?"},
{"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
{"role": "user", "content": "Do you have mayonnaise recipes?"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
You may need to tune the Temperature setting for different applications. Typically, a lower Temperature is helpful for tasks that require deterministic outcomes. Additionally, for tasks demanding adherence to specific formats or function calls, explicitly including formatting instructions is advisable.
Benchmarks
BOLAA
Webshop
| LLM Name | ZS | ZST | ReaAct | PlanAct | PlanReAct | BOLAA |
|---|---|---|---|---|---|---|
| Llama-2-70B-chat | 0.0089 | 0.0102 | 0.4273 | 0.2809 | 0.3966 | 0.4986 |
| Vicuna-33B | 0.1527 | 0.2122 | 0.1971 | 0.3766 | 0.4032 | 0.5618 |
| Mixtral-8x7B-Instruct-v0.1 | 0.4634 | 0.4592 | 0.5638 | 0.4738 | 0.3339 | 0.5342 |
| GPT-3.5-Turbo | 0.4851 | 0.5058 | 0.5047 | 0.4930 | 0.5436 | 0.6354 |
| GPT-3.5-Turbo-Instruct | 0.3785 | 0.4195 | 0.4377 | 0.3604 | 0.4851 | 0.5811 |
| GPT-4-0613 | 0.5002 | 0.4783 | 0.4616 | 0.7950 | 0.4635 | 0.6129 |
| xLAM-v0.1-r | 0.5201 | 0.5268 | 0.6486 | 0.6573 | 0.6611 | 0.6556 |
HotpotQA
| LLM Name | ZS | ZST | ReaAct | PlanAct | PlanReAct |
|---|---|---|---|---|---|
| Mixtral-8x7B-Instruct-v0.1 | 0.3912 | 0.3971 | 0.3714 | 0.3195 | 0.3039 |
| GPT-3.5-Turbo | 0.4196 | 0.3937 | 0.3868 | 0.4182 | 0.3960 |
| GPT-4-0613 | 0.5801 | 0.5709 | 0.6129 | 0.5778 | 0.5716 |
| xLAM-v0.1-r | 0.5492 | 0.4776 | 0.5020 | 0.5583 | 0.5030 |
AgentLite
Please note: All prompts provided by AgentLite are considered "unseen prompts" for xLAM-v0.1-r, meaning the model has not been trained with data related to these prompts.
Webshop
| LLM Name | Act | ReAct | BOLAA |
|---|---|---|---|
| GPT-3.5-Turbo-16k | 0.6158 | 0.6005 | 0.6652 |
| GPT-4-0613 | 0.6989 | 0.6732 | 0.7154 |
| xLAM-v0.1-r | 0.6563 | 0.6640 | 0.6854 |
HotpotQA
| Easy | Medium | Hard | ||||
|---|---|---|---|---|---|---|
| LLM Name | F1 Score | Accuracy | F1 Score | Accuracy | F1 Score | Accuracy |
| GPT-3.5-Turbo-16k-0613 | 0.410 | 0.350 | 0.330 | 0.25 | 0.283 | 0.20 |
| GPT-4-0613 | 0.611 | 0.47 | 0.610 | 0.480 | 0.527 | 0.38 |
| xLAM-v0.1-r | 0.532 | 0.45 | 0.547 | 0.46 | 0.455 | 0.36 |
ToolBench
| LLM Name | Unseen Insts & Same Set | Unseen Tools & Seen Cat | Unseen Tools & Unseen Cat |
|---|---|---|---|
| TooLlama V2 | 0.4385 | 0.4300 | 0.4350 |
| GPT-3.5-Turbo-0125 | 0.5000 | 0.5150 | 0.4900 |
| GPT-4-0125-preview | 0.5462 | 0.5450 | 0.5050 |
| xLAM-v0.1-r | 0.5077 | 0.5650 | 0.5200 |
MINT-BENCH
| LLM Name | 1-step | 2-step | 3-step | 4-step | 5-step |
|---|---|---|---|---|---|
| GPT-4-0613 | - | - | - | - | 69.45 |
| Claude-Instant-1 | 12.12 | 32.25 | 39.25 | 44.37 | 45.90 |
| xLAM-v0.1-r | 4.10 | 28.50 | 36.01 | 42.66 | 43.96 |
| Claude-2 | 26.45 | 35.49 | 36.01 | 39.76 | 39.93 |
| Lemur-70b-Chat-v1 | 3.75 | 26.96 | 35.67 | 37.54 | 37.03 |
| GPT-3.5-Turbo-0613 | 2.73 | 16.89 | 24.06 | 31.74 | 36.18 |
| AgentLM-70b | 6.48 | 17.75 | 24.91 | 28.16 | 28.67 |
| CodeLlama-34b | 0.17 | 16.21 | 23.04 | 25.94 | 28.16 |
| Llama-2-70b-chat | 4.27 | 14.33 | 15.70 | 16.55 | 17.92 |
Tool-Query
| LLM Name | Success Rate | Progress Rate |
|---|---|---|
| xLAM-v0.1-r | 0.533 | 0.766 |
| DeepSeek-67B | 0.400 | 0.714 |
| GPT-3.5-Turbo-0613 | 0.367 | 0.627 |
| GPT-3.5-Turbo-16k | 0.317 | 0.591 |
| Lemur-70B | 0.283 | 0.720 |
| CodeLlama-13B | 0.250 | 0.525 |
| CodeLlama-34B | 0.133 | 0.600 |
| Mistral-7B | 0.033 | 0.510 |
| Vicuna-13B-16K | 0.033 | 0.343 |
| Llama-2-70B | 0.000 | 0.483 |
Runs of Salesforce xLAM-v0.1-r on huggingface.co
167
Total runs
0
24-hour runs
-13
3-day runs
-48
7-day runs
-583
30-day runs
More Information About xLAM-v0.1-r huggingface.co Model
Url of xLAM-v0.1-r
xLAM-v0.1-r huggingface.co Url
Provider of xLAM-v0.1-r huggingface.co
Other API from Salesforce
Updated: August 01 2023
Total runs:
1.1M
Run Growth:
-401.7K
Growth Rate:
-35.95%
Total runs:
779.7K
Run Growth:
621.2K
Growth Rate:
98.54%
Total runs:
301.4K
Run Growth:
10.0K
Growth Rate:
3.33%
Total runs:
287.2K
Run Growth:
53.6K
Growth Rate:
18.69%
Total runs:
251.1K
Run Growth:
9.1K
Growth Rate:
3.64%
Total runs:
186.8K
Run Growth:
37.5K
Growth Rate:
93.70%
Total runs:
181.1K
Run Growth:
57.5K
Growth Rate:
31.75%
Total runs:
169.1K
Run Growth:
-143.1K
Growth Rate:
-84.67%
Total runs:
98.4K
Run Growth:
-6.8K
Growth Rate:
-7.06%
Total runs:
83.0K
Run Growth:
36.5K
Growth Rate:
43.73%
Total runs:
64.7K
Run Growth:
16.2K
Growth Rate:
25.10%
Total runs:
55.7K
Run Growth:
-887
Growth Rate:
-1.52%
Total runs:
53.0K
Run Growth:
18.6K
Growth Rate:
34.33%
Total runs:
40.5K
Run Growth:
-9.1K
Growth Rate:
-21.71%
Total runs:
35.7K
Run Growth:
-8.8K
Growth Rate:
-24.61%
Total runs:
35.2K
Run Growth:
-1.3K
Growth Rate:
-3.79%
Total runs:
33.0K
Run Growth:
11.9K
Growth Rate:
88.03%
Total runs:
31.0K
Run Growth:
8.7K
Growth Rate:
28.15%
Total runs:
30.5K
Run Growth:
5.8K
Growth Rate:
18.94%
Total runs:
26.3K
Run Growth:
3.7K
Growth Rate:
13.94%
Total runs:
24.4K
Run Growth:
-5.2K
Growth Rate:
-22.97%
Total runs:
21.6K
Run Growth:
9.4K
Growth Rate:
42.76%
Total runs:
18.7K
Run Growth:
12.5K
Growth Rate:
66.83%
Total runs:
12.5K
Run Growth:
-4.5K
Growth Rate:
-36.20%
Total runs:
9.3K
Run Growth:
4.1K
Growth Rate:
44.14%
Total runs:
8.3K
Run Growth:
-3.6K
Growth Rate:
-43.66%
Total runs:
8.3K
Run Growth:
88
Growth Rate:
1.07%
Total runs:
8.0K
Run Growth:
4.1K
Growth Rate:
51.45%
Total runs:
7.0K
Run Growth:
-11.2K
Growth Rate:
-160.01%
Total runs:
5.8K
Run Growth:
-1.4K
Growth Rate:
-23.59%
Total runs:
4.9K
Run Growth:
-3.3K
Growth Rate:
-65.49%
Total runs:
4.0K
Run Growth:
-433
Growth Rate:
-11.17%
Total runs:
3.9K
Run Growth:
673
Growth Rate:
17.48%
Total runs:
3.1K
Run Growth:
298
Growth Rate:
9.63%
Total runs:
3.0K
Run Growth:
1.3K
Growth Rate:
41.56%
Total runs:
3.0K
Run Growth:
2.8K
Growth Rate:
91.81%
Total runs:
3.0K
Run Growth:
-137
Growth Rate:
-4.08%
Total runs:
2.6K
Run Growth:
1.3K
Growth Rate:
45.14%
Total runs:
2.2K
Run Growth:
-2.4K
Growth Rate:
-109.04%
Total runs:
2.1K
Run Growth:
1.3K
Growth Rate:
59.66%
Total runs:
2.0K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1.9K
Run Growth:
1.6K
Growth Rate:
83.06%
Total runs:
1.6K
Run Growth:
1.3K
Growth Rate:
78.84%
Total runs:
1.5K
Run Growth:
1.4K
Growth Rate:
95.33%
Total runs:
1.5K
Run Growth:
1.1K
Growth Rate:
73.07%
Total runs:
1.5K
Run Growth:
63
Growth Rate:
4.29%
Total runs:
1.5K
Run Growth:
-3.3K
Growth Rate:
-228.31%
Total runs:
1.4K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1.4K
Run Growth:
-851
Growth Rate:
-62.53%
Total runs:
1.3K
Run Growth:
83
Growth Rate:
6.49%
Total runs:
1.2K
Run Growth:
-1.4K
Growth Rate:
-117.20%
Total runs:
1.2K
Run Growth:
-516
Growth Rate:
-45.66%
Total runs:
1.1K
Run Growth:
445
Growth Rate:
39.45%
Total runs:
1.1K
Run Growth:
35
Growth Rate:
3.24%
Total runs:
1.0K
Run Growth:
-52
Growth Rate:
-4.98%
Total runs:
1000
Run Growth:
165
Growth Rate:
16.50%
Total runs:
839
Run Growth:
287
Growth Rate:
34.21%
Total runs:
833
Run Growth:
-110
Growth Rate:
-13.21%
Total runs:
824
Run Growth:
70
Growth Rate:
8.50%
Total runs:
710
Run Growth:
346
Growth Rate:
48.73%
Total runs:
538
Run Growth:
0
Growth Rate:
0.00%
Total runs:
469
Run Growth:
-155
Growth Rate:
-33.05%
Total runs:
423
Run Growth:
43
Growth Rate:
10.17%
Total runs:
406
Run Growth:
167
Growth Rate:
41.13%
Total runs:
241
Run Growth:
-410
Growth Rate:
-175.97%
Total runs:
222
Run Growth:
-213
Growth Rate:
-95.95%
Total runs:
217
Run Growth:
-51
Growth Rate:
-23.50%
Total runs:
199
Run Growth:
-41
Growth Rate:
-20.60%
Total runs:
185
Run Growth:
52
Growth Rate:
28.11%
Total runs:
179
Run Growth:
-369
Growth Rate:
-201.64%
Total runs:
157
Run Growth:
-297
Growth Rate:
-189.17%
Total runs:
134
Run Growth:
-337
Growth Rate:
-251.49%
Total runs:
119
Run Growth:
-8.9K
Growth Rate:
-7473.11%
Total runs:
117
Run Growth:
-369
Growth Rate:
-315.38%
Total runs:
116
Run Growth:
86
Growth Rate:
74.14%
Total runs:
113
Run Growth:
106
Growth Rate:
93.81%
Total runs:
108
Run Growth:
0
Growth Rate:
0.00%
Total runs:
107
Run Growth:
91
Growth Rate:
85.05%
Total runs:
105
Run Growth:
89
Growth Rate:
84.76%
Total runs:
100
Run Growth:
-295
Growth Rate:
-295.00%
Total runs:
97
Run Growth:
-127
Growth Rate:
-130.93%
Total runs:
96
Run Growth:
-137
Growth Rate:
-142.71%
Total runs:
87
Run Growth:
-99
Growth Rate:
-113.79%
Total runs:
85
Run Growth:
-206
Growth Rate:
-242.35%
Total runs:
73
Run Growth:
5
Growth Rate:
6.85%
Total runs:
56
Run Growth:
28
Growth Rate:
50.00%
Total runs:
50
Run Growth:
-98
Growth Rate:
-184.91%
Total runs:
41
Run Growth:
34
Growth Rate:
85.00%
Total runs:
39
Run Growth:
33
Growth Rate:
84.62%