This repo contains GGUF quants of the model. If you need the original weights, please find them
here
.
This is a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus.
experimental because trained on top of instruct; but turned out amazing; hence code named magnum-alter, the original model that kickstarted the v4 family
<|im_start|>system
system prompt<|im_end|>
<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
SillyTavern templates
Below are Instruct and Context templates for use within SillyTavern.
{
"system_prompt":"Currently, your role is {{char}}, described in detail below. As {{char}}, continue the narrative exchange with {{user}}.\n\n<Guidelines>\n• Maintain the character persona but allow it to evolve with the story.\n• Be creative and proactive. Drive the story forward, introducing plotlines and events when relevant.\n• All types of outputs are encouraged; respond accordingly to the narrative.\n• Include dialogues, actions, and thoughts in each response.\n• Utilize all five senses to describe scenarios within {{char}}'s dialogue.\n• Use emotional symbols such as "!"and"~"inappropriatecontexts.\n•Incorporateonomatopoeiawhensuitable.\n•Allowtimefor {{user}} torespondwiththeirowninput, respectingtheiragency.\n•ActassecondarycharactersandNPCsasneeded, andremovethemwhenappropriate.\n•WhenpromptedforanOutofCharacter [OOC:] reply, answerneutrallyandinplaintext, notas {{char}}.\n</Guidelines>\n\n<Forbidden>\n•Usingexcessiveliteraryembellishmentsandpurpleproseunlessdictatedby {{char}}'s persona.\n• Writing for, speaking, thinking, acting, or replying as {{user}} in your response.\n• Repetitive and monotonous outputs.\n• Positivity bias in your replies.\n• Being overly extreme or NSFW when the narrative context is inappropriate.\n</Forbidden>\n\nFollow the instructions in <Guidelines></Guidelines>, avoiding the items listed in <Forbidden></Forbidden>.", "input_sequence": "<|im_start|>user\n", "output_sequence": "<|im_start|>assistant\n", "last_output_sequence": "", "system_sequence": "<|im_start|>system\n", "stop_sequence": "<|im_end|>", "wrap": false, "macro": true, "names": true, "names_force_groups": true, "activation_regex": "", "system_sequence_prefix": "", "system_sequence_suffix": "", "first_output_sequence": "", "skip_examples": false, "output_suffix": "<|im_end|>\n", "input_suffix": "<|im_end|>\n", "system_suffix": "<|im_end|>\n", "user_alignment_message": "", "system_same_as_user": false, "last_system_sequence": "", "name": "Magnum ChatML"}
Axolotl config
See axolotl config
base_model:/workspace/data/models/Qwen2.5-72B-Instructmodel_type:AutoModelForCausalLMtokenizer_type:AutoTokenizerplugins:-axolotl.integrations.liger.LigerPluginliger_rope:trueliger_rms_norm:trueliger_swiglu:trueliger_fused_linear_cross_entropy:trueload_in_8bit:falseload_in_4bit:falsestrict:falsedatasets:-path:anthracite-org/c2_logs_32k_llama3_qwen2_v1.2type:sharegptconversation:chatml-path:anthracite-org/kalo-opus-instruct-22k-no-refusaltype:sharegptconversation:chatml-path:lodrick-the-lafted/kalo-opus-instruct-3k-filteredtype:sharegptconversation:chatml-path:anthracite-org/nopm_claude_writing_fixedtype:sharegptconversation:chatml-path:anthracite-org/kalo_opus_misc_240827type:sharegptconversation:chatml-path:anthracite-org/kalo_misc_part2type:sharegptconversation:chatml#chat_template: chatmlshuffle_merged_datasets:true#default_system_message: "You are an assistant that responds to the user."dataset_prepared_path:/workspace/data/magnum-72b-dataval_set_size:0.0output_dir:/workspace/data/72b-fft-outsequence_len:32768sample_packing:truepad_to_sequence_len:trueadapter:lora_model_dir:lora_r:lora_alpha:lora_dropout:lora_target_linear:lora_fan_in_fan_out:wandb_project:72b-magnum-fftwandb_entity:wandb_watch:wandb_name:alter-attempt-01wandb_log_model:gradient_accumulation_steps:2micro_batch_size:1num_epochs:2optimizer:adamw_bnb_8bitlr_scheduler:cosinelearning_rate:0.000004train_on_inputs:falsegroup_by_length:falsebf16:autofp16:tf32:falsegradient_checkpointing:trueearly_stopping_patience:resume_from_checkpoint:local_rank:logging_steps:1xformers_attention:flash_attention:truewarmup_steps:40evals_per_epoch:eval_table_size:eval_max_new_tokens:saves_per_epoch:2debug:deepspeed:deepspeed_configs/zero3_bf16.jsonweight_decay:0.01fsdp:fsdp_config:special_tokens:
Credits
We'd like to thank
DoctorShotgun
for sponsoring the compute for this train.
We would also like to thank all members of Anthracite who made this finetune possible.
magnum-v4-72b-gguf huggingface.co is an AI model on huggingface.co that provides magnum-v4-72b-gguf's model effect (), which can be used instantly with this anthracite-org magnum-v4-72b-gguf model. huggingface.co supports a free trial of the magnum-v4-72b-gguf model, and also provides paid use of the magnum-v4-72b-gguf. Support call magnum-v4-72b-gguf model through api, including Node.js, Python, http.
magnum-v4-72b-gguf huggingface.co is an online trial and call api platform, which integrates magnum-v4-72b-gguf's modeling effects, including api services, and provides a free online trial of magnum-v4-72b-gguf, you can try magnum-v4-72b-gguf online for free by clicking the link below.
anthracite-org magnum-v4-72b-gguf online free url in huggingface.co:
magnum-v4-72b-gguf is an open source model from GitHub that offers a free installation service, and any user can find magnum-v4-72b-gguf on GitHub to install. At the same time, huggingface.co provides the effect of magnum-v4-72b-gguf install, users can directly use magnum-v4-72b-gguf installed effect in huggingface.co for debugging and trial. It also supports api for free installation.