simpo-training Skill
Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO. Published by NousResearch in hermes-agent.
What is simpo-training Skill?
Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO. Published by NousResearch in hermes-agent. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Data analysis
- Data analysis use cases
Technical details
- Install or run with Copy skill directory
When to use simpo-training Skill
- Use it for data analysis.
Built with
Editorial notes
Source
- Creator: NousResearch
- Repository: NousResearch/hermes-agent
- Skill file: optional-skills/mlops/simpo/SKILL.md
What it does
Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.
Skill instructions
SimPO - Simple Preference Optimization Quick start SimPO is a reference-free preference optimization method that outperforms DPO without needing a reference model. Installation: bash Create environment conda create -n simpo python=3.10 && conda activate simpo Install PyTorch 2.2.2 Visit: https://pytorch.org/get-started/locally/ Install alignment-handbook git clone https://github.com/huggingface/alignment-handbook.git cd alignment-handbook python -m pip install . Install Flash Attention 2 python -m pip install flash-attn --no-build-isolation Training (Mistral 7B): bash ACCELERATELOGLEVEL=info accelerate launch \ --configfile accelerateconfigs/deepspeedzero3.yaml \ scripts/runsimpo.py \ trainingconfigs/mistral-7b-base-simpo.yaml Common workflows Workflow 1: Train from base model (Mistral 7B) Config (mistral-7b-base-simpo.yaml): yaml Model modelnameorpath: mistralai/Mistral-7B-v0.1 torchdtype: bfloat16 Dataset datasetmixer: HuggingFaceH4/ultrafeedbackbinarized: 1.0 datasetsplits: - trainp
Explore related resources
Frequently asked questions
What is simpo-training?
simpo-training is a open-source AI agent skill with Copy skill directory. Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO.
Who is simpo-training best for?
simpo-training is best for reusing agent instructions, scripts, and references, data analysis workflows.
How do I install simpo-training?
Install or run simpo-training using Copy skill directory. Check simpo-training for the latest setup command.
Is simpo-training actively maintained?
simpo-training may need a closer maintenance check before production use.
Auto-fetched from GitHub.
