huggingface-accelerate Skill

Source

• Creator: NousResearch
• Repository: NousResearch/hermes-agent
• Skill file: optional-skills/mlops/accelerate/SKILL.md

What it does

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

Skill instructions

HuggingFace Accelerate - Unified Distributed Training Quick start Accelerate simplifies distributed training to 4 lines of code. Installation: bash pip install accelerate Convert PyTorch script (4 lines): python import torch + from accelerate import Accelerator + accelerator = Accelerator() model = torch.nn.Transformer() optimizer = torch.optim.Adam(model.parameters()) dataloader = torch.utils.data.DataLoader(dataset) + model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader) for batch in dataloader: optimizer.zerograd() loss = model(batch) - loss.backward() + accelerator.backward(loss) optimizer.step() Run (single command): bash accelerate launch train.py Common workflows Workflow 1: From single GPU to multi-GPU Original script: python train.py import torch model = torch.nn.Linear(10, 2).to('cuda') optimizer = torch.optim.Adam(model.parameters()) dataloader = torch.utils.data.DataLoader(dataset, batchsize=32) for epoch in range(10): for batch in dataloader: bat