optimizing-attention-flash Skill
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding win Published by NousResearch in hermes-agent.
What is optimizing-attention-flash Skill?
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding win Published by NousResearch in hermes-agent. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Developers using hermes-agent
Technical details
- Install or run with Copy skill directory
When to use optimizing-attention-flash Skill
- Use it for developers using hermes-agent.
Built with
Editorial notes
Source
- Creator: NousResearch
- Repository: NousResearch/hermes-agent
- Skill file: optional-skills/mlops/flash-attention/SKILL.md
What it does
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding win
Skill instructions
Flash Attention - Fast Memory-Efficient Attention Quick start Flash Attention provides 2-4x speedup and 10-20x memory reduction for transformer attention through IO-aware tiling and recomputation. PyTorch native (easiest, PyTorch 2.2+): python import torch import torch.nn.functional as F q = torch.randn(2, 8, 512, 64, device='cuda', dtype=torch.float16) [batch, heads, seq, dim] k = torch.randn(2, 8, 512, 64, device='cuda', dtype=torch.float16) v = torch.randn(2, 8, 512, 64, device='cuda', dtype=torch.float16) Automatically uses Flash Attention if available out = F.scaleddotproductattention(q, k, v) flash-attn library (more features): bash pip install flash-attn --no-build-isolation python from flashattn import flashattnfunc q, k, v: [batch, seqlen, nheads, headdim] out = flashattnfunc(q, k, v, dropoutp=0.0, causal=True) Common workflows Workflow 1: Enable in existing PyTorch model Copy this checklist: Flash Attention Integration: - [ ] Step 1: Check PyTorch version (≥2.2) - [ ] Step 2:
Explore related resources
Frequently asked questions
What is optimizing-attention-flash?
optimizing-attention-flash is a open-source AI agent skill with Copy skill directory. Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction.
Who is optimizing-attention-flash best for?
optimizing-attention-flash is best for reusing agent instructions, scripts, and references.
How do I install optimizing-attention-flash?
Install or run optimizing-attention-flash using Copy skill directory. Check optimizing-attention-flash for the latest setup command.
Is optimizing-attention-flash actively maintained?
optimizing-attention-flash may need a closer maintenance check before production use.
Auto-fetched from GitHub.
