waza-runner Skill

Source

• Creator: microsoft
• Repository: microsoft/waza
• Skill file: waza-runner/SKILL.md

What it does

Run evaluations on Agent Skills to measure their effectiveness. USE FOR: "run skill evals", "evaluate my skill", "test skill quality", "check skill triggers", "skill compliance check", "measure skill performance", "run evals on [skill-name]", "grade skill execution". DO NOT USE FOR: writing skills (use skill-authoring)

Skill instructions

Skill Eval Runner Evaluate Agent Skills like you evaluate AI Agents This skill runs evaluations on other skills to measure their effectiveness using the same patterns that power AI agent evaluations. When to Use - Running quality evaluations on a skill - Testing if a skill triggers on correct prompts - Measuring skill behavior quality - Generating eval reports for CI/CD Commands Run Evals Run evals on <skill-name Initialize Eval Suite Create evals for <skill-name Generate Report Generate eval report for <skill-name Workflow 1. Check for Eval Suite: Look for eval.yaml in the skill directory 2. Load Tasks: Parse task definitions from tasks/.yaml 3. Execute: Run each task through the configured graders 4. Report: Output results in JSON or Markdown format Metrics Measured | Metric | Description | Default Threshold | |--------|-------------|-------------------| | Task Completion | Did the skill accomplish the goal? | 80% | | Trigger Accuracy | Was skill invoked on correct prompts? | 90% | |