Run Eval Skill
Run evaluations against a Copilot Studio agent via the Power Platform Evaluation API. Works on DRAFT agents — no publish step required. Lists test sets, starts a run, polls until complete, fetches results, and proposes YAML fixes for failures. Use when the user wants to test agent changes without publishing. Published by microsoft in skills-for-copilot-studio.
What is Run Eval Skill?
Run evaluations against a Copilot Studio agent via the Power Platform Evaluation API. Works on DRAFT agents — no publish step required. Lists test sets, starts a run, polls until complete, fetches results, and proposes YAML fixes for failures. Use when the user wants to test agent changes without publishing. Published by microsoft in skills-for-copilot-studio. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Testing
- Testing use cases
Technical details
- Install or run with Copy skill directory
When to use Run Eval Skill
- Use it for testing.
Built with
Editorial notes
Source
- Creator: microsoft
- Repository: microsoft/skills-for-copilot-studio
- Skill file: skills/run-eval/SKILL.md
What it does
Run evaluations against a Copilot Studio agent via the Power Platform Evaluation API. Works on DRAFT agents — no publish step required. Lists test sets, starts a run, polls until complete, fetches results, and proposes YAML fixes for failures. Use when the user wants to test agent changes without publishing.
Skill instructions
Run Evaluation (PPAPI) Run evaluations against a Copilot Studio agent's draft — no publish needed. The caller (test agent) must provide --client-id and --workspace. If you don't have the client ID, return immediately and tell the caller to run test-auth first. All eval-api commands run in the foreground. NEVER use runinbackground. Step 1: List test sets and let the user choose bash node ${CLAUDESKILLDIR}/../../scripts/eval-api.bundle.js list-testsets --workspace <path --client-id <id - No test sets found: Tell the user to create one in Copilot Studio (Evaluate tab New evaluation). Stop. - One test set: Tell the user which one you're using and proceed. - Multiple test sets: Show them all and ask the user to pick. Do not proceed until they answer. Step 2: Ask about authenticated execution — MANDATORY, do not skip You MUST ask this question and wait for the user's answer before starting the run. Ask the user: Does your agent use authenticated knowledge sources or connector actions (tools)
Explore related resources
Frequently asked questions
What is Run Eval?
Run Eval is a open-source AI agent skill with Copy skill directory. Run evaluations against a Copilot Studio agent via the Power Platform Evaluation API. Works on DRAFT agents — no publish step required.
Who is Run Eval best for?
Run Eval is best for reusing agent instructions, scripts, and references, testing workflows.
How do I install Run Eval?
Install or run Run Eval using Copy skill directory. Check Run Eval for the latest setup command.
Is Run Eval actively maintained?
Run Eval may need a closer maintenance check before production use.
Auto-fetched from GitHub.
