Run Eval Skill
Run evaluations against a Copilot Studio agent via the Power Platform Evaluation API. Works on DRAFT agents — no publish step required. Lists test sets, starts a run, polls until complete, fetches results, and proposes YAML fixes for failures. Use when the user wants to test agent changes without publishing. Published by microsoft in skills-for-copilot-studio.
Decision snapshot
Is this a fit?
Testing, Includes SKILL.md, Reusable instructions
Compatibility not yet detected.
Can write or update data, Write actions, Filesystem access, Shell access
Copy skill directory
2 months ago · MIT license
Confirm write permissions and scope before use. Access note: Local files. Access note: Shell access.
What is Run Eval Skill?
Run evaluations against a Copilot Studio agent via the Power Platform Evaluation API. Works on DRAFT agents — no publish step required. Lists test sets, starts a run, polls until complete, fetches results, and proposes YAML fixes for failures. Use when the user wants to test agent changes without publishing. Published by microsoft in skills-for-copilot-studio. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement. See how SkillIndex evaluates profiles.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Testing
- Testing use cases
Declared skill metadata
- Source file: skills/run-eval/SKILL.md
Allowed tools declared by source
These fields retain source and confidence evidence from the indexed SKILL.md.
Compatibility and setup
- Install or run with Copy skill directory
- Local files
- Shell access
Requirements and access
Security and permissions
Review permissions before connecting any MCP server to an agent. Pay special attention to whether it can read local files, write data, call external services, or perform destructive actions.
When to use Run Eval Skill
- Use it for testing.
Built with
Editorial notes
Source
- Creator: microsoft
- Repository: microsoft/skills-for-copilot-studio
- Skill file: skills/run-eval/SKILL.md
What it does
Run evaluations against a Copilot Studio agent via the Power Platform Evaluation API. Works on DRAFT agents — no publish step required. Lists test sets, starts a run, polls until complete, fetches results, and proposes YAML fixes for failures. Use when the user wants to test agent changes without publishing.
Skill instructions
Run Evaluation (PPAPI) Run evaluations against a Copilot Studio agent's draft — no publish needed. The caller (test agent) must provide --client-id and --workspace. If you don't have the client ID, return immediately and tell the caller to run test-auth first. All eval-api commands run in the foreground. NEVER use runinbackground. Step 1: List test sets and let the user choose bash node ${CLAUDESKILLDIR}/../../scripts/eval-api.bundle.js list-testsets --workspace <path --client-id <id - No test sets found: Tell the user to create one in Copilot Studio (Evaluate tab New evaluation). Stop. - One test set: Tell the user which one you're using and proceed. - Multiple test sets: Show them all and ask the user to pick. Do not proceed until they answer. Step 2: Ask about authenticated execution — MANDATORY, do not skip You MUST ask this question and wait for the user's answer before starting the run. Ask the user: Does your agent use authenticated knowledge sources or connector actions (tools)
Verified compatibility and discovery
Frequently asked questions
What is Run Eval?
Run Eval is a open-source AI agent skill with Copy skill directory. Run evaluations against a Copilot Studio agent via the Power Platform Evaluation API. Works on DRAFT agents — no publish step required.
Who is Run Eval best for?
Run Eval is best for reusing agent instructions, scripts, and references, testing workflows.
How do I install Run Eval?
Install or run Run Eval using Copy skill directory. Check Run Eval for the latest setup command.
Is Run Eval actively maintained?
Run Eval may need a closer maintenance check before production use.
Project health auto-fetched from the source repository.
Maintain this resource?
Review this source-backed profile, send a correction with evidence, or link to it from your documentation. Claims verify your relationship to the project; profile facts still require source evidence and editorial review.
