Create Eval Set Skill
Create a test set CSV file for import into Copilot Studio's in-product Evaluate tab. Reads the agent's topics, instructions, and knowledge sources to generate meaningful test cases with appropriate graders (General quality, Compare meaning, Exact match, etc.). Use when the user asks to create, prepare, or generate eval Published by microsoft in skills-for-copilot-studio.
What is Create Eval Set Skill?
Create a test set CSV file for import into Copilot Studio's in-product Evaluate tab. Reads the agent's topics, instructions, and knowledge sources to generate meaningful test cases with appropriate graders (General quality, Compare meaning, Exact match, etc.). Use when the user asks to create, prepare, or generate eval Published by microsoft in skills-for-copilot-studio. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Testing
- Design and media
- Testing use cases
- Design and media use cases
Technical details
- Install or run with Copy skill directory
When to use Create Eval Set Skill
- Use it for testing.
- Use it for design and media.
Built with
Editorial notes
Source
- Creator: microsoft
- Repository: microsoft/skills-for-copilot-studio
- Skill file: skills/create-eval-set/SKILL.md
What it does
Create a test set CSV file for import into Copilot Studio's in-product Evaluate tab. Reads the agent's topics, instructions, and knowledge sources to generate meaningful test cases with appropriate graders (General quality, Compare meaning, Exact match, etc.). Use when the user asks to create, prepare, or generate eval
Skill instructions
Create Evaluation Test Set Create a test set CSV file that can be imported into Copilot Studio's Evaluate tab for in-product agent evaluation. Phase 1: Understand the Agent Read the agent's YAML files to understand what it does: 1. Glob: /agent.mcs.yml — find the agent 2. Read agent.mcs.yml — get the agent's instructions, description, and capabilities 3. Read settings.mcs.yml — check orchestration mode (generative vs classic) 4. Glob: /topics/.mcs.yml — list all topics 5. Read key topics (especially non-system ones) — understand trigger phrases, conversation flows, expected behaviors 6. Check for knowledge sources, actions, and connected tools Phase 2: Design Test Cases Create test cases that cover: | Category | What to test | Example | |----------|-------------|---------| | Core functionality | Main topics and capabilities | Questions matching trigger phrases | | Knowledge/generative | Knowledge source responses | Questions the agent should answer from its knowledge | | System topics
Explore related resources
Frequently asked questions
What is Create Eval Set?
Create Eval Set is a open-source AI agent skill with Copy skill directory. Create a test set CSV file for import into Copilot Studio's in-product Evaluate tab.
Who is Create Eval Set best for?
Create Eval Set is best for reusing agent instructions, scripts, and references, testing workflows, design and media workflows.
How do I install Create Eval Set?
Install or run Create Eval Set using Copy skill directory. Check Create Eval Set for the latest setup command.
Is Create Eval Set actively maintained?
Create Eval Set may need a closer maintenance check before production use.
Auto-fetched from GitHub.
