mcs-eval Skill
Use this skill to test an agent's quality after building or fixing it. Runs eval test sets against a published agent via Direct Line API (default, auto), CopilotStudio SDK, Power CAT Kit, or MCS Native Eval (for MCP agents). Results are written per-test to evalSets[].tests[].lastResult in agentspec.json. Use after /mcs Published by microsoft in MCS-Agent-Builder.
What is mcs-eval Skill?
Use this skill to test an agent's quality after building or fixing it. Runs eval test sets against a published agent via Direct Line API (default, auto), CopilotStudio SDK, Power CAT Kit, or MCS Native Eval (for MCP agents). Results are written per-test to evalSets[].tests[].lastResult in agentspec.json. Use after /mcs Published by microsoft in MCS-Agent-Builder. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Testing
- Writing
- Testing use cases
- Writing use cases
Technical details
- Install or run with Copy skill directory
When to use mcs-eval Skill
- Use it for testing.
- Use it for writing.
Built with
Editorial notes
Source
- Creator: microsoft
- Repository: microsoft/MCS-Agent-Builder
- Skill file: .claude/skills/mcs-eval/SKILL.md
What it does
Use this skill to test an agent's quality after building or fixing it. Runs eval test sets against a published agent via Direct Line API (default, auto), CopilotStudio SDK, Power CAT Kit, or MCS Native Eval (for MCP agents). Results are written per-test to evalSets[].tests[].lastResult in agentspec.json. Use after /mcs
Skill instructions
MCS Evaluation Runner Run evaluation tests for an agent and write results back to agentspec.json so the dashboard can display them. Eval Runners — Default is Direct Line | Runner | When | Speed | |--------|------|-------| | Direct Line (default) | Agent has no user-delegated MCP tools | Fast (~2s/test) | | Direct Line + Interactive | Agent requires OAuth sign-in | ~30s/test + user input | | CopilotStudio SDK | User specifies --sdk | ~3s/test | | Power CAT Kit | User specifies --powercat | Slow (server-side) | | MCS Native Eval | Agent uses MCP/user-delegated tools, or --native | User-driven | Auto-detection: MCP/user-delegated tools → MCS Native Eval. Otherwise → Direct Line. See reference/alternative-runners.md for SDK, Power CAT Kit, and Native Eval details. Input /mcs-eval {projectId} {agentId} Run all eval sets /mcs-eval {projectId} {agentId} --set boundaries,quality Run specific sets /mcs-eval {projectId} {agentId} --native Force MCS Native Eval /mcs-eval {projectId} {agentId} --s
Explore related resources
Frequently asked questions
What is mcs-eval?
mcs-eval is a open-source AI agent skill with Copy skill directory. Use this skill to test an agent's quality after building or fixing it. Runs eval test sets against a published agent via Direct Line API (default, auto), CopilotStudio SDK, Power CAT Kit, or MCS Native Eval.
Who is mcs-eval best for?
mcs-eval is best for reusing agent instructions, scripts, and references, testing workflows, writing workflows.
How do I install mcs-eval?
Install or run mcs-eval using Copy skill directory. Check mcs-eval for the latest setup command.
Is mcs-eval actively maintained?
mcs-eval may need a closer maintenance check before production use.
