agent-evaluation Skill
Help a learner evaluate and observe an AI agent on Microsoft Foundry — quality/safety evaluators, tracing, and reading results. Use for Lesson 3 tasks: measuring agent performance, adding observability, or interpreting evaluation output. Published by microsoft in Building-AI-Agents-From-Zero-To-Production.
What is agent-evaluation Skill?
Help a learner evaluate and observe an AI agent on Microsoft Foundry — quality/safety evaluators, tracing, and reading results. Use for Lesson 3 tasks: measuring agent performance, adding observability, or interpreting evaluation output. Published by microsoft in Building-AI-Agents-From-Zero-To-Production. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Testing
- Testing use cases
Technical details
- Install or run with Copy skill directory
When to use agent-evaluation Skill
- Use it for testing.
Built with
Editorial notes
Source
- Creator: microsoft
- Repository: microsoft/Building-AI-Agents-From-Zero-To-Production
- Skill file: .github/skills/agent-evaluation/SKILL.md
What it does
Help a learner evaluate and observe an AI agent on Microsoft Foundry — quality/safety evaluators, tracing, and reading results. Use for Lesson 3 tasks: measuring agent performance, adding observability, or interpreting evaluation output.
Skill instructions
Agent evaluation & observability Guidance for measuring and improving agents (Lesson 3). Guardrails (always apply) - Platform: Microsoft Foundry; framework: Microsoft Agent Framework. - Use the evaluation/observability APIs from the current agent-framework surface (Foundry evals + tracing). Model for any judge/eval step: gpt-5.1 (never gpt-4o / gpt-4.1). - Endpoints/models from .env; az login for auth. Concepts a learner should grasp 1. Why evaluate: agents are non-deterministic — you need repeatable metrics (groundedness, relevance, coherence, safety) to know whether a change helped or hurt. 2. Observability vs evaluation: tracing captures what happened on each run (spans, tool calls, tokens); evaluation scores those runs against criteria. 3. Close the loop: trace → evaluate → adjust instructions/tools → re-evaluate. How to help - Point the learner at lesson-3-agent-evals/ and its README for the exact evaluator setup. - Encourage a small, fixed test set of prompts so results are compa
Explore related resources
Frequently asked questions
What is agent-evaluation?
agent-evaluation is a open-source AI agent skill with Copy skill directory. Help a learner evaluate and observe an AI agent on Microsoft Foundry — quality/safety evaluators, tracing, and reading results.
Who is agent-evaluation best for?
agent-evaluation is best for reusing agent instructions, scripts, and references, testing workflows.
How do I install agent-evaluation?
Install or run agent-evaluation using Copy skill directory. Check agent-evaluation for the latest setup command.
Is agent-evaluation actively maintained?
agent-evaluation may need a closer maintenance check before production use.
Auto-fetched from GitHub.
