vally-eval Skill
Author, validate, and run Vally evaluation suites for agent skills. TRIGGERS: create eval, write eval, add eval, run eval, validate eval, vally eval, eval.yaml, add stimulus, map test to eval, migrate test to eval, eval graders, eval scoring, add eval to CI. Published by microsoft in GitHub-Copilot-for-Azure.
Decision snapshot
Is this a fit?
Testing, Documentation, Writing, Includes SKILL.md
Compatibility not yet detected.
Permission behavior not yet detected.
Copy skill directory
2 months ago · NOASSERTION license
No specific cautions were detected. Review the source and requested permissions before installing.
What is vally-eval Skill?
Author, validate, and run Vally evaluation suites for agent skills. TRIGGERS: create eval, write eval, add eval, run eval, validate eval, vally eval, eval.yaml, add stimulus, map test to eval, migrate test to eval, eval graders, eval scoring, add eval to CI. Published by microsoft in GitHub-Copilot-for-Azure. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement. See how SkillIndex evaluates profiles.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Testing
- Documentation
- Writing
- Testing use cases
- Documentation use cases
Declared skill metadata
- Declared author: Microsoft
- Declared license: MIT
- Source file: .github/skills/vally-eval/SKILL.md
These fields retain source and confidence evidence from the indexed SKILL.md.
Compatibility and setup
- Install or run with Copy skill directory
When to use vally-eval Skill
- Use it for testing.
- Use it for documentation.
- Use it for writing.
Built with
Editorial notes
Source
- Creator: microsoft
- Repository: microsoft/GitHub-Copilot-for-Azure
- Skill file: .github/skills/vally-eval/SKILL.md
What it does
Author, validate, and run Vally evaluation suites for agent skills. TRIGGERS: create eval, write eval, add eval, run eval, validate eval, vally eval, eval.yaml, add stimulus, map test to eval, migrate test to eval, eval graders, eval scoring, add eval to CI.
Skill instructions
Vally eval suites Skills in the azure-skills plugin are required to have integration tests that run prompts against an LLM agent to evaluate whether they help the agent accomplish goals in target scenarios. Such integration tests are written as vally eval suites, using vally as the underlying tool for running tests and grading the agent outcome. Write vally eval suites Vally eval suites are written as yaml documents. All eval suites share eval spec. Refer to the official documentation on the schema of the spec and the schema of the eval suites writing-eval-specs. Vally eval suites for azure-skills plugin have the following file layout. The shared eval spec is located at <repo-root/.vally.yaml. The eval suites are categorized by plugin and skills. The eval suites for each skill are located at <repo-root/evals/<plugin-dirname/<skill-name/.yaml, e.g. <repo-root/evals/azure-skills/azure-ai/eval.yaml. Use meaningful file names to categorize tests. If a skill needs fixture files for its eval
Verified compatibility and discovery
Frequently asked questions
What is vally-eval?
vally-eval is a open-source AI agent skill with Copy skill directory. Author, validate, and run Vally evaluation suites for agent skills. TRIGGERS: create eval, write eval, add eval, run eval, validate eval, vally eval, eval.yaml, add stimulus, map test to eval, migrate test to.
Who is vally-eval best for?
vally-eval is best for reusing agent instructions, scripts, and references, testing workflows, documentation workflows, writing workflows.
How do I install vally-eval?
Install or run vally-eval using Copy skill directory. Check vally-eval for the latest setup command.
Is vally-eval actively maintained?
vally-eval may need a closer maintenance check before production use.
Project health auto-fetched from the source repository.
Maintain this resource?
Review this source-backed profile, send a correction with evidence, or link to it from your documentation. Claims verify your relationship to the project; profile facts still require source evidence and editorial review.
