inspect-eval-execution Skill
Guide for running SABER inspectai evaluations locally. Use this when asked to run, re-run, or configure an inspect eval for any SABER domain. Published by microsoft in ACESEvals.
What is inspect-eval-execution Skill?
Guide for running SABER inspectai evaluations locally. Use this when asked to run, re-run, or configure an inspect eval for any SABER domain. Published by microsoft in ACESEvals. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Deployment
- Design and media
- Deployment use cases
- Design and media use cases
Technical details
- Install or run with Copy skill directory
When to use inspect-eval-execution Skill
- Use it for deployment.
- Use it for design and media.
Built with
Editorial notes
Source
- Creator: microsoft
- Repository: microsoft/ACESEvals
- Skill file: .github/skills/inspect-eval-execution/SKILL.md
What it does
Guide for running SABER inspectai evaluations locally. Use this when asked to run, re-run, or configure an inspect eval for any SABER domain.
Skill instructions
To run SABER inspectai evaluations locally, follow this process: 1. Understand the Domain Structure Each domain lives in domains/<domain/ and follows this layout: domains/<domain/ <domain.py @task entry point — calls createtask() eval.yaml Domain metadata, Docker image definitions tasks/ global.yaml Default prompts, tools, maxsteps, aggregation <taskgroup/ <task.yaml Individual task definitions compose/ sandbox.compose.yml Docker Compose for sandbox containers scoring/ Domain-specific scoring strategies tools/ Domain-specific MCP tools prompts/ Jinja2 prompt templates (instructions/, assistants/, etc.) docker/ Dockerfiles for sandbox images The <domain.py file is minimal — it just calls saber.task.createtask(kwargs) which wires up config loading, prompt rendering, scoring, tools, and the agent. 2. Build Docker Images First Before running evals, ensure Docker images are built: bash Build all images for a domain uv run saber build <domain Or auto-build during eval (slower, builds missing
Explore related resources
Frequently asked questions
What is inspect-eval-execution?
inspect-eval-execution is a open-source AI agent skill with Copy skill directory. Guide for running SABER inspectai evaluations locally. Use this when asked to run, re-run, or configure an inspect eval for any SABER domain.
Who is inspect-eval-execution best for?
inspect-eval-execution is best for reusing agent instructions, scripts, and references, deployment workflows, design and media workflows.
How do I install inspect-eval-execution?
Install or run inspect-eval-execution using Copy skill directory. Check inspect-eval-execution for the latest setup command.
Is inspect-eval-execution actively maintained?
inspect-eval-execution may need a closer maintenance check before production use.
Auto-fetched from GitHub.
