eval-monitoring Skill
Guide for monitoring running SABER evaluations, checking progress, and managing eval batches. Use this when asked to monitor evals, check progress, produce a status report, or manage concurrent eval runs. Also covers Docker health and resource management. Published by microsoft in ACESEvals.
What is eval-monitoring Skill?
Guide for monitoring running SABER evaluations, checking progress, and managing eval batches. Use this when asked to monitor evals, check progress, produce a status report, or manage concurrent eval runs. Also covers Docker health and resource management. Published by microsoft in ACESEvals. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Deployment
- Design and media
- Deployment use cases
- Design and media use cases
Technical details
- Install or run with Copy skill directory
When to use eval-monitoring Skill
- Use it for deployment.
- Use it for design and media.
Built with
Editorial notes
Source
- Creator: microsoft
- Repository: microsoft/ACESEvals
- Skill file: .github/skills/eval-monitoring/SKILL.md
What it does
Guide for monitoring running SABER evaluations, checking progress, and managing eval batches. Use this when asked to monitor evals, check progress, produce a status report, or manage concurrent eval runs. Also covers Docker health and resource management.
Skill instructions
Eval Monitoring Skill Quick Status Check Run the monitoring script for an instant report: bash ./scripts/monitorevals.sh For continuous monitoring (every 15 minutes): bash ./scripts/monitorevals.sh --watch 15 Manual Monitoring Commands 1. Check Running Processes bash Count running evals ps aux | grep "inspect eval" | grep -v grep | wc -l See model and agent for each ps aux | grep "inspect eval" | grep -v grep | \ grep -oP '(?<=--model )"?[^\s"]+' | tr -d '"' 2. Check Eval File Progress Staleness note: .eval ZIP files are flushed in batches by inspectai, so sample counts read from them lag 2–3 minutes behind the live results visible in the inspect view UI. This is expected — treat .eval reads as "near-real-time" rather than exact. python Quick sample count for an in-progress eval python3 -c " import zipfile, json with zipfile.ZipFile('path/to/file.eval') as z: samples = [n for n in z.namelist() if n.startswith('samples/') and n.endswith('.json')] header = json.loads(z.read('header.json'
Explore related resources
Frequently asked questions
What is eval-monitoring?
eval-monitoring is a open-source AI agent skill with Copy skill directory. Guide for monitoring running SABER evaluations, checking progress, and managing eval batches.
Who is eval-monitoring best for?
eval-monitoring is best for reusing agent instructions, scripts, and references, deployment workflows, design and media workflows.
How do I install eval-monitoring?
Install or run eval-monitoring using Copy skill directory. Check eval-monitoring for the latest setup command.
Is eval-monitoring actively maintained?
eval-monitoring may need a closer maintenance check before production use.
Auto-fetched from GitHub.
