inspect-eval-log-analysis Skill

Source

• Creator: microsoft
• Repository: microsoft/ACESEvals
• Skill file: .github/skills/inspect-eval-log-analysis/SKILL.md

What it does

Guide for parsing and analyzing inspectai .eval log files. Use this when asked to interpret eval results, extract tool calls, find scores, or investigate agent behavior from .eval logs.

Skill instructions

To parse and analyze inspectai .eval log files, follow this process: 1. Understanding the .eval File Format .eval files are ZIP archives (not plain text). They contain structured JSON: <hash.eval (ZIP) ├── header.json Eval metadata ├── journal/ │ ├── start.json Eval config, plan, model info │ └── summaries/ │ └── 1.json Per-epoch summary ├── samples/ │ └── <taskidepoch1.json Full sample data per task ├── summaries.json Aggregated score summaries └── reductions.json Score reduction results Never try to read .eval files as plain text. Always use zipfile: python import json, zipfile with zipfile.ZipFile("logs/<file.eval", "r") as zf: print(zf.namelist()) See all files inside 2. Extract Sample Data (Messages, Tool Calls, Scores) The richest data is in the samples/ JSON files. Each sample contains the full conversation: python import json, zipfile with zipfile.ZipFile("logs/<file.eval", "r") as zf: for name in zf.namelist(): if not name.startswith("samples/"): continue sample = json.loads(z