huggingface-community-evals Skill
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .evalresults publication, or community-evals automation. Published by openai in plugins.
Decision snapshot
Is this a fit?
Data analysis, Includes SKILL.md, Reusable instructions
Compatibility not yet detected.
Permission behavior not yet detected.
Copy skill directory
19 days ago
No specific cautions were detected. Review the source and requested permissions before installing.
What is huggingface-community-evals Skill?
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .evalresults publication, or community-evals automation. Published by openai in plugins. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement. See how SkillIndex evaluates profiles.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Data analysis
- Data analysis use cases
Declared skill metadata
- Source file: plugins/hugging-face/skills/community-evals/SKILL.md
These fields retain source and confidence evidence from the indexed SKILL.md.
Compatibility and setup
- Install or run with Copy skill directory
When to use huggingface-community-evals Skill
- Use it for data analysis.
Built with
Editorial notes
Source
- Creator: openai
- Repository: openai/plugins
- Skill file: plugins/hugging-face/skills/community-evals/SKILL.md
What it does
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .evalresults publication, or community-evals automation.
Skill instructions
Overview This skill is for running evaluations against models on the Hugging Face Hub on local hardware. It covers: - inspect-ai with local inference - lighteval with local inference - choosing between vllm, Hugging Face Transformers, and accelerate - smoke tests, task selection, and backend fallback strategy It does not cover: - Hugging Face Jobs orchestration - model-card or model-index edits - README table extraction - Artificial Analysis imports - .evalresults generation or publishing - PR creation or community-evals automation If the user wants to run the same eval remotely on Hugging Face Jobs, hand off to the hugging-face-jobs skill and pass it one of the local scripts in this skill. If the user wants to publish results into the community evals workflow, stop after generating the evaluation run and hand off that publishing step to ~/code/community-evals. All paths below are relative to the directory containing this SKILL.md. When To Use Which Script | Use case | Script | |---|--
Verified compatibility and discovery
Frequently asked questions
What is huggingface-community-evals?
huggingface-community-evals is a open-source AI agent skill with Copy skill directory. Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware.
Who is huggingface-community-evals best for?
huggingface-community-evals is best for reusing agent instructions, scripts, and references, data analysis workflows.
How do I install huggingface-community-evals?
Install or run huggingface-community-evals using Copy skill directory. Check huggingface-community-evals for the latest setup command.
Is huggingface-community-evals actively maintained?
huggingface-community-evals may need a closer maintenance check before production use.
Project health auto-fetched from the source repository.
Maintain this resource?
Review this source-backed profile, send a correction with evidence, or link to it from your documentation. Claims verify your relationship to the project; profile facts still require source evidence and editorial review.
