tensorrt-llm Skill
High-throughput LLM inference on NVIDIA GPUs. Published by NousResearch in hermes-agent.
Decision snapshot
Is this a fit?
Deployment, Includes SKILL.md, Reusable instructions
Compatibility not yet detected.
Permission behavior not yet detected.
Copy skill directory
29 days ago · MIT license
No specific cautions were detected. Review the source and requested permissions before installing.
What is tensorrt-llm Skill?
High-throughput LLM inference on NVIDIA GPUs. Published by NousResearch in hermes-agent. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement. See how SkillIndex evaluates profiles.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Deployment
- Deployment use cases
Declared skill metadata
- Declared author: Orchestra Research
- Declared license: MIT
- Source file: optional-skills/mlops/tensorrt-llm/SKILL.md
These fields retain source and confidence evidence from the indexed SKILL.md.
Compatibility and setup
- Install or run with Copy skill directory
When to use tensorrt-llm Skill
- Use it for deployment.
Built with
Editorial notes
Source
- Creator: NousResearch
- Repository: NousResearch/hermes-agent
- Skill file: optional-skills/mlops/tensorrt-llm/SKILL.md
What it does
High-throughput LLM inference on NVIDIA GPUs.
Skill instructions
TensorRT-LLM NVIDIA's open-source library for optimizing LLM inference with high performance on NVIDIA GPUs. When to use TensorRT-LLM Use TensorRT-LLM when: - Deploying on NVIDIA GPUs (A100, H100, GB200) - Need maximum throughput (24,000+ tokens/sec on Llama 3) - Require low latency for real-time applications - Working with quantized models (FP8, INT4, FP4) - Scaling across multiple GPUs or nodes Use vLLM instead when: - Need simpler setup and Python-first API - Want PagedAttention without TensorRT compilation - Working with AMD GPUs or non-NVIDIA hardware Use llama.cpp instead when: - Deploying on CPU or Apple Silicon - Need edge deployment without NVIDIA GPUs - Want simpler GGUF quantization format Quick start Installation bash Docker (recommended) — images are on NGC (nvcr.io), not Docker Hub. Replace x.y.z with the desired version (e.g. 1.2.1). Browse tags on NGC: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tensorrt-llm/containers/release/tags docker pull nvcr.io/nvidia/tensor
Verified compatibility and discovery
Frequently asked questions
What is tensorrt-llm?
tensorrt-llm is a open-source AI agent skill with Copy skill directory. High-throughput LLM inference on NVIDIA GPUs.
Who is tensorrt-llm best for?
tensorrt-llm is best for reusing agent instructions, scripts, and references, deployment workflows.
How do I install tensorrt-llm?
Install or run tensorrt-llm using Copy skill directory. Check tensorrt-llm for the latest setup command.
Is tensorrt-llm actively maintained?
tensorrt-llm may need a closer maintenance check before production use.
Project health auto-fetched from the source repository.
Maintain this resource?
Review this source-backed profile, send a correction with evidence, or link to it from your documentation. Claims verify your relationship to the project; profile facts still require source evidence and editorial review.
