serving-llms-vllm Skill
vLLM: high-throughput LLM serving, OpenAI API, quantization. Published by NousResearch in hermes-agent.
What is serving-llms-vllm Skill?
vLLM: high-throughput LLM serving, OpenAI API, quantization. Published by NousResearch in hermes-agent. This profile combines repository metadata with install, compatibility, and usage signals so developers can quickly decide whether it fits their agent workflow before opening the source repository.
Automated repository signals based on public metadata such as recency, license, installation evidence, and adoption. These are not a security audit or endorsement.
Key capabilities
- Includes SKILL.md support
- Reusable instructions support
- Developers using hermes-agent
Technical details
- Install or run with Copy skill directory
When to use serving-llms-vllm Skill
- Use it for developers using hermes-agent.
Built with
Editorial notes
Source
- Creator: NousResearch
- Repository: NousResearch/hermes-agent
- Skill file: skills/mlops/inference/vllm/SKILL.md
What it does
vLLM: high-throughput LLM serving, OpenAI API, quantization.
Skill instructions
vLLM - High-Performance LLM Serving When to use Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism. Quick start vLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache) and continuous batching (mixing prefill/decode requests). Installation: bash pip install vllm Basic offline inference: python from vllm import LLM, SamplingParams llm = LLM(model="meta-llama/Llama-3-8B-Instruct") sampling = SamplingParams(temperature=0.7, maxtokens=256) outputs = llm.generate(["Explain quantum computing"], sampling) print(outputs[0].outputs[0].text) OpenAI-compatible server: bash vllm serve meta-llama/Llama-3-8B-Instruct Query with OpenAI SDK python -c " from openai import OpenAI client = OpenAI(baseurl='http://localhost:8000/v1', apikey='EMPTY') print(client.chat.completions.create( model='m
Explore related resources
Frequently asked questions
What is serving-llms-vllm?
serving-llms-vllm is a open-source AI agent skill with Copy skill directory. vLLM: high-throughput LLM serving, OpenAI API, quantization.
Who is serving-llms-vllm best for?
serving-llms-vllm is best for reusing agent instructions, scripts, and references.
How do I install serving-llms-vllm?
Install or run serving-llms-vllm using Copy skill directory. Check serving-llms-vllm for the latest setup command.
Is serving-llms-vllm actively maintained?
serving-llms-vllm may need a closer maintenance check before production use.
Auto-fetched from GitHub.
