Overview
Description
TensorRT-LLM helps you run large language models and visual generation models faster on NVIDIA GPUs, cutting inference costs while boosting throughput. It gives you a simple Python API to define and optimize models, plus ready-made runtimes for production deployment. The bottom line: get more tokens per second with less hardware.
Technical scorecard
License
Not specified
Commercial use
No
OpenAI-compatible API
No
REST API
No
Fine-tuning support
No
Quantization support
No
Docker available
Yes
GUI / no-code available
No
Telemetry
Opt-out
Offline after setup
No
Data & Privacy
Does it send data online?
This listing is not marked offline after setup.
Does it store history?
Not verified in this directory yet. Review the upstream docs for persistence, logs, and workspace storage.
License checks?
The listed license needs review before commercial use.
Telemetry?
Opt-out
Last verified: Jul 22, 2026. Maintainer verification should be treated as directory guidance, not legal advice.
Deploying TensorRT-LLM in production?
Stop silent telemetry leaks before they trigger an unintended data leak. Get our Airgap Certainty Blueprint for TensorRT-LLM.
Request Privacy and Telemetry AuditSetup & Installation
A developer can usually get this running with standard docs.
# Start with the official project repository
# https://github.com/NVIDIA/TensorRT-LLMHardware Requirements
Works Well With