NVIDIA T4 Tensor Core GPU
Accelerating AI Training and Inference
Introduction
The NVIDIA T4 GPU is a powerful server accelerator designed to provide scalable performance for AI training and inference. Its 70-watt, low-profile design is optimized for server deployment, offering revolutionary multi-precision inference performance to accelerate a wide range of popular applications.
This advanced GPU is built into a compact, 70-watt, low-power PCIe form factor, optimized for server scalability, and built to deliver outstanding AI performance.
Performance Benchmarks
Inference Performance:
A comparison of one NVIDIA T4 GPU against a server with dual Xeon Gold 6140 CPUs shows significant performance gains:
- GNMT: 36X
- ResNet-50: 27X
- DeepSpeech2: 21X
Training Performance:
A comparison of two NVIDIA T4 GPUs against a server with dual Xeon Gold 6140 CPUs shows significant performance gains:
- ResNet-50 (FP16/FP32): 9.3X
Specifications
| Specification | Details |
|---|---|
| GPU Architecture | NVIDIA Turing Tensor Cores |
| Core Count | 320 |
| NVIDIA CUDA® Cores | 2560 |
| Single Precision (FP32) | 8.1 TFLOPS |
| Mixed Precision (FP16/FP32) | 65 TFLOPS |
| INT8 | 130 TOPS |
| INT4 | 260 TOPS |
| GPU Memory | 16 GB GDDR6 |
| Memory Bandwidth | 300 GB/s |
| ECC Support | Yes |
| System Interface | x16 PCIe Gen3 |
| Form Factor | PCIe Low Profile |
| Thermal Solution | Passive |
| Compute APIs | CUDA, NVIDIA TensorRT™™, ONNX |
Key Features
- Compact 70-Watt Design: The low-profile, 70-watt form factor optimizes T4 for scalable servers, offering 50x greater power efficiency compared to CPUs and significantly reducing operational costs.
- Revolutionary Multi-Precision Performance: The Turing Tensor Core technology delivers breakthrough AI performance across FP32, FP16, INT8, and INT4 precisions.
- Versatile Acceleration: The NVIDIA T4 GPU is ideal for accelerating deep learning, machine learning training and inference, video transcoding, and virtual desktops.
- Broad Framework Support: T4 supports all AI frameworks and network types, providing robust performance and efficiency for large-scale deployments.
Learn More
For more details on the NVIDIA T4, visit www.nvidia.cn/T4.
