Mellanox ConnectX-6 VPI 100Gigabit Ethernet Card User Manual

1. Introduction

This manual provides essential instructions for the installation, operation, and maintenance of your Mellanox ConnectX-6 VPI 100Gigabit Ethernet Card. The ConnectX-6 VPI is a high-performance network adapter designed for demanding data center applications, offering dual-port 200Gb/s InfiniBand and Ethernet connectivity. It supports various speeds including HDR, HDR100, EDR, FDR, QDR, DDR, and SDR InfiniBand, as well as 200, 100, 50, 40, 25, and 10Gb/s Ethernet.

Please read this manual thoroughly before installing and operating the device to ensure proper functionality and to prevent damage.

2. Package Contents

Verify that all items are present and in good condition upon opening the package. If any items are missing or damaged, contact your vendor immediately.

• Mellanox ConnectX-6 VPI 100Gigabit Ethernet Card (Model: MCX653106A-ECAT-SP)
• Tall Bracket (pre-installed)

3. Setup and Installation

This section outlines the steps for physically installing the Mellanox ConnectX-6 VPI adapter card into your system.

3.1 System Requirements

• Available PCIe 3.0 or PCIe 4.0 x16 expansion slot.
• Compatible operating system (e.g., Linux, Windows Server).
• Appropriate drivers (available from Mellanox/NVIDIA website).

3.2 Installation Steps

1. Power Off: Ensure your computer system is completely powered off and disconnected from the power source.
2. Open Chassis: Open the computer chassis to access the motherboard and expansion slots.
3. Locate PCIe Slot: Identify an available PCIe 3.0 or 4.0 x16 slot.
4. Remove Slot Cover: Remove the metal slot cover corresponding to the chosen PCIe slot.
5. Insert Adapter Card: Carefully align the Mellanox ConnectX-6 VPI card with the PCIe slot and press firmly until it is securely seated. Ensure the bracket is aligned with the chassis opening.
6. Secure Card: Secure the adapter card with the screw or retention clip provided by your chassis.
7. Close Chassis: Close the computer chassis and reconnect the power source.
8. Connect Cables: Connect appropriate QSFP56 cables to the dual ports on the adapter card.

Figure 1: Mellanox ConnectX-6 VPI 100Gigabit Ethernet Card. This image shows the green circuit board with the Mellanox ConnectX-6 chip, two QSFP56 ports on the left, and the PCIe connector at the bottom.

4. Operating Instructions

After physical installation, the adapter card requires software configuration to become fully operational.

4.1 Driver Installation

1. Boot System: Power on your computer system.
2. Download Drivers: Visit the official Mellanox/NVIDIA website to download the latest drivers for your ConnectX-6 VPI card and operating system.
3. Install Drivers: Follow the provided instructions to install the drivers. This typically involves running an installer or compiling modules for Linux.
4. Reboot (if required): Reboot your system if prompted by the driver installation process.

4.2 Network Configuration

Once drivers are installed, configure the network interfaces through your operating system's network settings. The ConnectX-6 VPI supports both InfiniBand and Ethernet protocols, allowing flexible deployment.

• IP Address Assignment: Assign static or dynamic IP addresses to the Ethernet interfaces.
• InfiniBand Configuration: Configure InfiniBand subnets and partitions as required for your high-performance computing environment.
• Verify Connectivity: Use standard network tools (e.g., ping, ip a, ibstat) to verify network connectivity and status.

5. Maintenance

Proper maintenance ensures the longevity and optimal performance of your Mellanox ConnectX-6 VPI card.

• Keep Clean: Periodically inspect the card for dust accumulation. Use compressed air to gently clean the card and surrounding components. Ensure the system is powered off before cleaning.
• Airflow: Ensure adequate airflow within the computer chassis to prevent overheating.
• Driver Updates: Regularly check the Mellanox/NVIDIA website for updated drivers and firmware. Keeping drivers current can improve performance, stability, and security.
• Cable Integrity: Ensure all connected cables (QSFP56) are securely seated and free from damage.

6. Troubleshooting

This section provides solutions to common issues you might encounter.

6.1 Common Issues and Solutions

• Card Not Detected:
  • Ensure the card is fully seated in the PCIe slot.
  • Verify the PCIe slot is functional by testing with another known-good card if possible.
  • Check BIOS/UEFI settings to ensure PCIe slots are enabled.
• No Network Connectivity:
  • Confirm that the correct drivers are installed and loaded.
  • Check cable connections at both the adapter card and the network switch/router.
  • Verify network configuration (IP addresses, subnet masks, gateways).
  • Check network switch/router status and port configuration.
• Slow Network Speeds:
  • Ensure you are using appropriate QSFP56 cables rated for 100Gb/s or 200Gb/s.
  • Verify that the network switch/router ports are configured for the correct speed and duplex settings.
  • Update drivers and firmware to the latest versions.
  • Check system resource utilization (CPU, memory) for bottlenecks.

For further assistance, refer to the official Mellanox/NVIDIA support documentation or contact their technical support.

7. Specifications

8. Warranty and Support

For detailed warranty information, please refer to the documentation provided with your purchase or visit the official Mellanox/NVIDIA website. Technical support and additional resources, including updated drivers and firmware, can be found on the manufacturer's support portal.

Mellanox/NVIDIA Support Website: https://www.nvidia.com/en-us/networking/