In the context of InfiniBand networking, DAC (Direct Attach Copper) cables are used to provide high-speed data connections with different port speeds for various applications and data centre environments. Common port speed standards in modern computing infrastructure include FDR, EDR, HDR and NDR:
FDR (Fourteen Data Rate)
- Data Rate: 14 Gbps per lane, or 56 Gbps for a 4-lane connection.
- Use Case: Often used in high performance computing (HPC) systems and data centres from around 2011 onward, FDR has been widely adopted due to its reliability and cost-effectiveness for clusters needing strong performance.
- Compatibility: FDR systems can be backward compatible with previous generations like Quad Data Rate (QDR) and Double Data Rate (DDR) but will operate at the lower data rate when connected to slower hardware.
EDR (Enhanced Data Rate)
- Data Rate: 25 Gbps per lane, or 100 Gbps for a 4-lane connection.
- Use Case: EDR is suitable for demanding HPC applications and was introduced around 2014-2015 to handle increased workloads with minimal latency.
- Compatibility: EDR devices can be backward compatible with FDR and QDR speeds, allowing for flexibility within mixed-speed environments.
HDR (High Data Rate)
- Data Rate: 50 Gbps per lane, or 200 Gbps for a 4-lane connection.
- Use Case: HDR is intended for modern data centres and advanced HPC applications requiring very high throughput. Introduced around 2017, it enables extremely low latency and high speed connections suitable for data intensive tasks.
- Compatibility: HDR is backward compatible with EDR, FDR, and QDR, which allows for mixed environments and staged upgrades within a data centre or HPC environment.
NDR (Next Data Rate)
- Data Rate: 100 Gbps per lane, or 400 Gbps for a 4-lane connection.
- Use Case: NDR is the latest advancement in InfiniBand data rate standards, aiming to meet the needs of next-generation supercomputing and AI training workloads requiring huge bandwidth and minimal latency. Released in 2022, it’s suited for top-tier HPC clusters and AI workloads.
- Compatibility: NDR infrastructure is backward compatible with previous standards (HDR, EDR, FDR), making it easier to transition networks to the latest standard.
|
Port Speed Standard |
Data Rate per Lane (Gbps) |
4-Lane Data Rate (Gbps) |
|
FDR |
14 |
56 |
|
EDR |
25 |
100 |
|
HDR |
50 |
200 |
|
NDR |
100 |
400 |
The choice of InfiniBand speed depends on the specific needs of the infrastructure:
- FDR and EDR are well-suited for general HPC clusters and data centres that balance cost with performance.
- HDR and NDR are ideal for environments where maximum performance and minimum latency are crucial, such as AI and machine learning clusters.
See our Tech Talk post for more information on InfiniBand technology and its relevance in the future of high end computing.
