The Key Advantages of InfiniBand

Wireless broadband technology, or IB, is a high-performance computing network communication standard known for its high throughput and low latency. With the development of network technology, the demand for bandwidth and low latency is increasing, and the demand for InfiniBand is also growing. This article will give you a detailed understanding of the advantages of the InfiniBand network.

What is InfiniBand

Before starting the article, we first need to briefly introduce what InfiniBand is. It is a high-speed, high-bandwidth, low-latency network communication protocol and interface standard. It was originally designed to connect computing clusters and storage systems, especially high-performance computing environments. InfiniBand is generally suitable for application scenarios that require large amounts of data transmission and low latency, such as AI model training, autonomous driving, etc.

Benefits of InfiniBand

Why is InfiniBand so popular in today's network? I believe you will have the answer in your mind after reading the following content.

InfiniBand Supports Simplified Network Management

InfiniBand was originally developed for SDN and is a network architecture managed by a subnet manager, which can configure and ensure the continuous operation of the local subnet. All channel adapters and switches must implement a subnet management agent (SMA) to communicate with the subnet manager. Each subnet requires at least one subnet manager to perform initialization and reconfiguration when the link is connected or disconnected. A subnet manager is selected as the primary manager through an arbitration process, and the other subnet managers are in the standby state. If the primary manager fails, the standby manager will take over to ensure the continuous operation of the network.

CPU Offload

CPU offload is an important means of accelerating computing. InfiniBand networks can use the least CPU resources for data transmission, as follows:

The hardware offloads the entire transport layer protocol stack.

Bypassing the kernel for zero-copy.

Write data directly from the memory of one server to the memory of another server through RDMA without using the CPU.

High Bandwidth

With the development of network technology, the bandwidth of InfiniBand is getting higher and higher. This is mainly because InfiniBand is used for server connections in high-performance computing, which makes its bandwidth requirements higher and more stringent. From 10Gbps since the development of InfiniBand technology to the current 800Gbps, InfiniBand bandwidth has made great progress.

QSFPTEK provides you with InfiniBand modules, DAC, and AOC at different rates. You can check the table below to learn more about our InfiniBand products:

Low latency

Generally, Ethernet switches use MAC table lookup addressing and store-and-forward, which requires a long processing time. InfiniBand switches only need to check the forwarding path based on the 16-bit LID and use Cut-Through technology to shorten the forwarding time to less than 100ns. At the network card level, RDMA technology can also be used to significantly speed up the latency of data packet encapsulation and decapsulation.

Scalability and flexibility

A major advantage of the InfiniBand network is that a single subnet can deploy up to 48,000 nodes, and its scalability and flexibility are self-evident. In addition, the IB network does not rely on ARP or other broadcast mechanisms, which can avoid broadcast storms or other bandwidth waste. Multiple InfiniBand subnets can also be interconnected through routers and switches. The following are the network topologies supported by InfiniBand:

InfiniBand Architecture Support QoS

QoS is a set of technologies and mechanisms for managing and optimizing network traffic. It is usually used to ensure the priority of key applications and services in the network and enhance their performance and reliability.

First, InfiniBand uses virtual channels to implement QoS. Virtual channels are independent logical communication links that share a physical link. Each physical link can support up to 15 standard virtual channels and one management channel.

Second, InfiniBand uses service level marking to mark the SL field in the packet header, dividing it from 0 to 15, and each level corresponds to a different priority.

InfiniBand also supports the differentiated service architecture defined by IETF RFC 2474 and 2475 to implement QoS functions similar to DiffServ.

Finally, InfiniBand can control the parameters of each communication flow by responding to path records or multi-path record queries. Administrators can optimize their paths to ensure that high-priority traffic is transmitted through the best path. QSFPTEK offers data center switches that support QoS functions.

Network Stability and Resilience

InfiniBand networks have a Self-Healing Networking mechanism based on InfiniBand switches. When a network failure occurs during long-term operation, this mechanism can reduce the link recovery time to 1 millisecond.

Load Balancing

InfiniBand supports multipath routing, which allows data packets to choose multiple paths in the network. This feature enables the network to select paths based on real-time traffic and adjust data transmission paths to achieve load balancing. Multipath routing can effectively avoid congestion on a single path and improve network efficiency and reliability.

SHARP Support

InfiniBand supports the SHARP protocol, which is a protocol for HPC systems. It implements data aggregation and reduction operations at the network layer, reduces data transmission volume and latency, and thus improves network efficiency and performance. It can optimize collective communications and avoid repeated data transmission between nodes. It plays an important role in big data computing and artificial intelligence.

Conclusion

In conclusion, the birth of InfiniBand technology has brought about a crucial change in data centers and high-performance computing applications. It not only provides high bandwidth and low latency but also has considerable scalability and flexibility. And with the development of technology, the bandwidth it can provide is also growing. Deploying an InfiniBand network can significantly improve performance and solve the worries of subsequent expansion. If you have any questions about InfiniBand, please feel free to contact QSFPTEK's CCIE/HCIE engineers at [email protected].