InfiniBand

InfiniBand is a high-speed, high-bandwidth, low-latency network communication protocol and interface standard that can improve CPU utilization, reduce latency, and simplify data center management issues. InfiniBand is widely used in replication and distributed work between servers, SAN and DAS between servers and storage devices, and LAN, WAN, and Internet communications between servers and networks.

How Does InfiniBand Work

InfiniBand typically uses a two-layer architecture that separates the network layer from the physical layer and the data link layer. Each layer has specific responsibilities. The physical layer provides direct point-to-point connections between devices through high-bandwidth serial links, while the data link layer is responsible for managing the transmission and reception of packets between devices. The network layer provides key InfiniBand features such as QoS, virtualization, and RDMA. The synergy between the layers makes InfiniBand a powerful tool for supporting low-latency, high-bandwidth high-performance computing (HPC) workloads.

Advantages of InfiniBand

InfiniBand offers several advantages, some of which we'll cover next.

RDMA Support

InfiniBand supports RDMA, which is remote direct access to memory. It can directly access the memory remotely without the data passing through the CPU, allowing data to be transmitted in the memory. This can not only effectively reduce transmission latency, but also improve CPU resource utilization, especially in HPC application environments, improving the overall efficiency of the system.

CPU Offloading

CPU offloading is an important means of accelerating computing, and the InfiniBand network can use the least CPU resources for data transmission. They are functionally compatible. InfiniBand supports the RDMA function, which can directly write data from the memory of one server to the memory of another server without going through the CPU for processing. It can achieve zero-copy around the kernel.

InfiniBand supports QoS

First, InfiniBand implements QoS through virtual channels. A virtual channel is an independent logical communication path that shares a single physical link. Each physical link can support up to 15 standard virtual channels and one management channel.

Second, InfiniBand uses service level marking (SL) to mark the SL field in the packet header and divide it into 0 to 15 levels, each corresponding to a different priority.

In addition, InfiniBand supports the differentiated service architecture defined by IETF RFC 2474 and 2475, thereby implementing QoS functions similar to DiffServ.

Finally, InfiniBand can also control the parameters of each communication flow by responding to path records or multi-path record queries. Administrators can optimize these paths to ensure that high-priority traffic is transmitted through the optimal path.

SHARP Support

SHARP is a protocol for HPC systems that aims to improve the communication efficiency of HPC clusters. By implementing data aggregation and reduction operations in network devices instead of sending all data back to the central node for processing. This can optimize collective communication and avoid repeated data transmission between nodes, which can significantly speed up the computing process in applications that need to process large amounts of data and perform complex collective operations.

Application of InfiniBand

InfiniBand plays an indispensable role in today's industry, and its characteristics make it a key solution for multiple industries.

HPC Environment

In the HPC environment, InfiniBand provides high bandwidth and low latency to help it quickly process large amounts of data, and through protocols such as RDMA and SHARP, it helps it quickly transmit and avoid repeated data transmission, making it more efficient.

Storage Network

InfiniBand also plays a very important role in storage networks. It can bypass the CPU and directly transmit data in the application memory, which realizes efficient data access and transmission and improves the performance of the storage system.

Conclusion

As a vital part of modern networks, InfiniBand has brought about a crucial change for applications such as data centers and high-performance computing that need to process large amounts of data simultaneously. It not only brings low latency and high bandwidth to these applications but its scalability and flexibility are also obvious to all. And with the development of technology, its performance is constantly improving, which enables it to meet the growing business needs. If you have any questions about InfiniBand, please feel free to contact us.