InfiniBand or IB is a computing networking of communications standard that is used in high-performing computers. It features low latency and very high throughput. You can use data connectivity within and among computers.
You can also use it for either switched or direct interconnect between storage systems. Its design is meant to be scalable and uses the switched fabric network topology, which we shall be talking about later on in this review.
InfiniBand was created in 1999, as a merger between two competing designs – the Next Generation I/O and the Future I/O. This later led to the formation of the InfiniBand Trade Association – IBTA. Read on to learn more about InfiniBand technology.
What Is the IBTA?
IBTA includes companies, such as Dell, Compaq, IBM, Hewlett-Packard, Sun, Intel, and Microsoft. At the time of its creation, it was considered one of the most powerful interconnectors of computers. Its architecture was released in 2000.
As of 2014, InfiniBand is one of the most used interconnectors in Supercomputers. Intel and Mellanox are the main inventors of InfiniBand, and in 2016, Oracle Corporation announced that they had engineered their own InfiniBand, which would be used in their own product lines.
As an interconnector, InfiniBand competes with fiber channel, Ethernet, and Intel Omni-Path. Initially, the vision for creating IB was as a replacement for PCI in the I/O Ethernet in the cluster interconnect, machine room, and fiber channel.
IBTA envisaged decomposing the server hardware on the IB fabric. However, after the burst for the dot-com bubble, there were some delays and hesitations in the industry in the investment of such a far-reaching technological jump.
Which Topology Does it Use?
InfiniBand makes use of the switched fabric topology as opposed to the earlier and shared medium of Ethernet. All of its transmissions start and end at the channel adapter. All of the processors contain a host channel adapter – HCA and the peripheral also has a targeted channel adapter.
All of these adapters can be able to exchange information for security purposes and for the sake of quality of service or QoS.
Messaging System
InfiniBand transmits its data packets up to 4KB, and they are taken together so they can form a message. Their messages are listed below.
- Remote direct memory access for a read or write
- A channel sends or receive message
- A transaction-based operation – this can be reversed
- A multi-cast transmission
- An atomic operation
Physical Interconnection
Other than the board form factor connection, InfiniBand can also use both the passive as well as active copper, or up to 10 meters, with an optical fiber cable of up to 10km. In this case, you should use QSFP connectors.
The InfiniBand Association has also specified the CXP connector system as the one to use for speeds that are up to 120 Gbit/s. This is better than the copper active optical cables.
API
InfiniBand doesn’t have any standard API. The only standard listed is a set of verbs such as IBV post send, and IBV open device. These are abstract representations of the methods and functions which must exist.
The syntax of the functions is usually left to the vendors, and sometimes the reference can be called verbs API. The de-facto standard software stack was developed by the OpenFabrics Alliance. This was released under two main licenses, the BSD License, and the GPL2.
You can also find it under the name GNU for Linus, and FreeBSD and Mellanox OFED for Windows. The product names, in this case, are WinOF-2 and WinOF. These are attributed to the host controller driver that matches the specific ConnectX to either 3 or 5 devices.
Ethernet Over InfiniBand
The Ethernet over InfiniBand is usually abbreviated as EoIB. This is an Ethernet implementation that is over the InfiniBand connector technology and protocol. It enables for multiple Ethernet Bandwidths that are varying on the InfiniBand Version.
Ethernet’s implemented the Internet Protocol Suite that is usually referred to as the TCP/IP. This is very common, and what most people use, but it is different from some of the other implementations used in the InfiniBand protocol IP over IB.
Conclusion
Those are some of the facts that you need to know about InfiniBand technology. This is similar to Ethernet, but for supercomputers, and with a higher transmission speed, and various ways of sending messages.
