InfiBand breaks server bottleneck
For many large organizations, it’s not the CPU that’s the bottleneck on server performance. It’s the I/O subsystem. Often there’s just no way to squeeze all the data a server’s applications need from disk to CPU and back again.
There are several attempts on the drawing board to boost system I/O. Storage area networks, or SANs, have gotten a lot of press. Most SANs today run over a transport called Fibre Channel, which provides a separate switched infrastructure between servers and storage devices. More recently, vendors have been developing iSCSI, a standard way of sending the SCSI storage protocol over IP networks. A company called Nishan Systems even provides IP storage switches that support both Fibre Channel and iSCSI connections.
Each approach has its advantages and disadvantages. All share one common disadvantage: When data arrives at the server, it’s limited by the speed of the 133MHz PCI bus to 1 gigabit per second–admittedly nothing to sneeze at, but slower than the rest of the storage infrastructure can go.
InfiniBand is a new technology that does away with the concept of an I/O bus. Instead, it offers a point-to-point switched I/O fabric between a host channel adapter on the server and a target channel adapter on the storage device. A single InfiniBand link operates at 2.5 gigabits per second in a single direction, while a bidirectional link has twice that rated capability. While real-world traffic will fall short of the rated throughput, it should be a significant fraction thereof. By adding multiple channels, vendors can build a really obese pipe between storage and CPU resources.
The end node devices can be interconnected by cascaded switches. That gives InfiniBand the ability to scale from a single server up to a system of a much larger scope.
InfiniBand’s Progenitors were two earlier groups: Next-Generation I/O, put forth by Intel, Microsoft, and Sun; and Future I/O, supported by Compaq, IBM, and Hewlett-Packard. Rather than compete, the two camps decided to unite as the InfiniBand Trade Association to fight the I/O bottleneck, and thereby make the resulting products more appealing to the entire industry.
The initial spec for InfiniBand was released late last year. This year, vendors have been building prototypes and running them together in interoperability tests, including a recent multivendor Plugfest. According to Alisa Nessler, CEO of Lane 15 Software, an InfiniBand management software vendor, the Plugfest results were mostly positive.
If testing continues to go well, expect to see the first InfiniBand products available for sale early next year. Initially, they’ll be available only for high-end server and storage clusters, but as the technology becomes more popular and prices decline, expect InfiniBand to find its way into even departmental servers.
InfiniBand won’t replace Fibre Channel or iSCSI, but it’s a welcome additional alternative for filling bandwidth requirements. If you haven’t committed to either existing camp yet and you can afford to wait a year, keep your eye on InfiniBand.
Articles | Administrator | 
Tuesday, September 4, 2001 12:00 pm
Subscribe