We all know that vendors sometimes, ahem, stretch the truth. So everyone looks to third-party reviews for unbiased commentary. It’s even better if that commentary comes from someone who’s had extensive experience with the product (think “long term road test”) rather than a cursory drive.

This is why I was so thrilled to see the comments below from Dan Shipley of Supplies Network.

Dan is a Xsigo customer. He wrote these comments in reply to an earlier blog post of mine that discussed the growth of InfiniBand.

Dan’s remarks very effectively get at the question of, “why IB?”

I thought they were worth highlighting, so I re-posted them below. (The picture above was pulled from Google Maps to provide a little visual context.)

If you prefer to read Dan’s remarks in their original form, you can find them at the bottom of this page.

——————————————–

It is important to note that Ethernet is not a competitor to Infiniband on a features level, only on a price level. Infiniband has features Ethernet can only dream of. FCoE is a better competitor from a features standpoint, but it can’t compete on price or performance. 10G Ethernet is just faster Ethernet. Unless you are using one of Xsigo’s new Ethernet directors, it doesn’t give you anything but speed. In the virtual world, total cost of ownership is not driven by initial capital cost, but by ongoing management overhead, and system flexibility. That is why so many are moving to virtualized interconnects, like FCoE or Infiniband. However, when you compare the cost and performance of FCoE VS Infiniband, IB is the clear winner.

Take a recent issue we faced… We decided to build out a virtual desktop environment. Like many companies, we wanted to make sure that the PC’s were secured from each other (different business units), as well as from the servers, and our PCI environment. To do that, we decided to create separate groups of virtual desktops that each had a vLAN. In addition, we decided to add the DMZ vLAN to the mix.

In total, we added 5 vLANs to our VMWare cluster. For each vLAN, we created a pair of redundant vNICs (10 total vNICs on each ESX server). We have 16 ESX servers in that cluster, for a total of 160 vNICs across the cluster. We were able to create the 160 vNICs in about 15 minutes. Next, we configured the vSwitches on the ESX servers, which took about 45 minutes. So, in an hour, we had set up five separate networks (with redundant connections) on 16 separate ESX servers, without ever leaving our desks.

How much money did we just save over what we would have faced with traditional Ethernet? A ton, and that is if we could have even done it, as the servers wouldn’t have the room for that many NICs, much less the switch ports…

Looking to the future, not only does IB have a roadmap that keeps it ahead of processor and bus speeds, but it is also provides complete offload of protocol stacks by using RDMA enabled protocols, like NFSoRDMA and iSER. As storage vendors continue to push into SSD’s, file transfers are becoming more like a momory-to-memory transfer, which is exactly what IB was built for.

Some final thoughs on IB are its extreme scalability (clusters with over 10k nodes have been built), ability to automatically build meshed and trunked networks (that don’t have the LAG/Etherchannel limitation of a conversation being limited to the speed of a single port in the group), and the strong security model of virtualized Ethernet or Fibre Channel adapters running on a non-native (IB) transport layer, and it seems like IB is hard to beat.

The only real reasons I can think of to prevent architects from choosing IB more often is fear of a “new” network (i.e. the IT group doesn’t have any IB people), and fear of choosing a smaller networking vendor (i.e. not Cisco). However, after having used the Xsigo solution, having little IB experience is no problem at all. You don’t even know it is there, and don’t have to have any IB knowledge. It is kind of like an iPhone… it just works. As far as choosing a small/new networking vendor (compared to Cisco), having some major players like VMWare and Accenture choose Xsigo overcame any issues I had there. I can say I have been very happy with the solution, but more importantly, the support I have received.

I hope others considering Xsigo would take a closer look. Once you have realized that the future is virtual, there are two choices: FCoE and IB. If you openly compare the two, there isn’t much competition, and who wants to be locked into a single vendor’s expensive FCoE implementation?