Virtualization and the Cisco Nexus Switch Combine to Kill Blade Servers
It is amazing how frequently we hear IT managers talk about deploying blade servers as an integral component of their new virtual infrastructures – as if there were an obvious synergy between VMware and blade server architectures. The opposite, in fact, is nearly always the case. There are three situations in which it makes sense to use blade servers in a physical (aka 1.0) data center: a limitation of data center space, the need to hot swap blade servers and a desire to simplify cable management. The first two are rendered irrelevant by the server consolidation and VMotion capabilities of a virtualized data center. The advent of the Cisco Nexus switches eliminates the third. Server manufactures will mourn the loss of customer captivity that blades promote, but their time has nonetheless passed.
Why Blades Have Never Made Much Sense in Virtual Infrastructure Architectures
Blade servers have always been an impediment to an optimal virtual infrastructure because they introduce limitations in efficiently utilizing power and cooling resources, budget, flexibility, manageability, bios and firmware updates, performance and troubleshooting.
Power and Heat: Limited power and cooling resources are typically a larger constraint than space even in physical data centers. They become a far more obvious concern in virtual infrastructures where the space issues are resolved through server consolidation. Server manufacturers like to claim that blades are more energy efficient than traditional rack mount servers, but this is only true on an individual basis. The combination of blade density and a power-hungry blade chassis results in a higher power consumption per rack and generates more heat. This same density inhibits heat dissipation by restricting the air flows that circulate through traditional rack mount servers.
Budget: Blade servers require a significantly larger investment than stand-alone servers as well as a much more expensive replacement cost.
Limited Flexibility: Standardization is lacking between different brands of blade servers. Purchasing a blade server locks an organization into a manufacturer’s vision of a proprietary chassis, blades and management system. In order to take advantage of a new generation of chip from Intel or AMD, for instance, the entire chassis may need to be thrown away. Since blade servers are more expensive than traditional servers, they discourage incorporating new server technology such as a faster bus. And while the number of PCI slots in blade servers has increased, they are still less than we’d like in order to set up multiple virtual switches since VMware only allocates a set amount resources to a single vSwitch.
Manageability: Blade servers require their own management consoles which tend not integrate into virtualization platforms such as VMware. This can lead to awkward situations where multiple engineers need to simultaneously work on the VMware servers, yet only one can access the blade server’s management console.
Bios & Firmware Updates: Bios updates on the blade server chassis and firmware updates on the virtual interconnects add management overhead during maintenance periods and can also lead to downtime. With traditional servers, the virtual machines can simply be VMotioned off of the server being updated.
Performance Aspect of Virtual Interconnects: Many blade servers are purchased with virtual interconnects (blade switches) in order to eliminate the requirement of dedicated Ethernet and fibre channel cables along with dedicated uplink ports. Virtual interconnects, though, add an extra layer of virtualization that can degrade performance on both fibre and on Ethernet networks.
Troubleshooting: Blade server environments require more complex troubleshooting as problems can be related to the blade, chassis or interconnect.
Cisco Nexus Switches
Virtualization is quickly revolutionizing the data center – enabling server provisioning in minutes, accelerating and integrating the test/development cycle, and making downtime from server failure a thing of the past. But the dynamic virtual data center cannot be achieved successfully with just server virtualization. The underlying network infrastructure must be capable of handling a dense virtual machine environment while providing the deterministic performance, security, availability and manageability required. Rob Whitely of Forster Research says that once an organization virtualizes around 25% of its servers, it begins to experience the network as a bottleneck. As desktops increasingly become virtualized and move to the data center, the network demands become even more pronounced.
Cisco Nexus switches resolve the networking challenges of virtual infrastructure. The Nexus product line consists of a virtual switch: the Nexus 1000V, and two hardware switches: the Nexus 5000 and 7000. Cisco holds 1,500 patents on the Nexus 7000 alone. Nexus was actually developed outside of Cisco as part of a company called Nuova that included both Cisco executives and a VMware co-founder. Cisco helped fund Nuova after a couple of years of development and work on Fibre Channel Over Ethernet in the various standards bodies. Cisco eventually purchased the company which became Cisco’s Server Access Virtualization Business Unit (SAVBU)
The Cisco Nexus Switches enable a consistent provisioning process from the virtual machines to the core switches. The Nexus 5000, and next year the Nexus 7000, models include 10G lossless packet connectivity and unified I/O to enable much faster virtualization functions such as VMotion, Distributed Resource Scheduling and High Availability. The Nexus 1000v replaces the VMware virtual switch in the hypervisor, while the 1000v and Nexus 5000 both leverage VM-Link to allow network security and QOS profiles to be distributed between ESX servers during VMotion of virtual machines.
The one appeal of blade servers still extant in a virtual data center was a reduction in cable management complexity because individual adapters are not required for each blade. As Figure 1 shows, however, a Consolidated Network Adapter (CNA) in conjunction with Nexus allows for multiple interfaces/ protocols on a single network adapter – thereby eliminating the blade benefit. This results in a much more elegant solution by simply connecting stand-alone servers to the ESX hosts via the Nexus switch.
Figure 1: Nexus Unified I/O Fabric in a VMware Infrastructure
The Nexus Nail in the Blade Server Coffin
The blade server concept conflicts with a Nexus strategy. The switches that blades incorporate entail an extra physical device between the server and the physical network which is equivalent to an extra hop on layer one. This results in increased latency and network traffic. Traditional servers, on the other hand, enable Nexus connectivity directly to the ESX hosts – a more efficient architecture. And Nexus provides a single point of management for all virtual and physical switches.
Contrary to popular sentiment, we have long advocated against the use of blade servers in a virtual infrastructure architecture. Pointing out the myriad disadvantages of blade servers has always enabled us to successfully convince our clients to go the traditional rack mounted server route instead. This is true for even fairly large organizations – one of our clients recently virtualized 915 physical servers onto 39 new HP DL580 quad-core 4-CPU boxes with 128GB of RAM. As organizations increasingly embark upon enterprise virtualization as their platform, and as Cisco makes its Nexus foot print increasingly prominent in the data center, the blade servers as we know them today will gradually become history.
Blade Server / Data Center Evolution
Like most technologies, blade servers are far more likely to evolve rather than to just disappear. It is an easy reach to to imagine Cisco themselves developing a server blade for the Nexus 7000 switch. This would enable a direct connect to Cisco’s high speed fabric using Nexus supervisor modules. Each server blade would then have up to 46Gps of bandwidth. There would be near zero latency between server blades, meaning that ESX host to host communications would travel at the bus speed of the Nexus frame (1150Gps.). As long as all the ESX hosts are on the Nexus 7000 farm, network traffic congestion will be nonexistent. If this scenario should unfold, instead of talking about just a unified fabric, we’ll likely be talking about a unified data center.
Written By: Steve Kaplan and Mondy Carpio