Fast Virtualized Hadoop And Spark On All-Flash Disks

Fast Virtualized Hadoop And Spark On All-Flash Disks – VMware White Paper

August 16, 2017

Best Practices for Optimizing Virtualized Big Data Applications on VMware vSphere 6.5

Best practices are described for optimizing Big Data applications running on VMware vSphere®. Hardware, software, and vSphere configuration parameters are documented, as well as tuning parameters for the operating system, Hadoop, and Spark. The Hewlett Packard Enterprise ProLiant DL380 Gen9 servers used in the test featured fast Intel processors with a large number of cores, large memory (512 GiB), and all-flash disks. Test results are shown from two MapReduce and three Spark applications running on three different configurations of vSphere (with 1, 2, and 4 VMs per host) as well as directly on the hardware. Among the virtualized clusters, the fastest configuration was 4 VMs per host due to NUMA locality and best disk utilization. The 4 VMs per host platform was faster than bare metal for all tests with the exception of a large (10 TB) TeraSort test where the the bare metal advantage of larger memory overcame the disadvantage of NUMA misses.

This paper will show how to best deploy and configure the underlying vSphere infrastructure, as well as the Hadoop cluster, in such an environment. Best practices for all layers of the stack will be documented and their implementation in the test cluster described. The performance of the cluster will be shown both with the TeraSort suite, TestDFSIO HDFS stress tool, and the new Spark machine learning benchmarks.

Download and read the ‘Fast Virtualized Hadoop And Spark On All-Flash Disks’ VMware White Paper Here

VMware VMware is the leader in virtualization and cloud infrastructure solutions that enable businesses to thrive in the Cloud Era. Customers rely on VMware to help them transform the way they build, deliver and consume Information Technology resources in a manner that is evolutionary and based on their specific needs. With 2013 revenues of $5.21 billion, VMware has more than 500,000 customers and 75,000 partners. The company is headquartered in Silicon Valley with offices throughout the world and can be found online at www.vmware.com.

Share your view, leave a comment below: Cancel reply

| LATEST FEATURED RESOURCES

White Papers

Application Lifecycle Management with Stratusphere UX – White Paper

Enterprises today are faced with many challenges, and among those at the top of the list is the struggle surrounding the design, deployment, management and operations that support desktop applications. The demand for applications is increasing at an exponential rate, and organizations are being forced to consider platforms beyond physical, virtual and cloud-based environments. Users […]

2018 Will Be The Year of Citrix Migration! eG Innovations and DABCC Survey Reveals Industry Trends for XenApp and XenDesktop 7.x Migration

IGEL Community Releases Free “How-To Install and Configure the IGEL Software Platform” Made Easy Book

IGELl’s Security Enhancements for Thin Clients – White Paper

White Paper: IT Performance Monitoring Tools – Reading Between the Lines

Does Deploying Citrix in the Cloud Make Performance Monitoring Easier? – White Paper

View All White Papers >>

Downloads

Download Commvault VM Backup and Recovery: end-to-end VM backup, recovery and cloud management

Commvault’s ability to provide end-to-end VM backup, recovery and cloud management creates a significantly better way to build, protect and optimize VMs throughout their lifecycle. Our best-in-class software for VM backup, recovery and cloud management delivers a number of significant benefits, including: VM recovery with live recovery options; backup to and in the cloud; custom-fit […]

Download Commvault Data Platform: enterprise-wide data protection and management

Download ‘NAKIVO Backup & Replication for VMware, Hyper-V, and AWS EC2’ Full-Featured Free Trial

Download deviceTRUST Dynamic context awareness for Citrix

Download NAKIVO Free VM Backup and Replication for VMware & Hyper-V

FREE eG Enterprise Logon Simulator for Citrix XenApp and XenDesktop

View All Downloads >>

On-Demand Webinars

What’s Going on in EUC Printing – A Technical Deep Dive!

The IGEL Community and ThinPrint invite you to watch the following technical deep dive webinar. The agenda is to technically bring you up to speed on what’s going on in the EUC Printing space today along with a deep dive into new methods, technologies, printing scenarios and a discussion on why printing still matters. You […]