VMware Video: Running Spark on VMware Cloud on AWS
This demo shows the Standalone Spark workload that we saw previously running on on-premises vSphere – now running on a set of 12 Worker virtual machines and 1 spark driver virtual machine on the VMware Cloud on AWS platform. A Spark cluster is started up on those 13 virtual machines from the command line and a logistic regression workload with 1 million examples and 10,000 features is executed on it and shown in the Spark console. This data is generated during the run. The entire test takes less than 3 minutes to complete and the resulting Model Training Time is shown to be 15.88 seconds. This would be something that a data scientist/developer would be interested in testing.
This video is from the fine folks at VMware.