Mobile Apps: The Miniaturization of Big Data
What is Big Data?
Big Data is a phrase used by many large corporations today to describe extremely large datasets, sometimes on the order of petabytes (1,024 terabytes) or exabytes (1,024 petabytes), that are so large they are difficult or impossible to process using traditional tools and databases. In some cases, it is not the size of the dataset that causes it to be referred to as "big data"; it is the speed at which the data is captured or made available that is too large to process or filter using conventional means. And sometimes, organizations use "big data" to mean not the data itself, but the technology or resources used to store it or process it. As such, it is an amorphous term that at this point in time has no fixed meaning, but generally refers to previously incomprehensibly large data and the technology used to manage it.
What is Hadoop?
Hadoop, or Apache Hadoop, is an open-source framework created in 2011, written in Java, C, and Shell script, that enables the processing of unlimited sizes of Big Data on generic, industry-standard hardware (machines known as "clusters") through a processing system called MapReduce. This is in contrast to proprietary hardware and software that is usually limited in terms of the data or throughput (or both) that it can handle. Hadoop is by nature unlimited, and it can import mismatched and unrelated data sets, even those of completely different media types. Now, disparate data sets in multiple locations in different formats can all be integrated in one Hadoop collection and managed with one set of tools. Hadoop builds in protection from hardware and software failures through redundancy, and it was built to expect such failures. The term "Hadoop" comes from the name of a toy elephant belonging to the son of Cloudera engineer Doug Cutting, the inventor of Apache Hadoop.
What is Business Intelligence?
Business Intelligence (BI) is the tools and systems needed to extract answers to strategic business questions that can be answered by Big Data. Often these questions have to do with market research, customer profiling, customer support, inventory and distribution analysis, statistical analysis, and product profitability. In the past, multiple applications, such as spreadsheets and databases were needed to process each part of Big Data separately. Now, using specialized software designed for BI on Hadoop, Big Data can be examined with one or just a few tools that include analytics, visualization, and data mining, so that information can be turned into intelligence.
Many vendors such as SAP, Oracle, IBM, and others now offer connectors from their software to Hadoop setups, using industry standard protocols such as SQL.
Mobile Apps for Big Data
Mobile Apps for Big Data tend to be focused on monitoring, dashboards, or high-level management tasks, as processing Big Data is not a task suitable for a mobile device.
Some companies using mobile apps for Big Data include:
- Guess? — This clothing company’s buyers typically are mobile and carry a device with them to tap into data about sales, inventory, and logistics. For its custom application, Guess? tapped HP’s Vertica Analytics platform and an iPad App called G-Mobile. Currently in use in the U.S., the company plans to roll out the platform to Asia and Europe in the near future.
- Empirix — Guess? (see above) is not the only company making use of HP’s Vertica platform — Empirix is using it for end-to-end testing, monitoring, and analytics of contact centers and unified communications systems for cable operators, airlines, telecoms, insurance agencies, and banks. Empirix has an OEM arrangement with HP for its product Empirix Intellisight, specifically focused on telecommunications and mobile business analytics.
Is It Safe?
With any centralized system weakness will always follow. As recently seen in the NYSE and American Airlines incidents even the most secure systems can be breached. One of the most interesting examples of big data safety can be found in Bluecoats recent piece on the history of hacking. In 2014 EBAY’s main database was hit by a massive hack exposing more than 145 million records. Much of this included login information and other confidential documents. Despite these inevitable hacks Big Data still allows us to do more information than ever before.
Other companies working on mobile solutions using Big Data include:
- ScanBuy — Scanbuy’s Scanlife App is used by over 6 million people to get product and pricing information from QR codes, UPC codes and other real-world printed packaging. Now, the company is going to be tapping into Big Data to deliver the same information, but more intelligently, to provide smarter and more targeted answers for consumers.
- Microsoft — The software giant has integrated Hadoop cluster management and deployment into Windows Azure for mobile use using the HDInsight service. Based on the Hortonworks Data Platform 1.1, Azure can support iOS, Android, and Windows Phone 7.5 and 8. It also will provide Dropbox integration, and it has the same storage API and data querying support that Microsoft offers in other products.
- DMI (Digital Management, Inc.) — DMI provides responsive web, back-office, and back-end mobile services for Google Cloud Platform to customers such as Anheuser-Busch Inbev, Vodafone, Tesco, Virgin, Unilever, Telenor, and others. DMI will be expanding its platform collaboration to North America after extensive customer implementations in Europe.
Written by: David Schwartz is a technology angel investor and startup consultant based out of Aspen, Colorado. In addition to taking several startups to IPO stage he spends most of his time mentoring young tech entrepreneurs. You can connect with David onGoogle+ and Linkedin