(By Mani) EMC Corporation (NYSE: EMC
) and VMware,Inc. (NYSE: VMW
) announced a Hadoop distribution Pivotal HD, which would be a tough competitor to Red Hat, Inc. (NYSE: RHT
) as early indicators point that the Pivotal has the potential to bring a radical shift the enterprise software market.
The latest announcement comes after both companies launched a new firm Pivotal Initiative in December to tap the rapidly growing big data and cloud application market.
Pivotal HD, which is virtualized by VMware, along with HAWQ represents the world's first true SQL processing for Hadoop, expanding the platform's reach to structured query language (SQL) programmers.
The combination of EMC, VMware and Pivotal is expected to gain significant market share in the new IT world in the next 6-8 months, while Red Hat may have to struggle.
Red Hat provides open-source software products to the enterprise community via its Enterprise Linux operating system. It also offers Red Hat Virtualization, a virtualization solution for server and desktop computers that combines the kernel-based virtual machine hypervisor with the oVirt open source virtualization management system. Further, it provides various cloud, storage, and systems management offerings, as well as offers training, consulting, and support services.
"Red Hat (RHT) and Pivotal are probably going to collide 6 – 8 months from now, with probably Pivotal winning," Global Equities Research analyst Trip Chowdhry wrote in a note to clients.
EMC's new Pivotal HD is an Apache Hadoop distribution that combines EMC Greenplum massively parallel processing (MPP) database technology with the Apache Hadoop framework, which is considered as one of the most cost-effective and flexible open source Big Data platform.
Pivotal HD is GreenPlum specific Hadoop implementation. GreenPlum takes the raw bits from Apache.org and make enhancements to those raw bits, similar to other Hadoop distributions like Cloudera, MapR and HortonWorks.
The solutions would be integrated by a technology called HAWQ, which was developed by EMC's data analytics arm Greenplum. HAWQ technology outperformed Cloudera, which was the first to develop the SQL interface on top of Hadoop, known as Impala.
New HAWQ technology brings more than 100X performance improvement to a wide range of query types and workloads, making Pivotal HD the single most powerful Hadoop distribution in the industry.
What it really means is that it takes zero dollars to get the Hadoop for GreenPlum, and GreenPlum can put investment dollars to focus on the analytics part of the stack and not on the infrastructure part of the stack as Pivotal gets the infrastructure part of the stack free from Apache.org
Pivotal HD is Free for first 50 nodes and customers will pay Pivotal for using the Virtualization Extensions. This business model is similar to VMWare's Vcenter. In addition, Hadoop Virtualization only works on VMWare Hypervisor, which means VMWare ecosystem benefits from Pivotal HD.
Moreover, the industry is demanding a SQL interface to Hadoop because they have a lot of people with SQL skills which is the prime language to interact with databases.
Hadoop has rapidly emerged as the preferred solution for Big Data analytics applications that grapple with vast repositories of unstructured data. It is flexible, scalable, inexpensive, fault-tolerant, and enjoys rapid adoption rates and a rich ecosystem surrounded by massive investment.
However, customers face high hurdles to broadly adopting Hadoop as their singular data repository due to a lack of useful interfaces and high-level tooling for Business Intelligence and data mining—component. Pivotal HD addresses these challenges as SQL-trained data analysts, and standard BI tools can now easily connect to, query, and analyze data sets stored in the Hadoop file system (HDFS).
By offering the full spectrum of the SQL interface, and by extension, the entire ecosystem of products that support SQL, customers no longer need an army of developers to build a dashboard or run a report. Unlike competitive Hadoop distributions, Pivotal HD does this without moving data between systems or using connectors that require users to store the data twice.
Red Hat recently said it would contribute its Red Hat Storage Hadoop plug-in to the Apache Hadoop open community to transform Red Hat Storage into a fully-supported, Hadoop-compatible file system for big data environments. Red Hat is building a robust network of ecosystem and enterprise integration partners to deliver comprehensive big data solutions to enterprise customers.
"Pivotal HD is a Superior Offering" was the unanimous view we got," said Chowdhry who spoke to about 10 people who have done and tested various Hadoop implementations