Hadoop Struggles and BI Deals: What’s Going On?

June 21, 2019

The news hasn’t been good in the Hadoop world over the past two weeks, with MapR and Cloudera both revealing themselves to be in bad straits. Meanwhile, the market for BI tools suddenly surged, with Tableau, Looker, and ZoomData all getting snapped up. So what’s driving this sudden flurry of activity?

First, let’s start with Apache Hadoop. The Hadoop community has been struggling for some time, even before Cloudera agreed to acquire rival Hortonworks in a blockbuster deal last October. For years, practitioners have complained that Hadoop is too complex, and that users must be experts in difficult-to-use tools to get useful data out of it.

That complexity has fueled a run of anti-Hadoop-ism in the big data market, and has caused architects to rethink their ideas about building giant data lakes designed to address all of an enterprise’s analytic needs. Instead of singular do-it-all data platform, we’ve seen a resurgence of investment in more specialized point products, the data silo problem be damned.

However, Hadoop still had a couple of things going for it. First, it’s still better than what came before it. The capability to store tens of petabytes on parallel HDFS clusters at a fraction of the cost of a typical SAN deployment is a big deal. Also, we thought that Hadoop had a bigger runway and more time. Since enterprises have already committed billions to building these big data lakes, the idea that they would just pull the plug because there were some technical challenges seemed unlikely. (After all, what major IT project doesn’t have any technical challenges?)

What’s more, companies in heavily regulated industries, like financial services and healthcare, can’t easily move to the cloud, so they were assumed to be committed to the Hadoop approach for the long run. And while banks and hospitals may still be running Hadoop clusters in 10 years, that hasn’t stopped a good cross-section of users in other industries from giving up on their Hadoop clusters in favor of the cloud.

That benefit has accrued to Amazon Web Services, Microsoft Azure, and Google Cloud, all of which offer a range of big data storage and processing services that, if not based on actual Hadoop components, at least offer the equivalent to Hadoop. Clearly, the cloud giants are eating into the margins of Cloudera and MapR Technologies sales teams, and we’ve seen the results manifest with the surprise announcements in the past two weeks.

The future is not clear for either Cloudera and MapR. While there are similarities in the two companies’ positions, there are big differences too.

Cloudera does not have a permanent CEO at the moment, and it still hasn’t shipped the new converged Hadoop distribution, dubbed Cloudera Data Platform (CDP), that will replace the old Cloudera and Hortonworks distributions. During its first quarter ended April 30, Cloudera said customers are holding off investing in the old Hadoop products since they know the new CDP is due by the end of the year. That fact led Cloudera to dramatically lower its revenue expectations for the year, which upset stockholders, who pushed Cloudera’s stock (NYSE: CLDR) down 40% the following day.

Cloudera is still banking on its hybrid multi-cloud strategy in giving it an edge over its cloud rivals. As easy as AWS, Google, and Microsoft make onboarding into big data processing on the cloud, many customers are wary of cloud lock in. Cloudera knows this, which is why it’s proposing to be your trusted data management partner, able to chart a clearer data management course across its customers’ cloud-to-cloud and cloud-to-on-premise systems.

CLDR is up a little bit today, but it’s nowhere near where it was this time last week. However, there are reports of some sizable chunks of CLDR being bought at the stock’s historic low price. The last chapter on Cloudera has yet to be written.

As bad as Cloudera’s position is, MapR’s is much worse. Like Cloudera, the company was also taken by surprise by poor first quarter sales. But in MapR’s case, the bad first quarter led a major investor to back out of a deal to infuse more capital into the company, which has already raised $280 million but is not yet profitable. The company is still searching for a deal, but it must find funding by the end of this week to avoid laying off more than 120 employees at its Santa Clara, California headquarters.

Despite developing what’s considered the Cadillac Hadoop distribution and attracting many big-name customers, MapR today is on a lifeline. If the company doesn’t find a source of funding, it will likely be forced to sell itself to another firm. The odds of the MapR investors recouping their investments at that point seem rather poor.

Meanwhile, another big data player is on the hot seat following poor first-quarter results. Pivotal Software, which once had a Hadoop distribution but now sells a variety of open source software via its Cloud Foundry, saw its stock price plummet more than 40% last week.

Pivotal is banking on its software being used to help companies make the transition to the cloud as part of their digital transformation stories. But enterprises are finding it might be easier just to place their IT chips directly on the cloud vendors’ properties.

“Everybody in the data management space is facing the biggest headwind they have faced in 20 years,” Gartner analyst Merv Adrian told the Wall Street Journal. “The cloud eats everything.”

Forrester analyst Brian Hopkins has been predicting the cloud’s disruption of Hadoop for a while now. Back in 2017, he wrote something that, in light of recent events, looks ominously prescient. “We think Hadoop has a future and will have strong growth for at least two to three more years, but ultimately it will take its place alongside data warehouses and mainframes,” Hopkins wrote. “The pace of digital change in the age of the customer is simply too fast for any one technology to keep up for long.

We’re seeing a different but related dynamic at play in the business intelligence space, which has been in a state of consolidation since Qlik sold itself to private equity firm Thom Bravo in 2016 for $3 billion.

Last week, Google spent $2.6 billion to acquire Looker, an up-and-coming Santa Cruz, California-based company that had raised $280.5 million in venture funding to date. It was no fire sale for Looker, which has been growing quickly lately with its BI tool and data platform. The company’s approach to BI – that you should query the data where it resides and use data definitions defined by its LookML modeling language to ensure consistent results – appears to be resonating with customers.

Looker has been a close partner with Google, particularly with its BigQuery database. But the company has also worked with AWS Athena and Redshift, the Azure SQL Data Warehouse, and more recently Snowflake Computing, which itself has eaten into Hadoop’s once-formidable market share.

Looker founder and CTO Lloyd Tabb noted how data and workloads were moving to these cloud-based data warehouses two years ago. “We’ve seen really big companies who have big Hadoop clusters move their data to the cloud because they’re much more agile,” Tabb told Datanami back in October 2017. “We keep seeing this over and over. Big time companies who I can’t name…spent all this money building these big Hadoop infrastructures, and then they say, we can spend that engineering on figuring out answers to things instead of managing the cluster.”

The latest news is Salesforce’s acquisition of Tableau for $15.7 billion, a 30% premium over its stock price. Like Qlik, Tableau was considered one of BI’s “new guard” vendors to take over the market following the run of first-generation tools like Cognos and Business Objects. Now we’re seeing the new guard get gobbled up by vendors with big cloud aspirations.

And not to be outdone, LogiAnalytics announced today that it has acquired ZoomData, which allowed customers to analyze data in real time. It was the second acquisition of the year for Virginia-based LogiAnalytics, having nabbed Jinfonet earlier this year.

Those who have followed the analytics space closely shouldn’t be too surprised by the acquisitions, says Sudheesh Nair, CEO of ThoughtSpot, a Palo Alto, California-based provider of BI and search software.

“Whenever there is a fundamental shift in how people access and use data, we’ve seen major consolidation,” Nair tells Datanami. “In the mid-2000s as data visualization was on the rise, we saw a huge wave of consolidation from the traditional report-driven BI players. IBM acquired Cognos, SAP acquired Business Objects, and Oracle acquired Hyperion. As we move beyond data visualization into a world powered by AI and search, we’ll see more and more consolidation. Don’t expect companies whose signature differentiator was visualization to stay independent for long.”