It was the first widely-adopted open source distributed computing platform. But some geeks running it are telling Datanami that Hadoop “is great if you’re a data scientist who knows how to code in MapReduce or Pig…but as you go higher up the stack, the abstraction layers have mostly failed to deliver on the promise of enabling business analysts to get at the data.” Slashdot reader atcclears shares their report:
“I can’t find a happy Hadoop customer. It’s sort of as simple as that,” says Bob Muglia, CEO of Snowflake Computing, which develops and runs a cloud-based relational data warehouse offering. “It’s very clear to me, technologically, that it’s not the technology base the world will be built on going forward”… [T]hanks to better mousetraps like S3 (for storage) and Spark (for processing), Hadoop will be relegated to niche and legacy statuses going forward, Muglia says. “The number of customers who have actually successfully tamed Hadoop is probably less than 20 and it might be less than 10…”
One of the companies that supposedly tamed Hadoop is Facebook…but according to Bobby Johnson, who helped run Facebook’s Hadoop cluster before co-founding behavioral analytics company Interana, the fact that Hadoop is still around is a “historical glitch. That may be a little strong,” Johnson says. “But there’s a bunch of things that people have been trying to do with it for a long time that it’s just not well suited for.” Hadoop’s strengths lie in serving as a cheap storage repository and for processing ETL batch workloads, Johnson says. But it’s ill-suited for running interactive, user-facing applications… “After years of banging our heads against it at Facebook, it was never great at it,” he says. “It’s really hard to dig into and actually get real answers from… You really have to understand how this thing works to get what you want.”
Johnson recommends Apache Kafka instead for big data applications, arguing “there’s a pipe of data and anything that wants to do something useful with it can tap into that thing. That feels like a better unifying principal…” And the creator of Kafka — who ran Hadoop clusters at LinkedIn — calls Hadoop “just a very complicated stack to build on.”
Read more of this story at Slashdot.
https://slashdot.org/slashdot-it.pl?op=discuss&id=10407685&smallembed=1