About 5 years back, cloud computing was the best thing since sliced bread. Numerous books were written about what it is and how it will revolutionize everything. Really?
Move ahead few years, and we have a new kid on the block (or a shiny toy) called “big data”. Few books are already out on big data and how our lives would change forever. Maybe?
The search trends for “cloud computing” and “big data” indicate how cloud computing has lost is lustre, and “big data” is taking up the mindshare of boardrooms and data scientists alike.
Maybe there is a simple lesson for peddlers of the next IT revolution. Focus on talking less about what it is and what it can do, and more on creating measurable value for enterprises, i.e., positive impact on enterprises.
Maybe there is a more important lesson for buyers of “big data”. If your organization currently does not leverage existing data, ignore the IT peddlers, and instead focus on creating value from existing information assets – small, medium or big data. Once you have the organizational mindset, culture, and talent to act on insights and foresights from currently available data, even if they are in separate functional silos, then venture out into the world of “big data” in small steps.

A cautionary note on Big Data in Big Data News: Mad Men Taking Over.

“Today’s meme is big data and we are sold an image of omnipotent data and algorithms, where numbers are more important than meaning, a mix of truth and fiction conjured to advance the story-tellers’ interests. The mix is so powerful, the data so intoxicating, that the data sellers can’t see the implications of their bragging and the potential consequences to society.”

Krebbers (VP of architecture at Shell) said the best benefits of big data were realised by using a mixture of technologies (quoted in Data quality more important than fixating over big data)

“A mixture of more expensive in-memory, for speed; SQL, for a more economical option; and Hadoop, as a means of storing data you don’t know what to do with“

                                                   **************************

Even large, well-oiled organizations such as Shell are in a data-grab mode in terms of storing “all” data in Hadoop, without a well-thought through, business-driven data-capture and decisioning approach.

A quote from Splunk, which just launched Hunk – a new tool in the Hadoop ecosystem.  This quote captures the essence of the “Big Divide” in Big Data:

“Hadoop is becoming the OS for where you store data,” Clint Sharp, Splunk senior product manager for big data. “I talk to a lot of customers who are not struggling to put data into Hadoop, but they are struggling to get value out of it.”

                                                 *********************

Also, Hunk claims a voodoo feature: There is no requirement to “understand” data upfront, simply point Hunk at the Hadoop cluster and start exploring data immediately. 

If some of the so-called innovators in Big Data highlight such features, no wonder numerous Hadoop deployments fail, and Big Data gets a bad wrap. It is critical to remember that Big data are just data, despite all the hype.  Data are critically valuable to business decision-making, but understanding what it is, where it comes from, its inherent uncertainty and biases, how it could influence and impact decisions, and how it could augment our brain’s ability shouldn’t be ignored.

A nice metaphor for Big Data in an InformationWeek article NoSQL Vs. Hadoop: Big Data Spotlight At E2

“In contrast to NoSQL, Hadoop seems to be getting all the credit it deserves and then some. By many accounts, it’s the be-all and end-all of big data, despite the fact that the lion’s share of deployments today are little more than digital landfills.”

                                             *************************

It is high time we generate some value from digital landfills, starting with landfill gas.

A simple practical quote on generating value from data in Big data brings new vigor to health research

“Having somebody who knows how to do data and run a query is great, but until it gets to the person who understands what’s going on and what to do with that information, it doesn’t really matter,” Collins said.

                                                     *******************

Realizing business value by taking actions from insights trump our ability to generate the insights in the first place.

Comical aspect of the quote:  What does “do” data really mean?

Machines/sensors and Big Data

Quote in Big Data Will Drive the Industrial Internet

“A six hours flight from New York to LA on a twin-engine aircraft produces a 240-terabyte mountain of data,” writes Paul Mathai, applied research lead, Manufacturing & Hi-Tech, at Wipro. “The data can be analyzed to reveal every aspect of the engine’s performance and health.”

                                        ***********************

These data add up very quickly if you think about all the flights.   Are all these mountains of data valuable?

SpaceCurve CEO Slitz has an interesting perspective on public cloud:

SpaceCurve’s platform is designed to run on computer clusters with at least 1,000 nodes, and not in a public cloud because it takes too long to move datasets this large. “It’s like draining Lake Union into the Pacific with a garden hose,” Slitz says.

Comically simple bandwidth bottlenecks continue to be a hassle in big data analytics. 

Just read a nice quote in a press release from Beyond Analysis, a consulting firm:

“Analytical insights should be presented in a simple and striking fashion that will be meaningful to people across the organisation, enabling the kind of informed strategic decisions that will help retailers to thrive. To harness its true power, Big Data needs to be delivered in small bites and simple steps in order to produce truly powerful results.”

                             **********************************
Definitely concur that key business decision-makers, aka humans, can consume, digest, and internalize insights in tiny portions to help the development and/or refinement of business strategies, while machines consume big data and can support real-time granular decision-making in operations.