Big data: Buzzword for some, reality for others

I'm looking forward to my talk at WebExpo 2012 in Prague. Among many other things, I will talk about famous Peter Harris of Adpac. Peter Harris had his own personal Silicon Valley before there was anybody at The Peninsula (the story how he found his VC is legendary), and he's an icon for many of us at GoodData as we shared an office with him in 2010.

You might wonder what the connection between Mr Harris and big data is. While doing my research for the talk, I found a copy of Computerworld from 1981. One article is about Peter Harris; it starts with the following amazing line:

PORTLAND, Ore. – “Structured programming,” one of the software buzzwords for the 1980s, has been a reality for more than a decade in the data processing offices of the Georgia-Pacific Corp. (GP) headquarters here.

Mr Harris is the one who invented the term structured programming. That was in sixties. In 1981, he explains in a major  IT periodical that GO TO command is bad.

For me, that's insane. And it shows something unbelievable: What we see today as an important and doubtless trend, was actually a war of many battles. Many of them, apparently, belittling structured programming as yet another buzzword.

This is exactly where we are with big data. We feel the urge of discussing the buzzwordness of big data and we predict it will all settle down. Well, not all of us. Some have started to climb the ladder.

For many, the first rung is to realize big data is all around us. Next rungs are about collecting it and analyzing it.

I believe your first step into the world of big data should be different. First you should understand big data influences your business, and–if you're smart–big data can drive your business.

To analyze big data is futile if you don't take any action. That's why credit card companies work with big data for years without even calling it big data. Yes, the volume is epic but that's just one feature of big data (and definitely not the crucial one).

Dust is swirling, visibility is low. You can wait and read more and more definitions of big data. Or, you can be Peter Harris and do what it takes. His story shows the stakes can be incredibly high.

The best dashboard ever

There's hunger for dashboards outside. You can buy books about creating dashboards, you can hire consultants to create dashboards, you can spend a lot of time and a lot of money hunting your dashboard dream. The truth is, the best dashboard is very simple.

The only dashboard that anyone would ever need has three pieces.

At the top, there's a line chart, and the line goes always up. There's no explanation of axis or whatever (you can guess time is running to the right and money is running to the top but who knows and who cares). The only goal of the chart is to make you happy. The line's going up, hurray!

Then there are traffic lights on the left. The light tells you how you're doing. Is it green? Perfect. Is it yellow? Do something. Is it red? It's too late to do anything (however the line's going up, so you have enough time to pack your box and quit the job in a decent way).

Finally, there a text box telling you what to do. If the light is green, it's telling you what to do to keep it green. If the light is yellow, it's telling you what to do to make it green again. (If the light is red, don't waste your time reading the text box, just go, go!)

Now there's more than a joke in this dashboard. A friend of mine who's running a very small business has recently told me that BI tools are useless for her. "I don't need to slice and dice my sales, I don't need to measure how good my campaigns are," she has complained. "I just need to know what to do next."

Her point was clear: It's good to know your sales are decreasing for the last 6 months. And it's better to know that it's all because 85% of your existing customers in South Africa have declined your renewal package. Maybe you knew it without BI, maybe not. However even if it's a new fact for you, it's not telling you how to deal with it, how to fix it.

Davenport, Harris, and Morison are describing it in their recent book Analytics At Work as a shift from information to insight. Fully automated BI tools can help you with information: what happened, what is happening now, and what will happen.

To understand how and why did it happen, what's the next best action, and what's the best/worst that can happen, you need something more. You need people who can make the shift: understand information, and get insight out of it.

These people will create the dashboard discussed above, and it will be the best dashboard ever.

Reaching out for good data

When do you talk about good data? Google returns sentences like these:

• "We don't have really good data..."
• "Once we have good data..."
• "It's hard to make good policy decisions when they're not grounded in good data."
• "If you have good data..."
  
• "Do you have good data to validate your opinion?"
  

These are not positive statements but there's a hope the world will be better (once, if).

There is a difference between good data (data quality) and Good Data BI platform. However, I cannot resist to convert the sentences:

• "We don't have really Good Data..."
• "Once we have Good Data..."
• "It's hard to make good policy decisions when they're not grounded in Good Data."
• "If you have Good Data..."
  
• "Do you have Good Data to validate your opinion?"
  

It makes sense, doesn't it? Well it's not enough to have good data, you need a good analytical tool too.