“The Big Data boom has largely been fueled by a simple calculation: Data + Technology = Actionable Insights, Magic Ponies, and Superpowers.” — Matt Asay, ReadWrite
We’re living in a “go big or go home” culture. But when it comes to using data analytics to drive your business, taking a big data or bust approach is a mistake. A costly one.
Most successful data-driven companies don’t start with big data –> they’ve typically already optimized their operations, product experiences and marketing by gleaning insights from more traditional data and analysis techniques. And they have the people with experience using data to make decisions — people who crave more of it — long before big data comes into play. Big data is a long-term evolution, not an overnight revelation.
Which brings me to why I’m writing this post: Most companies are sitting on a gold mine of value in their not-so-big data. And they don’t need to spend big bucks on big data technology and people to harness that value.
Big data’s peer pressure problem
“Almost eight in ten users (79 percent) agree that ‘companies that do not embrace big data will lose their competitive position and may even face extinction.” — ‘Big Success With Big Data,’ Accenture
I’ve seen this many times before: A technology buzzword gathers so much velocity that even great companies and the smartest people are almost powerless to resist. Especially when studies like the Accenture report cited above suggest that companies must invest or “face extinction.” Strong words.
This can lead to serious peer pressure in the boardroom — which in turn causes some companies to over-invest early in a technology’s lifecycle, and before the company is ready for it. Worse, executives sometimes aren’t clear on the problem they are trying to solve, or how the technology will create a meaningful ROI.
“Despite the buzz around big data, most companies will see higher returns more quickly in their not-so-big data. Data they can use to make better decisions right now.”
Like many tech buzzwords before it, the definition of big data has become ambiguous over time because the term is overused and misused. Applied big data is, in reality, much more narrow than many people think.
So let me try to define it. Big data is the aggregation of large masses of information that are often disparate, combined with machine learning and algorithms that parse the data, identify patterns, and predict outcomes. For many businesses, big data may include a wide variety of customer information. But it also could include macro data that’s indirectly related yet still influences that business, like the weather, traffic or economic markets. Anything that could have any relation to the problem should be considered.
The point is, big data really does mean BIG. That means it’s suitable mostly for companies with both a diversity of rich data sources and the capability of harvesting that data to provide unique experiences that it couldn’t have otherwise.
Big data technology is most often applied to drive consumer applications that couldn’t be achieved otherwise — e.g. Netflix uses it to suggest content to us based on behavior patterns from millions of other similar users; similarly, Amazon makes recommendations to us based on our past purchases. Using big data over the course of the millions of transactions may optimize sales by less than a percentage point. But because of the sheer volume of transactions, that can make a big difference to companies like Amazon.
Most importantly, to be successful with big data, companies need to have a high level of data maturity — meaning they’ve been using analytics regularly as a source of business intelligence. They should have the people (including “data scientists”) in place to manage and leverage that big data investment. And they need to be savvy about the wide variety of factors that might impact the behavior they’re trying to predict. Without this maturity and people infrastructure, an investment in big data is likely to fail.
What not-so-big data looks like
“Organizations should start with the basics, and work up from there. Instead of being lured by the ‘shiny object’ syndrome and thinking you need a big Hadoop data lake or neural networks to solve a problem, seek the simplest answer.” — Alex Wood, Datanami
“Many data projects don’t deliver a clear return on investment. In some cases, there wasn’t enough up-front strategy. In others, an organization over-invested in more advanced technology than they really needed.”
The vast majority of companies today either don’t have enough data or the analytics maturity/skillset to justify a big investment in big data. BUT, even for those that do, there’s probably a lot more ROI to be harnessed out of their not-so-big data. Especially when it comes to their operations — the very processes they use to manage their business on a daily basis.
I have a bias here: I’ve built a business that helps companies put their data to good use by making it more actionable. We typically focus on operations — building tools that automate reports and save employees valuable time; or creating custom views of data using visualization tools and algorithms that give team members and executives real-time insights to make better decisions. Building big data solutions for many of these daily operations problems would be impractical and expensive for most of my clients. But not-so-big data gets the job done. Here are a few examples:
- A director in an analytics group at a Fortune 500 retailer wanted a better view of her product line’s inventory levels so that she could make better and quicker decisions on pricing and promotion. So, we built a dynamic inventory dashboard that flagged problem products in real time — e.g. where sales were lower than expected and inventory costs were growing. Part of the ROI was that the dashboard spared hours of her time running manual reports. Now she can spend her valuable time on solving the problem rather than pulling all of the data together and identifying the exceptions.
- A consumer packaged goods client needed an intelligent tool that would recommend the most cost effective and timely way to produce goods to meet channel demand. So, we established a system of record for team members to input various supplier data then created a tool powered by linear optimization that generated the optimal production plan.
- Executives at another company wanted to understand which products were more profitable to build and sell. We added production run-time data to their model — i.e. how long it took to manufacture something for their most constrained resources — in addition to materials cost and sale price. Now their team members can analyze margin flowing through their most constrained resources to optimize manufacturing plans.
Many of my clients are grappling with how big data will shape their future — and they should. Some of them have enough consumer data — and the opportunity to improve customer experiences, along with optimizing their sales — to justify a big data investment. But it’s very much a long-term play for them. In the meantime, they are harnessing their not-so-big data to make better business decisions today.
Big or not so big?
”[CIOs] are spending more than ever on technologies that support data science, with worldwide revenues for big data and business analytics expected to reach $150.8 billion this year, according to IDC. But there’s a dark side to this delirious spending: Most data analytics projects fail to yield measurable value.” — Clint Boulton, CIO Magazine
When investing in any new technology, a company should have a clear understanding of the specific business problem it’s trying to solve. In my opinion, many business leaders become so enamored with the allure of big data that they don’t have a focused plan for how they’ll use it — nor have they defined the return they expect to receive from that investment. If they instead focused on a business problem, they may find out they don’t need a big data solution to solve it.
A company should vet data projects the same way it evaluates the cost of building a consumer product, defining the problem it is trying to solve by answering questions like:
- What is the expected outcome?
- How will I use it to make business decisions?
- How will I measure its success?
- What is the expected return on investment — and over which time period?
As the above excerpt from CIO Magazine suggests, many data projects (big and small) don’t deliver a clear return on investment. In some cases, that’s because there wasn’t enough up-front strategy. In others, an organization over-invested in more advanced technology than it really needed. Or, it simply didn’t understand that big data isn’t necessarily something you build and see a return on investment in the short term.
Despite the buzz around big data, most companies will see higher returns more quickly in their not-so-big data. Data they can use to make better decisions right now.
When it comes to data, bigger isn’t necessarily better. At least, not if you want a return on your investment today.
Trying to make better use of your data? Not sure if you can benefit from big data? Email us – we’d be happy to talk and offer up some advice.
