Avoiding the Abyss of Big Data

Last week, we defined the critical differences between “data”, “information” and “insight”, and what it truly means to be data-driven in today’s digitized world. If you missed it, check it out before delving deeper into what today’s post is about: Why so many organizations fail at connecting analytics to action, and how to better equip your team for success when making the ever-critical transition to big data.

By now, the business benefits of big data have been hammered in. Now, it’s time to demystify big data by talking about its challenges and re-examining current frameworks and business processes. 

Breaking down the big data problem

So why are 71% of those attempting the analytical leap failing to get real results?

Part of it can be explained by where we are in the timeline of the data evolution: the formative years, Big Data 1.0 . Much like the early days of the Internet, there are growing pains associated with navigating through technologically-uncharted waters. We may understand the transformative potential of big data, but our tools and skill sets pre-date the new analytical approach. 

Shortcomings of traditional tools

Too many businesses are hindered by legacy systems, outdated applications, and decades-old storage methods. Attempting to work with big data on a spreadsheet is as inefficient as attempting to fix a car with a single screwdriver.

Today’s outsourced, multi-tiered supply chains require a new system with far-ranging data inputs and the capacity to process it. Excel and ERP systems lack the responsiveness as well as the algorithms to weigh various KPIs across departments, let alone between business partners or the global market.

Fortunately, there now exists a new generation of analytics software that is built entirely around managing complex supply chains through connected data. Many of these solutions are functionally a huge upgrade with the adoption of cloud computing, which makes storage inexpensive and easier to scale.

In the world of big data, there is no place for separate silos of information. Companies aiming to quickly climb through this learning curve should acquire a new tool set that is not only robust enough to handle the volume of data, but flexible enough to enable collaboration of key data sets across and beyond the enterprise. 

… But tools aren’t everything

Since tools are intended for use by humans, how they are utilized plays an important role in the outcome of data. A common mistake that stunts an organization’s ability to extract value from their analytics is where the decision maker positions themselves in the process. Often times, they use data to support their own decisions rather than letting it illuminate an unbiased idea, direction, or conclusion. Valuable insight comes from letting the data speak for itself.

Another common impulse for companies working with data for the first time is to spot the statistical pattern and then go looking for a problem that it solves. Instead, the approach should start with the problem and map the problem to the data. If the problem is shared across multiple organizational units, combine and analyze different data sets to discover context-rich insights.

If you don’t have a solid data strategy in place, start by asking questions: What would happen if we raise the price of the product by $1? When would be the best time to release a new feature? What were the quantifiable effects of changing a distributor? Some questions may be too complex to map out depending on how far along your company is in its data adoption.

Success in data relies on a cultural shift  

All the data in the world doesn’t equal data-driven productivity. Just like any major technological jumps in history, big data comes with a societal dimension: It requires us to re-think the traditional top-down hierarchy of an organization, how we evaluate business performance, and what it means for our occupational roles and responsibilities. It requires early champions, leadership support, active participation, and a willingness to change in order to succeed.

Former Chief Scientist at Amazon Andreas Weigend lays it out simply with his nine rules for big data. Of the list, more than half are about adjusting the culture around how we view data:

  1. Collect everything
  2. Start with the problem, not with the data
  3. Share data to get data
  4. Base the equation of your business on customer-centric metrics
  5. Drop irrelevant constraints
  6. Embrace transparency
  7. Make it trivially easy for people to connect, contribute, and collaborate
  8. Let people do what people are good at, and computers do what computers are good at
  9. Thou shalt not blame technology for barriers of institutions and society

What are your biggest challenges around managing big data? What are your best practices? Let us know in the comments section below. 

1 comment on “Avoiding the Abyss of Big Data

  1. ThomasMaloney
    September 20, 2018

    Regardless of whether you are an individual or a business owner, data is important to you in so many different ways. There is so much that you can extract from just a single set of data alone. From finding out how much you have spent to knowing whether or not your customers are satisfied with your services, everything lies in a set of data.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.