What’s Driving your Big Data Transformation Priorities?

What’s more important? Big Data infrastructure including cloud computing, storage, Hadoop and other data processing engines? Is it the algorithms used to automate and derive intelligence from the data? Or is it the talent of the data scientists, the visualizations that they develop, the stories that they tell, and the questions that they answer?

If you believe the technology companies marketing Big Data solutions or a large number of technology media journalists they will  focus on Big Data infrastructure. Organizations that already have a strong practice in analytics and reporting are more likely to focus on improving the talent, tools, and capabilities of their data scientists. Companies that already have a strong software practice and develop algorithms for other strategic needs are more likely to pursue machine learning and other data mining algorithms.

What’s Driving your Big Data Transformation Priorities?

From my perspective, it appears that organizational strengths and dynamics are driving the priorities in Big Data capability investments. This isn’t necessarily a bad thing and probably smart for organizations by investing in their strengths. But it isn’t sufficient and at some point, a balanced set of investments and changes over a longer period of activity will likely be required.

In other words, while leveraging your organizational strengths may be an easy on ramp to Big Data insights, it probably isn’t a complete holistic approach. At some point, many organizations that truly want to become “data driven” must transform or invest in new capabilities.

So in thinking about this transformation, here are some simple questions and guidelines on where to focus efforts

• How big is your data? – The bigger the data, the more likely you will need to look at infrastructure to store, process, and manage larger data sets.
• How fast do you need results? – The more likely your business derives value in presenting results faster for direct revenue, real time decision making or competitive advantage, the more likely you will need algorithms to directly churn data to drive other systems or decisions. Examples include Hadoop and other machine learning APIs.
• How complex is your data? – Complexity comes in many forms. It may be unstructured data, data that has rich relational metadata requiring subject matter expertise, or data that’s sparse and has other quality issues. In these situations, organizations will likely look to its subject matter experts and ideally data scientists to help provide insights, but they need to work differently. Data scientists are not glorified spreadsheet jockeys and produce different results than statisticians or BI analysts.