With the hot weather in the US northeast this weekend, I was able to catch up on some reading and research around big data, data science, and agile. In thinking about the culture required to leverage big data capabilities and data science programs, I found some interesting data worth sharing.The first comes from Computing Big Data 2016 –

This is showing that the collaboration between technologists, data scientists, and business leaders is a key success factor to make data scientists thrive in an organization. It also helps to have senior backing and either a strategy or set of priorities identified. This isn’t exactly surprising, but to address the collaboration required I have suggested using agile practices to perform discovery work and aligning data science and IT responsibilities in data science programs.

An Accenture Report on The Team Solution to the Data Scientist Shortage also accents agile practices as a method for growing and retaining data science talent

In addition, the time-proven wisdom about managing teams bears repeating: Data scientist teams, like others, flourish best when there is effective leadership, a strong mandate from above and clear goals. They require a path for taking projects from design through implementation. Like many projects in the IT world, they benefit from working in rapid, iterative sprints of preparation, analysis and review.

If agile is the answer to enable collaboration, then what is the problem or challenge to getting senior membership buy in? McKinsey articulates this well in their report Getting Big Impact from Big Data

Management teams frequently don’t see enough immediate financial impact to justify additional investments. Frontline managers lack understanding and confidence in the analytics and hesitate to employ it. Existing organizational processes are unable to accommodate advancements in analytics and automation, often because protocols for decision making require multiple levels of approval.

Agile Iterative Analytics Drives Buy In

Bottom line is that big data capabilities and data science programs take time to mature, but they don’t necessarily require extensive efforts to provide business value. These teams have to demonstrate quick wins to senior leaders so that they don’t lose interest in the program. Data science teams also have to take on some responsibility to help frontline managers to leverage the analytics and become more data driven.

If you want big impact from data science and big data, then think of demonstrating wins incrementally. Agile software teams perform frequent releases to drive incremental impact, capture feedback, and promote the next set of priorities. Data science programs should adopt similar practices.