Imagine your friend ran a large store and you asked him how he was doing. Imagine he responded by telling you how many products filled the shelves, the different types of products he sold, and how often he refreshed his inventory. You’re probably looking for some indication on revenue, sales, or customers and yet he’s telling you information on inventory. Puzzling.
Yet, that’s exactly what we’re doing as technologists in discussing volume, velocity, and variety of our Big Data initiatives. Do we have to measure Big Data initiatives by profit, revenue, or customer impact?
The answer is yes. Eventually. Ecommerce companies and others that can measure response directly based on clicks may have the easiest, direct approach to measuring impacts. But for others, it may not be so simple and it might take some experimentation to yield the proper metrics.
Developing Metrics on Analytics Delivered
So a good start might be to define metrics that help organizations determine how effective their big data initiatives are in performing analysis and delivering results. Here is one approach to get started:
-
-
Develop a Big Data Analysis Catalog – This catalog, would define some of the recurring analysis performed by someone (possibly a data scientist) in the organization. Below is
a simple catalog.
-
-
- Document the type of Analysis Performed – For each Data Analysis, develop some light weight documentation on the type of the analysis performed. You can do this in an unstructured way with a document or wiki page, or more structured by defining a child table in the Analysis Catalog. In the pipeline example above, this could list out specific reports and how the data is analyzed for trends or anomalies.
-
- Log your activity and results – Put the “science” discipline into the role of the data scientists and require them to maintain a lab book. For each analysis performed whether it’s reviewing a report or dashboard or performing an entire study, make sure there is a log of this activity and at least a qualification of the results.
- Value and trend the results – At the time of the analysis activity, it might be difficult to put a quantified, ideally monetary value on the results. So I would suggest a quarterly review of the analysis log to determine what activities provided value and to attempt to put a dollar value against them.
Why the need for Metrics?
Recall that Big Data initiatives – although largely marketed by the Media as infrastructure investments – need to become organizational change successes in order to deliver value. Executives and managers need to leverage the data and analytics in decisions and actions, but this requires some persuasion. Metrics are one way to drive this behavior. As value is delivered to one part of the organization, the results can be quantified and marketed to the next group.
Leave a Reply