You’re only as good as your data…

Big Data


Collecting big data is rarely the problem for business, but organising it and analysing it, to extract real value, is another matter entirely. In advance of the forthcoming itelligence conference in London, Asad Mahmood, Head of Database and Technology at itelligence says business should look to baseball for inspiration.

Have you ever heard of Bill James or Sabermetrics? James was, and is, a data revolutionary. At night he was a security guard in a factory, by day he used his love of data to analyse baseball players. What James found by analysing thousands of baseball stats was that the big professional baseball teams were getting it all wrong. They were analysing the wrong data and spending too much time on tired and dated stats, like batting averages, that ultimately misled them. What James did was develop new scoring metrics which uncovered hidden value in players, metrics such as On-Base Percentage, Runs Created and Defensive Runs Saved. Those metrics, when put into practice by baseball teams, such as the Oakland Athletics, enabled smaller teams to compete with the New York Yankees, despite having a fraction of their budgets.

The SAP Big Data Connection

What’s all this got to do with SAP? Well, baseball had many of the same problems that besets business, namely not a lack of data exactly, but too much of it and in the wrong place. Baseball, like a modern factory, lends itself to counting things and sheds off big data at an extraordinary rate. The problem therefore is not getting big data, the problem is making sense of it. Too often, organisations are siloed in their thinking, unable to analyse how one data set from the finance department is impacting production. Similarly, data is inaccurate, incomplete, inaccessible and inconsistent. What’s caused this? There are a number of culprits, including multiple systems being in use, often due to organisational mergers, through to poor input and validation measures. But one of the core issues that often needs to be addressed is the relationship between the business and IT in an organisation, a relationship that is rarely collaborative when it comes to big data issues.

Unwillingness to make use of big data

In IT departments it is rare to find a consistent way of measuring and scoring big data quality and often there is no clear strategy and discipline for improving that data quality either, or eliminating duplication. The wider business suffers from an inability to analyse data dependencies across systems and complains of a lack of visibility, as to where the numbers or data is coming from. Inevitably, this breakdown results in a lack of confidence in the figures and the veracity of the metrics, but the wider business is unable to make the necessary interventions to get it right. This lack of scrutiny means that basic questions like, “How has this metric been arrived at?” go unasked. Too often there is ambiguity in terms of terminology and definitions which result in an inability to compare different data sets. This chaos, and breakdown in the credibility of the information, inevitably leads to a low level of adoption or unwillingness to make use of big data. And lack of adoption, for business, as with the Major League baseball teams, leads to bad decision making.

SAP Information Steward provides insight into big data quality

Fortunately, unlike Bill James you don’t have to pore over thousands of pages of statistics in order to develop the key metrics by which to measure organisational performance. Nowadays, SAP provides the tools you need to understand and analyse the trustworthiness of your information. With integrated data profiling and metadata management functionality, SAP Information Steward can provide continuous insight into the quality of your data, giving you the power to improve the effectiveness of your operational, analytical, and governance initiatives. And what this will give you, ultimately, is competitive advantage. If you know ahead of your competitors what is going on, and you trust your data, you can make the adjustments which improve margins or customer service or shorten R&D development time.

When Sabermetrics first became known to a wider audience with the publication of Michael Lewis’ book, Moneyball, it was hated by the baseball fraternity who just wanted to carry on with things as they were and regarded it as a threat. Interestingly, however, it was loved by business who took the central message about data and coined the mantra, “If we can’t measure it, we don’t do it.”

The itelligence Conference 2016, ‘Making Digital Real’, was held at 155 Bishopsgate on 17th May 2016.

For further information and to register for the 2017 event please go to

Author: Asad Mahmood

Similar posts

Read more
Modern data platform
Read more
Read more
Read more
Read more
Read more

1 thought on “You’re only as good as your data…”

  1. Great post. I’m confronting a couple of these problems.

Comments are closed.


For any questions please feel free to contact us.