Data Quality in BI “as we go along”

In the last year I have been to several conferences and network meetings about DW and BI. I have heard several customer stories.  They all tell about the fantastic economic reporting system, and how easy it has become for the managers to access information.

When they are asked about customer data quality issues, they often say “we take that as
we go along” or “we will cross that bridge later”.  Their main reason is that they need to
get the financial reporting in place, and this takes all their human and financial resources.

Do they really think their reports will be correct if they base their calculation on bad data?  How do they “take it” down the road?

We know that professional data quality tools finds another 1-5 per cent new duplicates then the cleansing tool included in the BI-package. Let’s assume that it is 2 per cent on average. A couple of customer cases i have listened to have 2 million customer records. 2 per cent accounts to 40.000 customers! I would think this would influence those economic reports. Is it your best customers maybe?

Lack of trustworthy data can pull the rug under many BI and DW projects. Why is it these problems are not taken seriously.

Here are some previous postings on the topic:
Ten Mistakes to Avoid in Data Quality Management 
Survey shows BI Key Benefits are Intangible
“Without proper data, BI are meaningless”
Laissez-Faire: the biggest obstacle to improved Data Quality


Survey shows BI Key Benefits are Intangible

In the latest DM Review it is referred to a market survey that isconducted by Noetix Corp.  It shows that IT professionals consider the intangible benefits of BI solutions to be of more value than any tangible benifits, and therefore find it very difficult to measure the ROI of these applications.

For me this is another proof that even though we cannot decide the cost of poor data quality, it is important to keep the focus on it.  Most costs are hidden and intangible, but you can see results if you do something about it.

The results also show that it’s the quality of the data and information and not the quantity
quantity that is important for the decision makers.