Using Your Data with Confidence
Unhappy customers, fragmented business processes and siloed data sources are all driving up the cost of doing business within the Utilities sector. In fact, poor data quality accounts for 20% of business process costs.*
Data quality affects labor productivity by as much as 20% and operational expenses by 20-30%. Nearly 60% of organizations don’t measure the financial cost of poor data quality, according to a Gartner survey.
Although there may be challenges at capturing issues at the source by using a data quality firewall or other means, it is still a better option from both cost and ease of discovery (the opportunity for fixing at the enter level versus the aggregated data, where the impact on metrics can be significant), than trying to retroactively fix them downstream where the impact of the bad data has filtered through to many reporting systems and users.
How does Data Quality Impact Business?
We need to fix the issues at the source to minimize cost to the business and the impact on downstream business processes, performance, and reporting. If the data is inaccurate at the source level, inaccuracies will be introduced in all reports at all levels downstream. Report data won’t match, insights will be inaccurate and, worst of all, poor business decisions may be made based on the incorrect information.
Siloed Data Systems
Downstream collection of data in disparate data sources, such as a call center, billing, or new customer connection is probably creating a number of data quality issues. Only 33% of companies are confident in their data quality**
Ever been in the position of talking to customer service about a bill you received to be told they can’t find you in the system? Differently defined rules or no rules at all causes customer information for a single customer appears to be unrelated. It may be as simple as a misspelled address or last name, or abbreviated words – Street versus St. for example. There may not be much that can be done about a simple misspelling of a name, but we can control abbreviations and standard words by using predefined lists or address selection based on zip code. This eliminates some of the confusion when creating a view of your customer and improving your customer’s perception of their importance to your company.
Bringing these systems together into one holistic view is one very important step in your BI path.
Setting up processes and rules to avoid issues in siloed data sources in the first place should be put in place. Reducing free-form data entry where possible and having the same selection choices to help regulate the data entry can be a huge first step. For example, GA, Ga. and Georgia are all the same to us but are not considered the same when computationally comparing an address. Identifying a cross-functional team to plan and determine the rules is essential for success. The data needs to conform, but it also needs to make sense to those who capture and use it. Consider not only the quality of the data but also the intended consumers of that data. Use inline analytics to enrich and reduce errors during data entry, for example, consider a drop-down instead of allowing free-form entry. This would guarantee consistency. Keep your eye on the goal. Ultimately, you are doing this to save money and enhance customer analysis, customer communication and relationships.
Implementing a Data Quality Process
First, you need to identify the causes of bad data quality and define processes to improve those issues. This will have been completed by your governance team. Causes of data quality issues will differ between companies, but areas of consideration could be:
- Training of data collection personnel.
- Experience of data collection personnel.
- Business Units with different quality expectations.
- The perceived importance of what is collected.
- Upstream process consideration and use of data.
- Unaware of the need to consolidate data for downstream reporting.
Defining processes to prevent issues with future data collection will take a cross-functional team, as the impact may be widespread. You need to make sure there is buy-in and understanding from all parties to make sure any changes are adopted and implemented.
Now comes the implementation. Looking at the data, common profiling areas for review are:
- Look for records that are candidates for merging. (Like duplicates with similar names, i.e. Robert Jones and Bob Jones)
- Look for records with incomplete data (missing state, etc.)
- Look for records with inconsistent data. For example, two records with the same name and address, but different contact numbers.
Consolidate and report on your findings. Can we auto correct records or do we need to highlight those that can’t for further investigation or manual fix by the data quality team?
Once we have a clear picture (profile) of the data, we can implement rules in Information Steward. We can create rules and standards that can be applied to your datasets and will identify records that need review, as well as automatically fix records if the change is a simple rule (see the GA, Ga., Georgia example above).
We can use the data visualization in Information Steward to display results in data quality scorecards. This enriches your data for user consumption and reduces costs by eliminating redundant data.
Over time, new data collection may be added, so the need for continuous assessment and governance is essential for ongoing quality data. Periodically checking and updating the governance rules and policies, and ensuring any changes are implemented and monitored, must be part of the data quality process. Assigning resources and time to keep the standards high must be considered part of ongoing data stewardship.
What is the cost of not doing having a data quality strategy? Well, 56% of CEOs are concerned with their companies’ data quality.*** Maintaining a timely audit of all data saves money in downstream maintenance, time savings for business and IT with one point of data governance, rule enforcement and execution. It also provides the users of the data with the confidence they need to operate in their decision-making roles.
**The cost of poor data quality, Gartner).
***KPMG 2017 CEO Global Outlook
If this blog didn’t quite answer your question or you have other Business Intelligence questions or concerns, please feel free to contact us.