The Importance of Defining Data Quality Metrics
In our eBook, The Clean Data Initiative, we explore the reasons why companies have dirty data and provide guidance to fix data quality issues that have been overlooked. The way a company defines their data quality metrics directly affects the consistency and accuracy of data. Without a well defined framework, companies will make decisions and investments using dirty data, which can lead to:
- Inventory Issues – slower turnover, stock write-downs
- Fulfillment issues – out-of-stocks, inability to deliver orders, inefficient sales promotions
- Logistics – rework and distribution issues, unnecessary deliveries, extra freight costs
There is a direct top-line and bottom-line impact due to dirty data. Every organization is working with different data sets and different data quality metrics. This naturally brings up the next question, how do companies consistently measure and report on the quality of their data?
The Data Quality Framework
There is no universal definition for clean data, meaning companies approach this differently, creating data quality definitions from scratch and developing new implementation methods. To create a framework, your data quality team needs to define the metrics they want to track by:
- Determining where dirty data has a negative business impact
- Defining the policy they wish to establish (e.g. minimize shipping costs)
- Identifying the data elements that must have quality data
- Defining what metric needs to be tracked for each data element
- Setting up acceptable thresholds for each metric
When creating metrics, the goal is to create the dimensional metric for each data element. For example, “Customer_LName_Completeness” may be a data quality metric created to ensure that every record has a Customer Last Name. The threshold for this data element may be 100%.
Resolving Data Quality Issues and Making it Count
You can now run an automated test against the data to ensure data issues are being resolved and that the data quality is approaching the threshold level. When encountering data issues, there are four possible action steps:
- Accept the record with the issue
- Reject the record for further resolution
- Fix the record based on defined business rules
- Plug in a default value to continue processing the data
With the information being tracked on a dashboard, it is possible to track the quality of the data over time and ensure that improvement is made. By implementing processes to clean your data, you remove inconsistencies and begin making decisions with quality data. Be precise, and you will obtain he business results you want.
Data Quality Next Steps
Getting to clean data requires recognizing the internal organization constraints that might be holding you back, understanding where to start on such a journey, determining the cost of a clean data effort, and developing across the various stakeholders. Our eBook, The Clean Data Initiative helps you on this journey, whether it is understanding the basics of Data Quality, or helping you choose a data quality tool.
Sense Corp is a data and digital consulting company transforms businesses for the digital era. With 20 years of experience, our team of experts provide actionable insights and effective solutions to you and your team.
To learn about what we do, visit our website or contact us.