Data monitoring & governance is now seen as a priority in nearly all enterprises. Data-related in production data environments have grown to a point where they must be addressed. Yet data monitoring & governance is still relatively immature and not defined precisely. Efforts have often focused on establishing councils or committees that set up procedures to access or use the data. Executive management appears to find such approaches bureaucratic and unable to address the underlying data problems.
A significant area for improvement is data quality, and executive management sponsor data monitoring & governance in the hope that it will address data quality. However, data quality is itself poorly defined and usually a collection of issues that unlikely has one common fix.
The whole approach of first trying to identify types of data quality issues and then figuring out ways to remediate them is questioned here. It is proposed that monitoring (detecting exceptional events) and metering (gathering metrics on the health of the data) is a logical precursor to fixing any problems
Data quality issue detection and prevention is a continuous process. There are many people, processes and systems involved in a data centric system. For optimal results they all have to be orchestrated in a well-defined workflow created using the data monitoring & governance framework.
- Work with business to identify the key data points to be monitored & governed
- Implement the data monitoring & governance framework around it
- Automate the workflow to proactively identify issues and send alerts to appropriate users