Integrity Management
Data Quality Evaluation for Integrity Management Critical Information
DATA QUALITY MODEL
Credible integrity management decisions start with reliable data. While there are work-a-rounds for missing data, they generally involve conservative estimates that increase uncertainty. Erroneous data can be more damaging, as uncertainty may not be included, and the value may not be conservative enough. With potentially millions of records to check, reviewing data quality manually becomes an impossible task.
Integrity Plus offers a fit-for-purpose model that reviews key aspects of data quality:
Complete
Traceable
Correct
Recent
This model runs on New Century’s Spatial Risk Analyst (SRA) platform and it is available for purchase(1) or as a service. Spatial Risk Analyst connects directly to asset data sources (PODS, UPDM, GIS) and other compatible tabular and spatial sources to seamlessly integrate all information for simultaneous review. This powerful integration combined with user-defined rule sets allows for a more thorough data quality review than can be accomplished when looking at data elements in isolation.
Spatial Risk Analyst natively possesses features for evaluating missing data. Missing data can be identified and aggregated at many different levels to support company dashboards that show the percentage of missing data at the pipeline, system, or company level. SRA can produce tabular reports or spatial maps that consolidate ranges of missing data. This information can be provided to data owners to help close data gaps. For critical data that must be traceable, verifiable, and complete (TVC), SRA can also report on any TVC status stored in the source database.
While missing data creates many limitations, erroneous data is misleading. Without TVC sources, it can be difficult to verify whether data is correct. Our data quality model identifies potentially incorrect data by checking surrounding data for inconsistencies. Consistency checks are applied as rules that have been defined based on industry knowledge and in-house subject matter expertise. A few examples of simple rules include:
Wall thickness is valid given the pipe diameter
Long seam type is valid given the manufacturing year
Pipe coating is valid given the installation year
SMYS is valid for defined pipe grade
The complexity of rules that can be defined is limitless. The rules are completely transparent and configurable for continuous improvement. Over time, industry knowledge captured in these rules amounts to a diverse team of SMEs pouring over your data with each run of the model.
Some data is temporal, such as inspections. For temporal data, the recency of the data is important. Spatial Risk Analyst can apply rules to evaluate the freshness of the data based on code-required inspection frequency or operator-defined inspection schedules to identify stale information.
Data quality improvement efforts are most effective when clear metrics are presented and monitored. The data quality model can be executed and published frequently to drive company dashboards. If you don’t have an existing dashboard, let Integrity Plus help you design custom dashboards that bring metrics to life through vivid charts and maps.
SUPPORTING SERVICES
Our service team can also support the following activities to assist with your data quality program:
- Establishing Traceable, Verifiable, and Complete Records
- Positive Material Identification (PMI) (available through MISTRAS NDT)
- Prompt completion of data entry backlog, including centerline corrections and attribute data
- Data cleanup for missing or inaccurate data
- Database integrity “health check”
(1) The purchased model is subject to a license agreement and requires a licensed copy of Spatial Risk Analyst to run.
Some key features of the MAOP Validation Model include:
- Integrates data from multiple distributed data sources
- Processes both tabular and spatial data types
- Extendable rules that capture industry and SME knowledge
- Consolidate ranges of data quality issues for efficient communication
- Publish results to drive metrics on company dashboards
- Exports feature classes to identify issues on a map or dashboard
- Ease of use allows running frequently to monitor change and progress
Model requires Spatial Risk Analysis 3.0 or greater
Support data sources include:
- MS SQL Server, Oracle, MS Access, SQLite (tabular)
- File geodatabase, SDE, ArcRuntime, Feature Services (tabular or spatial)