FINBOURNE's data quality checks (DQ checks) help you maintain high-quality data in LUSID by enabling you to evaluate, quantify, and validate data that has already been loaded into the platform. This brings several business benefits including:
Improved process accuracy
Reduced manual checks
Faster identification and resolution of data issues
DQ checks allow you to define validation rules for your LUSID data and run those checks as part of automated workflows. You can also run checks on-demand via the LUSID API (primarily for testing purposes).
When checks identify issues, they generate detailed results that you can integrate into your data management processes.
Note
Currently, DQ checks support instrument data validation. Future releases will expand support to other types of LUSID data such as transactions.
How do DQ checks work in LUSID?
DQ checks consist of the following components:
Create check definitions: Define what to validate using rules and rulesets
Run: Execute checks within workflows or via the LUSID API
Results: Review and act on four types of validation results
Currently, check definitions are created via the LUSID API. In future, you will be able to manage check definitions via the web app.
Common use cases
Post-load verification: Automatically check data quality after each LUSID integration run. For example, run a check on the instruments that were created or updated in the latest integration run.
Automated exception management: Create exception tasks in the Workflow Service for data quality issues, routing them to the appropriate teams for resolution. Each exception task persists as a record for auditing purposes.
Getting started
To begin using DQ checks:
Contact us to enable the feature in your LUSID domain.
Identify what validation rules are important for your data quality requirements.
Read Understanding Data Quality Checks to learn the concepts.