Looking for the previous version? View v2024.06 documentation →
Bulk Row Count Comparison of Heterogenous Data Sources
Overview
Bulk Row Count Comparison scenarios validate that the record counts in source and target tables match across multiple tables in a single scenario run. This is the fastest way to verify volume consistency between two systems.
Prerequisites
- You are logged in to Data Trust with a Pro-User or Admin Pro-User role.
- Source and target database connections have been configured.
Steps
Path: Data Trust › Scenario Studio › Scenario Builder › Bulk Row Count Comparison of Heterogenous Data Sources
Navigate to the Scenario Studio module and select the Bulk Row Count Comparison of Heterogenous Data Sources option from the left sidebar or main menu. This opens the scenario builder canvas where you will configure the data source connections, transformation rules, and execution settings.
Drag the Rdbms Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.
The Navigator panel displays the available databases, schemas, and tables for the selected connection. Expand the tree to locate the required table or view, then click Select to add it to the widget configuration.
Select the appropriate connection from the dropdown to link this widget to its data source. Choosing the correct connection profile ensures the scenario reads from or writes to the intended database or file system.
Drag the Target Rdbms Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.
Select the appropriate connection from the dropdown to link this widget to its data source. Choosing the correct connection profile ensures the scenario reads from or writes to the intended database or file system.
This step involves clicking the Click on Compare Tab within the Bulk Row Count Comparison of Heterogenous Data Sources workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.
Drag the Count Compare Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.
The Open Compare widget and click on Filter button panel allows you to define criteria that limit which records are included in the comparison. Filters reduce data volume and processing time by excluding records that are out of scope for this reconciliation or validation run.
Click Get Metadata to automatically retrieve the column structure and data types from the connected data source. Review the retrieved metadata and deselect any columns that are not relevant before proceeding to configure validation rules.
Select the appropriate connection from the dropdown to link this widget to its data source. Choosing the correct connection profile ensures the scenario reads from or writes to the intended database or file system.
The Reconciliation widget maps source fields to target fields for comparison. Use Smart Mapping to automatically propose column matches, then manually adjust any fields that were incorrectly paired before saving.
The Reconciliation widget maps source fields to target fields for comparison. Use Smart Mapping to automatically propose column matches, then manually adjust any fields that were incorrectly paired before saving.
The Reconciliation widget maps source fields to target fields for comparison. Use Smart Mapping to automatically propose column matches, then manually adjust any fields that were incorrectly paired before saving.
Navigate to the Scenario Studio module and select the Bulk Row Count Comparison of Heterogenous Data Sources option from the left sidebar or main menu. This opens the scenario builder canvas where you will configure the data source connections, transformation rules, and execution settings.
Drag the Email Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.
The Email widget sends an automated result notification to configured recipients after the scenario finishes executing. Provide the recipient addresses, subject, and choose whether to attach the exception report or include a summary in the email body.
Click the Execute button to trigger the scenario and begin processing data from the configured sources. The scenario transitions to Monitor mode where you can track real-time execution progress and view interim results as each step completes.
The Scenario Execution Summary Page page presents the full outcome of the scenario execution including record counts, match rates, and any detected exceptions. Drill into exception records to inspect individual mismatches and use the Export to CSV option to share results with stakeholders.
The Email widget sends an automated result notification to configured recipients after the scenario finishes executing. Provide the recipient addresses, subject, and choose whether to attach the exception report or include a summary in the email body.