Bulk Data comparison of homogenous data sources – key-based and selective fields

You are viewing archived documentation for Data Trust v2024.06 (previous version).
Go to the latest v7.6 documentation →

Bulk Data based comparison of homogenous data sources-row based

Using TDR, users can perform a row count of data comparison between multiple pairs of data sets. Data Compare compares the Data from both the Source and Target profiles. Let’s perform bulk data comparison using homogenous source and target data sources.

To learn more, let us walk through the below procedure:

• Click the SCENARIO STUDIO module.
• Hover on the Scenario Builder snippet from the left pane options.
• Click the Technical Data Recon option.

• The user is navigated to a new TDR session page in create and edit mode.
• From the Data Source tab, let’s drag and drop an RDBMS Tables icon on to the TDR session page.
• Double tap the RDBMS Tables widget.

• An RDBMS Tables pop-up window is displayed.
• Let’s select Redshift from the Database Type drop-down list.
• Select Redshift-QA from the Connection Name drop-down list.

Note: As the user selects the Connection Name drop-down list, the Parallelism, Packet Size along with Server, Database, Port, and User Id are populated.

• Click the Select button.

• The RDBMS Table widget is closed saving the widget data and further collapsing into the TDR session page in create and edit mode.
• From the Data Source tab, drag and drop an RDBMS Tables widget as Target Data Source.
• Double tap on the RDBMS Tables widget.

• An RDBMS Tables pop-up window is displayed.
• Let’s suppose, select Database Type as Redshift and Connection Name as Redshift-JDBC.
• Click the Select button.

• The RDBMS Tables widget is closed saving the data and collapsing into the TDR session page in create and edit mode.
• Click the Compare tab.

• Drag and drop the Data Compare icon onto the TDR session page in create and edit mode and double tap on it.

• A Data Compare pop-up window is displayed.
• Click the Filter icon of the source database.

• A Filter slide screen with Schema / Owner, Object Name, and Max. No. of Values field options is displayed.
• Let’s, select pg_catalog from Schema / Owner drop-down list.
• Click the Get Metadata button.

• The selected Schema tables are populated.
• Click the Filter icon of the target data source.

• A Filter slide screen with Schema / Owner, Object Name, and Max. No. of Values field options are displayed.
• Let’s, select pg_catalog from Schema / Owner drop-down list.
• Click the Get Metadata button.

• The selected Schema tables are populated.
• Click the Propose icon.

• Mapping of similar fields for both the Connection Names (Redshift-QA and Redshift JDBC) are displayed.
• Click on Arrange Object Pairs By Mapping Order icon.

• This action arranges the object pairs by mapping order.
• Click the Save button.

• The Data Compare widget is closed saving the widget data and collapsing into the TDR session page in edit mode.
• Click the Additional Options tab.

• From the Additional Options tab, drag and drop the Email icon onto TDR create and edit page.
• Click the Email Settings widget.

• An Email Settings pop-up window is displayed.
• Click the Save button.

• The Email Settings widget is closed saving the data and collapsing into the TDR session page in create and edit mode.
• Click the Execute Now button.

• The user is navigated to a new tab displaying the Scenario Execution Summary results of that particular TDR Scenario.
• The Row Count is displayed under Compare Results tab.

• The TDR execution results are forwarded to the email address as shown here.