Bulk Row count comparison of homogenous data sources

You are viewing archived documentation for Data Trust v2024.06 (previous version).
Go to the latest v7.6 documentation →

Bulk Row count comparison of heterogenous data source

Using TDR, you can perform a row count of data comparison between multiple pairs of data sets. Count Compare compares the Row Count from the Source and Target Data sources. Let’s perform bulk row count compare using heterogenous source and target data sources.

To learn more, let us walk through the below procedure:

• Click the SCENARIO STUDIO module.
• Hover on the Scenario Builder snippet from the left pane options.
• Click the Technical Data Recon option.

• The user is navigated to a new TDR session page in create and edit mode.
• From the Data Source tab, let’s drag and drop an RDBMS Tables icon on to the TDR session page.
• Double tap the RDBMS Tables widget.

• An RDBMS Tables pop-up window is displayed.
• Let’s select Redshift from the Database Type drop-down list.

• Select Redshift-QA from the Connection Name drop-down list.

Note: As the user selects the Connection Name drop-down list, the Parallelism, Packet Size along with Server, Database, Port, and User Id are populated.

• Click the Select button.

• The RDBMS Table widget is closed saving the widget data and further collapsing into the TDR session page in create and edit mode.
• From the Data Source tab, drag and drop an RDBMS Tables widget as Target Data Source.
• Double tap on the RDBMS Tables widget.

• An RDBMS Tables pop-up window is displayed.
• Let’s select Database Type as PostgreSQL and Connection Name as Postgres_livewire_TPCDS_Read.
• Click the Select button.

• The RDBMS Tables widget is closed saving the data and collapsing into the TDR session page in create and edit mode.
• Click the Compare tab.

• Drag and drop the Count Compare icon onto the TDR session page in create and edit mode and double-tap on it.

• A Count Compare pop-up window is displayed.
• Click the Filter icon of the source database.

• A Filter slide screen with Schema / Owner, Object Name, and Max. No. of Values field options is displayed.
• Let’s, select pg_catalog from Schema / Owner drop-down list.
• Click the Get Metadata button.

• The selected Schema tables are populated.
• Click the Filter icon of the target data source.

• A Filter slide screen with Schema / Owner, Object Name, and Max. No. of Values field options is displayed.
• Let’s, select pg_catalog from Schema / Owner drop-down list.
• Click the Get Metadata button.

• The selected Schema tables are populated.
• Click the Propose icon.

• Mapping of similar fields for both the Connection Names (Redshift-QA and Redshift JDBC) are displayed.
• Click on Arrange Object Pairs By Mapping Order icon.

• This action arranges the object pairs by mapping order.
• Click the Save button.

• The Count Compare widget is closed saving the widget data and collapsing into the TDR session page in create and edit mode.
• Click the Additional Options tab.

• From the Additional Options tab, drag and drop the Email icon onto TDR create and edit page.
• Click the Email Settings widget.

• An Email Settings pop-up window is displayed.
• Click the Save button.

• The Email Settings widget is closed saving the data and collapsing into the TDR session page in create and edit mode.
• Click the Execute Now button.

• The user is navigated to a new tab displaying the Scenario Execution Summary results of that particular TDR Scenario.
• The Row Count is displayed under Compare Results tab.

• The TDR execution results are forwarded to the email address as shown here.