Bulk Data comparison of homogenous data sources – key-based and selective fields

You are viewing documentation for Data Trust v7.6 (latest).
Looking for the previous version? View v2024.06 documentation →

Data Trust › Scenario Studio › Scenario Builder

Bulk Data Key-Based Comparison of Homogenous Sources

Compare homogenous datasets using key columns and selective field matching.

Overview

This guide walks through the Bulk Data Key-Based Comparison of Homogenous Sources scenario in Data Trust Scenario Studio. Follow the numbered steps below to configure, execute, and review the results of this scenario type.

Note: Ensure all required connection profiles are configured in the Administration module before building this scenario.

Prerequisites

• You are logged in to Data Trust with a Pro-User or Admin Pro-User role.
• The relevant connection profiles have been configured in the Administration module.

Steps

Path: Data Trust › Scenario Studio › Scenario Builder › Bulk Data Key-Based Comparison of Homogenous Sources

Figure 1: TDR Option

Navigate to the Scenario Studio module and select the Bulk Data Key-Based Comparison of Homogenous Sources option from the left sidebar or main menu. This opens the scenario builder canvas where you will configure the data source connections, transformation rules, and execution settings.

Figure 2: Drag and drop RDBMS Widget

Drag the Rdbms Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Figure 3: Source Connection Details

Select the appropriate connection from the dropdown to link this widget to its data source. Choosing the correct connection profile ensures the scenario reads from or writes to the intended database or file system.

Figure 4: Drag and drop Target RDBMS Widget

Drag the Target Rdbms Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Figure 5: Target Connection Details

Select the appropriate connection from the dropdown to link this widget to its data source. Choosing the correct connection profile ensures the scenario reads from or writes to the intended database or file system.

Figure 6: Click on Compare tab

This step involves clicking the Click on Compare tab within the Bulk Data Key-Based Comparison of Homogenous Sources workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 7: Drag and drop Data Compare widget

Drag the Data Compare Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Figure 8: Click on Source Filter Button

The Click on Source Filter Button panel allows you to define criteria that limit which records are included in the comparison. Filters reduce data volume and processing time by excluding records that are out of scope for this reconciliation or validation run.

Figure 9: Provide Soce connection database

Select the appropriate connection from the dropdown to link this widget to its data source. Choosing the correct connection profile ensures the scenario reads from or writes to the intended database or file system.

Figure 10: Source Tables

This step involves interacting with the Source Tables within the Bulk Data Key-Based Comparison of Homogenous Sources workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 11: Target Database

This step involves interacting with the Target Database within the Bulk Data Key-Based Comparison of Homogenous Sources workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 12: Click on Propose button

This step involves clicking the Click on Propose button within the Bulk Data Key-Based Comparison of Homogenous Sources workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 13: Arrange Object By Mapping Order

The Reconciliation widget maps source fields to target fields for comparison. Use Smart Mapping to automatically propose column matches, then manually adjust any fields that were incorrectly paired before saving.

Figure 14: Click on Key Columns button

Select the key column(s) that uniquely identify each record in both the source and target datasets. Accurate key selection is critical — an incorrect key will cause the reconciliation to produce inaccurate match results.

Figure 15: Select Key Cloumn

Select the key column(s) that uniquely identify each record in both the source and target datasets. Accurate key selection is critical — an incorrect key will cause the reconciliation to produce inaccurate match results.

Figure 16: Click on another table Key Cliumn

This step involves clicking the Click on another table Key Cliumn within the Bulk Data Key-Based Comparison of Homogenous Sources workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 17: Select Key Column and click on Save

Click the Save button to persist the current widget or scenario configuration. A confirmation indicator appears on the canvas once the save is successful, and the widget thumbnail updates to reflect the configured state.

Figure 18: Key Columns are highleted

Select the key column(s) that uniquely identify each record in both the source and target datasets. Accurate key selection is critical — an incorrect key will cause the reconciliation to produce inaccurate match results.

Figure 19: Click on Aditional Options Tab

Navigate to the Scenario Studio module and select the Bulk Data Key-Based Comparison of Homogenous Sources option from the left sidebar or main menu. This opens the scenario builder canvas where you will configure the data source connections, transformation rules, and execution settings.

Figure 20: Drag and drop Email widget

Drag the Email Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Figure 21: Configure Email

The Email widget sends an automated result notification to configured recipients after the scenario finishes executing. Provide the recipient addresses, subject, and choose whether to attach the exception report or include a summary in the email body.

Figure 22: Click on Execute Button

Click the Execute button to trigger the scenario and begin processing data from the configured sources. The scenario transitions to Monitor mode where you can track real-time execution progress and view interim results as each step completes.

Figure 23: Scenario Execution Summary Page

The Scenario Execution Summary Page page presents the full outcome of the scenario execution including record counts, match rates, and any detected exceptions. Drill into exception records to inspect individual mismatches and use the Export to CSV option to share results with stakeholders.

Figure 24: Recived Email

The Email widget sends an automated result notification to configured recipients after the scenario finishes executing. Provide the recipient addresses, subject, and choose whether to attach the exception report or include a summary in the email body.