You are viewing documentation for Data Trust v7.6 (latest).
Looking for the previous version? View v2024.06 documentation →

Functional Data Reconciliation (FDR)

Build and execute a full field-level reconciliation scenario between source and target datasets.

Overview

The Functional Data Reconciliation (FDR) scenario compares source and target datasets at a field level to identify mismatches, missing records, and volume differences. The scenario builder uses a drag-and-drop canvas where you connect data sources, define field mappings, apply transformations, and configure execution and email settings.

Note: Ensure all required connection profiles are configured in the Administration module before building this scenario.

Prerequisites

You are logged in to Data Trust with a Pro-User or Admin Pro-User role.
At least one source and one target connection profile has been configured in the Administration module.

Steps

Path: Data Trust › Scenario Studio › Scenario Builder › Functional Data Reconciliation (FDR)

Scenario STudio

Figure 1: Scenario STudio

The Functional Data Reconciliation (FDR) create page is the starting point for building a new scenario. Provide a name for the scenario, choose the appropriate folder, and then proceed to configure data sources and execution logic on the canvas.

FDR Session

Figure 2: FDR Session

This step involves interacting with the FDR Session within the Functional Data Reconciliation (FDR) workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Start pop up

Figure 3: Start pop up

Choose Folder’ pop-up window

Figure 4: Choose Folder’ pop-up window

The Choose Folder’ pop-up window dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Add Folder’ pop-up window

Figure 5: Add Folder’ pop-up window

The Add Folder’ pop-up window dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Choose folder pop up window

Figure 6: Choose folder pop up window

The Choose folder pop up window dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Start widget

Figure 7: Start widget

drag and drop an RDBMS table widget

Figure 8: drag and drop an RDBMS table widget

Drag the An Rdbms Table Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Read data from Table’ pop-up window

Figure 9: Read data from Table’ pop-up window

The Read data from Table’ pop-up window dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Select connection and click on Navigator

Figure 10: Select connection and click on Navigator

Select the appropriate connection from the dropdown to link this widget to its data source. Choosing the correct connection profile ensures the scenario reads from or writes to the intended database or file system.

Click on Additional Settings button

Figure 11: Click on Additional Settings button

The Additional Settings panel provides advanced configuration options such as custom SQL filters, join conditions, and performance tuning parameters. Configure only the settings relevant to your scenario — leave optional fields at their default values unless a specific behaviour is required.

Additional options All fields

Figure 12: Additional options All fields

Navigate to the Scenario Studio module and select the Functional Data Reconciliation (FDR) option from the left sidebar or main menu. This opens the scenario builder canvas where you will configure the data source connections, transformation rules, and execution settings.

Click on Navigator

Figure 13: Click on Navigator

The Navigator panel displays the available databases, schemas, and tables for the selected connection. Expand the tree to locate the required table or view, then click Select to add it to the widget configuration.

Select table and click on preview data button

Figure 14: Select table and click on preview data button

Preview Data

Figure 15: Preview Data

The Preview Data option fetches a sample of records from the selected table to verify the connection and column structure. Confirm that the correct columns and sample data are visible before closing the preview and saving the widget.

Click on Select button

Figure 16: Click on Select button

This step involves clicking the Click on Select button within the Functional Data Reconciliation (FDR) workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

RDBMS widget Saved

Figure 17: RDBMS widget Saved

Click the Save button to persist the current widget or scenario configuration. A confirmation indicator appears on the canvas once the save is successful, and the widget thumbnail updates to reflect the configured state.

Drag and drop Target RDBMS widget

Figure 18: Drag and drop Target RDBMS widget

Drag the Target Rdbms Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

RDBMS Popup

Figure 19: RDBMS Popup

The RDBMS Popup dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Select connection and click on Navigator

Figure 20: Select connection and click on Navigator

Click on Additional options Button

Figure 21: Click on Additional options Button

All Additional options fields

Figure 22: All Additional options fields

Click on Navigator

Figure 23: Click on Navigator

Select table

Figure 24: Select table

Preview data

Figure 25: Preview data

Click on Select button

Figure 26: Click on Select button

Click on Transformations Tab

Figure 27: Click on Transformations Tab

The Transformations tab lists all available data transformation widgets including Filter, Derive, Aggregate, and Lookup. Add transformation widgets between data source and reconciliation or validation widgets to cleanse or reshape data before comparison.

Three types of transformations

Figure 28: Three types of transformations

Drag and drop Filter widget

Figure 29: Drag and drop Filter widget

Drag the Filter Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Filter popup

Figure 30: Filter popup

The Filter popup dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Drag and drop Field, Operators and Input fields

Figure 31: Drag and drop Field, Operators and Input fields

Drag the Field, Operators And Input Fields from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Click on Available Field Dropdown

Figure 32: Click on Available Field Dropdown

This step involves clicking the Click on Available Field Dropdown within the Functional Data Reconciliation (FDR) workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Click on Available Operators dropdown and select one

Figure 33: Click on Available Operators dropdown and select one

This step involves clicking the Click on Available Operators dropdown and select one within the Functional Data Reconciliation (FDR) workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Select Input Value

Figure 34: Select Input Value

This step involves selecting the Select Input Value within the Functional Data Reconciliation (FDR) workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Click on Reconciliation Tab

Figure 35: Click on Reconciliation Tab

The Reconciliation widget maps source fields to target fields for comparison. Use Smart Mapping to automatically propose column matches, then manually adjust any fields that were incorrectly paired before saving.

Map and Recon widget

Figure 36: Map and Recon widget

Drag and drop Map and Recon widget

Figure 37: Drag and drop Map and Recon widget

Drag the Map And Recon Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Mapp Source and Target to Recon widget

Figure 38: Mapp Source and Target to Recon widget

Select key

Figure 39: Select key

Select the key column(s) that uniquely identify each record in both the source and target datasets. Accurate key selection is critical — an incorrect key will cause the reconciliation to produce inaccurate match results.

All Action buttons in the Recon widget

Figure 40: All Action buttons in the Recon widget

Sort Options

Figure 41: Sort Options

Smart Mapping button

Figure 42: Smart Mapping button

Delete all Mapping button

Figure 43: Delete all Mapping button

Delete Selected MApping

Figure 44: Delete Selected MApping

Click on Upload Mapping Button

Figure 45: Click on Upload Mapping Button

Upload Mappings popup

Figure 46: Upload Mappings popup

The Upload Mappings popup dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Open File Popup

Figure 47: Open File Popup

The Open File Popup dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Selected Mapping

Figure 48: Selected Mapping

Click on Verify Button

Figure 49: Click on Verify Button

This step involves clicking the Click on Verify Button within the Functional Data Reconciliation (FDR) workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Click on Map Button

Figure 50: Click on Map Button

Fuzzy Mapping

Figure 51: Fuzzy Mapping

Historical Mapping Button

Figure 52: Historical Mapping Button

Search Button

Figure 53: Search Button

This step involves interacting with the Search Button within the Functional Data Reconciliation (FDR) workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Additional Settings Button

Figure 54: Additional Settings Button

Additional Options Popup

Figure 55: Additional Options Popup

Reporting Charectorstics Button

Figure 56: Reporting Charectorstics Button

The Reporting Charectorstics Button page presents the full outcome of the scenario execution including record counts, match rates, and any detected exceptions. Drill into exception records to inspect individual mismatches and use the Export to CSV option to share results with stakeholders.

Reporting Characteristics popup

Figure 57: Reporting Characteristics popup

The Reporting Characteristics popup dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Click on Save Button

Figure 58: Click on Save Button

Confirm Action Button

Figure 59: Confirm Action Button

Additional Options tab

Figure 60: Additional Options tab

All Additional Options

Figure 61: All Additional Options

Drag and drop Email widget

Figure 62: Drag and drop Email widget

Drag the Email Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Configure Email

Figure 63: Configure Email

The Email widget sends an automated result notification to configured recipients after the scenario finishes executing. Provide the recipient addresses, subject, and choose whether to attach the exception report or include a summary in the email body.

Email When Exception Found

Figure 64: Email When Exception Found

Click on Execute Button

Figure 65: Click on Execute Button

Click the Execute button to trigger the scenario and begin processing data from the configured sources. The scenario transitions to Monitor mode where you can track real-time execution progress and view interim results as each step completes.

All Action buttons in theFDR Create Page

Figure 66: All Action buttons in theFDR Create Page

FDR Monitor mode

Figure 67: FDR Monitor mode

Monitor mode displays the live execution status of the scenario with a progress indicator for each processing stage. Wait for all stages to reach a Completed or Passed state before navigating to the result summary to review the full output.

Result Summary Page

Figure 68: Result Summary Page

The Result Summary Page page presents the full outcome of the scenario execution including record counts, match rates, and any detected exceptions. Drill into exception records to inspect individual mismatches and use the Export to CSV option to share results with stakeholders.

Recived Email

Figure 69: Recived Email