Functional Data Reconciliation using Data snapshot as source

You are viewing archived documentation for Data Trust v2024.06 (previous version).
Go to the latest v7.6 documentation →

Functional Data Reconciliation using Data snapshot as source

Users can use snapshot output data of different Queries, Query Chains, and Scenarios (Reconciliation Output) and can compare the data sets (Source and Target) to the field by field level. Let’s execute an FDR scenario using output snapshot data of a Query Chain as a source.

To learn more, let us walk through the below procedure:

• Click the ‘SCENARIO STUDIO’ module.
• Hover on the ‘Scenario Builder’ snippet from the left pane options.
• Click the ‘Functional Data Recon’ option.

• This action navigates the user to a new FDR (Functional Data Recon) scenario session page.

• From the ‘Data Source’ tab, let’s drag and drop ‘Query Chain’ (source) icon onto the FDR session page.
• Double tap the ‘Read data from Table’ widget.

• A ‘Read data from Table’ pop-up window is displayed.
• Let’s select and checkbox ‘ajithRD 6/17/2022’ Query Chain.

Note: ‘Sessions’ and ‘Metadata’ tabs are displayed immediately after selecting the Query Chain.

• Click the ‘Select’ button.

• The ‘Query Chain’ widget is closed saving its data.
• The user is navigated to the FDR session page.

• From the ‘Data Source’ tab, let’s drag and drop a ‘Custom SQL’ (target) icon on to the FDR session page.
• Double tap the ‘Read data from custom SQL query’ widget.

• A ‘Read data from custom SQL query’ pop-up window is displayed.
• Let’s select ‘Postgre SQL’ as ‘Data Type’ drop-down list.
• Let’s select ‘Postgre SQL Northwind-QA’ as ‘Connection Name’ drop-down list.
• Click the ‘Navigator’ button.

• A ‘Navigator’ slide screen is displayed.
• Let’s select ‘Categories’ table from the ‘Public’ schema.

• Click the ‘Categories’ table.
• This action adds the custom query of Categories.
• Click the ‘Infer Metadata’

• This action loads the ‘Metadata’ displaying the columns and data types.
• Click the ‘Save’ button.

• This action saves the ‘Custom SQL’ widget data by closing it.
• Click the ‘Transformations’ tab.

• Let’s drag and drop an ‘Aggregation’ icon on to the FDR session page.
• Double tap the ‘Aggregate data from ajithRD 6/17/2022 10:13:00 AM’ widget.

• An ‘Aggregate data from ajithRD 6/17/2022 10:13:00 AM’ pop-up window is displayed.

Note: By default, all the columns are selected and check boxed.

• Now, select and checkbox only ‘category_desc’ and ‘Category_code’ by selecting Aggregation as ‘Min’ for ‘category_desc’ column and ‘Count’ for ‘Category_code.’
• Click the ‘Save’ button.

• This action saves the ‘Aggregation’ widget data and closes it.

• From the ‘Transformations’ tab, let’s drag and drop a ‘Filters’ icon on to the FDR session page.
• Double tap the ‘Filter data from PostgreSQL Northwind-QA’ widget.

• A ‘Filter data from PostgreSQL Northwind-QA’ pop-up window is displayed.

• Let’s select ‘Categoryid’ drop-down list for ‘Available Fields’ ‘Less than’ for ‘Available Operators, and ‘input’ as ‘100.’
• The Expression (categoryid < ‘100’) is added under the ‘Expression’ field.
• Click the ‘Save’ button.

• This action saves the ‘Filters’ widget data by closing it.
• Click the ‘Reconciliations’ tab.

• Let’s drag and drop ‘Map source & target fields’ icon onto the FDR session page.
• Double tap the ‘Map source & target fields’ widget.

• A ‘Map source & target fields’ pop-up window is displayed.
• Click the ‘Save’ button.

• This action pops-up a ‘Confirm action’ pop-up window.
• Click the ‘Verified’ button.

• This action saves the ‘Map source & target fields’ widget data by closing it.
• Click the ‘Additional Options’ tab.

• Let’s drag and drop an ‘Email’ icon onto the FDR session page.
• Double tap the ‘Email Settings’ widget.

• An ‘Email Settings’ pop-up window is displayed.

Note: Email ids are greyed due to confidential data.

• Click the ‘Save’ button.

• This action saves the ‘Email’ widget data by closing it.
• Click the ‘Execute Now’ button.

• A monitor mode is enabled.
• The FDR scenario is ‘Errored’ here.
• Let’s click the ‘Errored’ icon from the ‘Map source & target fields’ widget.

• This action navigates the user to the ‘Functional Data Reconciliations Exceptions’ session page.

• The ‘Reconciliation Result’ email report is displayed as a screenshot below.