You are viewing documentation for Data Trust v7.6 (latest).
Looking for the previous version? View v2024.06 documentation →

Data Trust › Scenario Studio › Scenario Builder

FDR using Remote FTP (CSV) and Snowflake

Reconcile a remote FTP CSV file against a Snowflake dataset.

Overview

This guide shows how to configure an FDR scenario using a remote FTP CSV file as the source and Snowflake as the target. The FTP adapter retrieves the CSV file and its contents are reconciled against Snowflake data.

Note: Ensure all required connection profiles are configured in the Administration module before building this scenario.

Prerequisites

• You are logged in to Data Trust with a Pro-User or Admin Pro-User role.
• An FTP connection profile and a Snowflake connection profile have both been configured.

Steps

Path: Data Trust › Scenario Studio › Scenario Builder › FDR using Remote FTP (CSV) and Snowflake

Figure 1: FDR Option

Navigate to the Scenario Studio module and select the FDR using Remote FTP (CSV) and Snowflake option from the left sidebar or main menu. This opens the scenario builder canvas where you will configure the data source connections, transformation rules, and execution settings.

Figure 2: Drag and drop CSV Widget

Drag the Csv Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Figure 3: FTP Details

This step involves interacting with the FTP Details within the FDR using Remote FTP (CSV) and Snowflake workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 4: Click on Preview Data

The Preview Data option fetches a sample of records from the selected table to verify the connection and column structure. Confirm that the correct columns and sample data are visible before closing the preview and saving the widget.

Figure 5: Preview Data popup

The Preview Data popup dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Figure 6: Drag and drop RDBMS Widget

Drag the Rdbms Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Figure 7: Select Connection and click on Navigator

Select the appropriate connection from the dropdown to link this widget to its data source. Choosing the correct connection profile ensures the scenario reads from or writes to the intended database or file system.

Figure 8: Select table and click on Select button

The Navigator panel displays the available databases, schemas, and tables for the selected connection. Expand the tree to locate the required table or view, then click Select to add it to the widget configuration.

Figure 9: Click on Reconciliation Tab

The Reconciliation widget maps source fields to target fields for comparison. Use Smart Mapping to automatically propose column matches, then manually adjust any fields that were incorrectly paired before saving.

Figure 10: Drag and drop Recon widget

Drag the Recon Widget from the widget palette and drop it onto the scenario canvas. Positioning the widget opens its configuration popup where you will set the connection, table, and any additional options.

Figure 11: Select Key Column

Select the key column(s) that uniquely identify each record in both the source and target datasets. Accurate key selection is critical — an incorrect key will cause the reconciliation to produce inaccurate match results.

Figure 12: Confirm Action Popup

The Confirm Action Popup dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Figure 13: Click on Execute Button

Click the Execute button to trigger the scenario and begin processing data from the configured sources. The scenario transitions to Monitor mode where you can track real-time execution progress and view interim results as each step completes.

Figure 14: Click on Exception Count

The Click on Exception Count page presents the full outcome of the scenario execution including record counts, match rates, and any detected exceptions. Drill into exception records to inspect individual mismatches and use the Export to CSV option to share results with stakeholders.

Figure 15: Result Summary Page

The Result Summary Page page presents the full outcome of the scenario execution including record counts, match rates, and any detected exceptions. Drill into exception records to inspect individual mismatches and use the Export to CSV option to share results with stakeholders.