You are viewing documentation for Data Trust v7.6 (latest).
Looking for the previous version? View v2024.06 documentation →

Data Trust › Scenario Studio › Scenario Builder

Anomaly Detection in a Validation Scenario

Use statistical anomaly detection to find records that deviate from expected patterns.

Overview

Anomaly Detection in a validation scenario uses statistical analysis to identify records that deviate significantly from expected patterns or historical norms. This guide walks through adding an Anomaly Detection rule to a validation scenario and reviewing the detected anomalies.

Note: Ensure all required connection profiles are configured in the Administration module before building this scenario.

Prerequisites

• You are logged in to Data Trust with a Pro-User or Admin Pro-User role.
• An RDBMS connection has been configured and the target table has sufficient historical data.

Steps

Path: Data Trust › Scenario Studio › Scenario Builder › Anomaly Detection in a Validation Scenario

Figure 1: Validation scenario navigation

Navigate to the Scenario Studio module and select the Anomaly Detection in a Validation Scenario option from the left sidebar or main menu. This opens the scenario builder canvas where you will configure the data source connections, transformation rules, and execution settings.

Figure 2: RDBMS widget

This step involves interacting with the RDBMS widget within the Anomaly Detection in a Validation Scenario workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 3: Read Data from Table pop up

The Read Data from Table pop up dialog exposes all available configuration options for this step. Review each field carefully and fill in the required values before clicking Save or OK to apply the configuration.

Figure 4: Selected table in RDBMS

This step involves selecting the Selected table in RDBMS within the Anomaly Detection in a Validation Scenario workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 5: Validation Rules tab

The Validation Rules tab lists all available business rule widgets that can be applied to the selected dataset. Drag the required rule widget onto the canvas, configure its threshold or pattern, then save before moving to the next step.

Figure 6: anomaly widget

Configure the anomaly widget parameters including the target column(s), threshold values, and any custom expressions. After saving, the rule is added to the scenario and will be evaluated against every record when the scenario executes.

Figure 7: Selected Columns

This step involves selecting the Selected Columns within the Anomaly Detection in a Validation Scenario workflow. Follow the on-screen prompts to complete this step before proceeding to the next configuration stage.

Figure 8: Execute now button

Click the Execute button to trigger the scenario and begin processing data from the configured sources. The scenario transitions to Monitor mode where you can track real-time execution progress and view interim results as each step completes.

Figure 9: Executed scenario

Click the Execute button to trigger the scenario and begin processing data from the configured sources. The scenario transitions to Monitor mode where you can track real-time execution progress and view interim results as each step completes.

Figure 10: Anomaly results

The Anomaly results page presents the full outcome of the scenario execution including record counts, match rates, and any detected exceptions. Drill into exception records to inspect individual mismatches and use the Export to CSV option to share results with stakeholders.