Metadata Extract & Enrich v7.6 beta
Overview
Metadata Extract & Enrich is where you configure and manage the metadata extraction and AI-powered enrichment process for each of your data source connections. This module reads the structural metadata (schemas, tables, columns, data types, keys) from your databases and enriches it with AI-generated descriptions, classifications, and data quality insights.
Target Persona: Data Engineers, Platform Administrators.
Prerequisites: Connection profiles must be created in Administration → Admin Console → Connections.
How This Module Helps
This module is the data discovery engine of RightSight. Before you can monitor metadata health, generate DQ rules, or build the Enterprise Trust Map, you need to extract metadata from your data sources. This module: - Discovers all schemas, tables, views, and columns in your connected databases - Enriches metadata with AI-generated descriptions and classifications - Schedules recurring extraction to detect changes over time - Feeds all downstream modules with fresh metadata
| Connects To | Relationship |
|---|---|
| Review & Confirm (Metadata) | Extracted objects go here for review |
| Metadata Observability | Extracted metadata becomes observable |
| Metadata Analysis | Extracted metadata is analyzable |
| AI Rule Agent | Agent uses extracted metadata to generate rules |
| Enterprise Trust Map | Only fully extracted profiles appear |
Navigation
| Method | Path |
|---|---|
| Sidebar | Observability → RightSight → Metadata Extract & Enrich |
| URL | /metadata-extraction |
| Wizard | Augmented DQ Config → Step 4 (Metadata) |
| Deep link |
/metadata-extraction?fromWizard=true&connectionId=<id> auto-opens config drawer |
Features & Workflows
1. Connection Profiles Grid
Displays all connection profiles as cards.
Each card shows: - Source icon — Database type icon (Snowflake, SQL Server, Oracle, PostgreSQL, etc.) - Profile name — Connection display name - Extraction status — Completed / In Progress / Not Started / Failed - Enrichment status — Completed / Not Started - Last run time — When extraction last executed - Attention badge — Count of items needing attention
Filter Badges
| Badge | Filters To |
|---|---|
| All | All connections |
| Extracted | Connections with completed extraction |
| Enriched | Connections with completed enrichment |
| Needs Attention | Connections with errors or requiring action |
| In Progress | Currently running extractions |
2. Configuring Extraction Scope
Steps
-
Click Config on any connection card.
-
The Configuration Drawer opens with 4 tabs.
Scope Tab
Configure what to extract:
| Field | Type | Description |
|---|---|---|
| Schemas | Multi-select | Select which schemas to include/exclude |
| Object Types | Checkboxes | Choose Tables, Views, or both |
| Exclusion Mode | Toggle | Include-only or Exclude mode |
| Patterns | Tags input | Regex patterns for table/schema name filtering (e.g., tmp_*, stg_*) |
Schedule Tab
Configure automated extraction:
| Field | Type | Description |
|---|---|---|
| Frequency | Dropdown | Daily, Weekly, Monthly, Custom cron |
| Time | Time picker | Execution time |
| Cron Expression | Text input | Advanced cron expression (for Custom) |
| Enable/Disable | Toggle | Turn scheduled extraction on/off |
Notifications Tab
| Field | Type | Description |
|---|---|---|
| Email Recipients | Tags input | Email addresses to notify |
| Alert Types | Checkboxes | Extraction completed, failed, enrichment completed, schema changes detected, row count changes |
Retention Tab
| Field | Type | Description |
|---|---|---|
| Snapshot Retention | Number | How many historical metadata snapshots to keep |
| Retention Period | Dropdown | Days/weeks/months of history |
- Click Save to apply configuration.
- The connection card reflects the updated configuration.
- If a schedule was set, extraction runs automatically at the configured time.
- Notification recipients will receive alerts for configured events.
3. Running Extraction (Force Sync)
Steps
- Click ⟳ Sync on a connection card.
- A Force Sync Confirm Dialog appears.
-
Click Confirm to trigger immediate extraction.
-
The card status updates to In Progress with a spinner.
-
The Logs Drawer auto-opens to show real-time progress.
- Extraction discovers all schemas, tables, views, and columns from the database.
- On completion, the card status updates to Completed with counts.
- Extracted objects appear in Review & Confirm (Metadata).
- Metadata becomes available in Metadata Observability and Metadata Analysis.
- Notification emails are sent to configured recipients.
4. Viewing Extraction Logs
Steps
-
Click 📋 Logs on any connection card.
-
The Logs Drawer opens showing:
| Section | Description |
|---|---|
| Real-time log stream | Live extraction/enrichment progress |
| Historical runs | Browse past extraction runs |
| Log levels | Filter by Info, Warning, Error |
| Duration | Total extraction time |
| Object counts | Schemas, tables, columns discovered |
-
Filter logs by level using the level selector.
-
Click any log entry for full details.
5. Searching Connection Profiles
- Type in the Search bar at the top.
- The grid filters to show matching connection profiles.
Cross-Module Links
| From Metadata Extract | Navigate To |
|---|---|
| Extraction complete → review objects | Review & Confirm (Metadata) |
| Extracted metadata → observable | Metadata Observability |
| Extracted metadata → analyzable | Metadata Analysis |
| Connection profile → AI rule generation | AI Rule Agent |
| Connection profile → Enterprise Trust Map | ETM — only fully extracted profiles appear |
| Wizard → next step | AI Rule Agent |
Tips & Best Practices
- Schedule extractions during off-peak hours to minimize database load.
- Start with a narrow schema scope, then expand as you validate.
- Use exclusion patterns to skip temporary/staging tables (e.g.,
tmp_*,stg_*). - Configure notifications for extraction failures to catch connectivity issues early.
- Monitor the Logs Drawer during first-time extraction to verify schema discovery.