Dataset Management
Dataset Management lets you review the object types a source collects and control how the source aggregates those objects into SailPoint. Use Dataset Management to view and manage resource attributes, datasets and their resources, enable dataset aggregation schedules, perform manual aggregation, and view aggregation history.
Note
Datasets are also available in SailPoint Agentic Fabric. Contact your Customer Success team and refer to SailPoint Agentic Fabric for more information.
For more information, refer to Managing Datasets and Resources for datasets, resources, owner correlation, and aggregation.
Datasets
A dataset is a named group of one or more related resources that the source aggregates together.
When aggregation runs for a dataset, the source collects all resources in that dataset from the managed system in one aggregation job. Activate or deactivate aggregation at the dataset level, not for individual resources.
Note
Each resource belongs to exactly one dataset for that source. The source defines which resources belong to each dataset. You can enable aggregation and review associated resources, but you cannot create or delete datasets or move resources between them.
Dataset Aggregation
Configure aggregation on the Dataset Aggregations tab for each dataset. Configure each dataset separately. During aggregation, your owner correlation rules are automatically applied to link machine identities to their human owners. For more information, refer to Configuring Dataset Aggregations.
Resources
A resource defines an object type from the managed system. It specifies the kind of object, the attributes the source collects, and how the source identifies and displays each object. Resources form the building blocks of the data model.
In the UI, each resource appears under Dataset Management > Resources with its schema. A resource does not run aggregation by itself. The dataset that owns the resource collects it during aggregation.
For more information, refer to Managing Resources.
Owner Correlation
Configure owner correlation on each resource to link aggregated objects to human owners. If you do not configure owner correlation, or if the mapping produces no match, an owner is not assigned. For more information, refer to Configuring Owner Correlation.
Supported Datasets
Go to Dataset Management > Datasets on your Starburst source to view the datasets the connector aggregates. Select a dataset to review its associated resources and configure aggregation.
Note
Configure aggregation for each dataset separately. For more information, refer to Dataset Aggregation.
Note
These resources do not define owner correlation. Each service account is identified by serviceAccountId and displayed by userName. Each cross-account IAM role is identified by roleArn and displayed by roleName. Each SQL job is identified by sqlJobId and displayed by name.
| Dataset name | Associated resources | Dataset ID | Aggregation schedule |
|---|---|---|---|
| Service Account | Service Account | starburst:service-account |
Every 12 hours (0 */12 * * *) |
| Cross-Account IAM Role | Cross-Account IAM Role | starburst:cross-account-iam-role |
Every 12 hours (0 */12 * * *) |
| SQL Job | SQL Job | starburst:sql-job |
Every 12 hours (0 */12 * * *) |
For resource schema attributes, refer to Supported Resources, Service Account Attributes, Cross-Account IAM Role Attributes, and SQL Job Attributes.
Dataset Aggregations
Dataset aggregation collects resource objects from the managed system and loads them into SailPoint. Repeat the configuration for each dataset on the source.
Configuring Dataset Aggregation
To configure dataset aggregation for a dataset:
- Go to Admin > Connections > Sources, and open your source.
- Go to Dataset Management > Datasets.
- Select the dataset you want to configure.
-
Select the Dataset Aggregations tab.
On the Dataset Aggregations tab, configure how resource objects from the selected dataset are aggregated into SailPoint. The tab contains Scheduled Aggregations, Manual Aggregation, and Latest Aggregation.
Scheduled Aggregations
Scheduled aggregations run on a schedule to keep the dataset synchronized with the managed system.
- On the Dataset Aggregations tab, select Enable Schedule to turn on scheduled aggregations.
When scheduled aggregations are enabled, SailPoint aggregates all resource objects in the dataset on the configured schedule.
Manual Aggregation
Manual aggregation runs once and collects all resource objects in the dataset.
- On the Dataset Aggregations tab, select Start Aggregation.
SailPoint starts the aggregation immediately. When the run completes, results appear in the Latest Aggregation section.
Viewing Aggregation Results
After an aggregation completes, review its status in the Latest Aggregation section on the Dataset Aggregations tab.
The Latest Aggregation section shows:
- Start Time - when the aggregation started
- Resource Objects Scanned - the number of resource objects the aggregation processed
- Status - whether the aggregation succeeded or failed
Select Details to view additional information about the most recent run. To view prior runs, select Aggregation History.
Supported Resources
For more information, refer to Managing Resources.
Note
When you enable dataset aggregation for a selected dataset, all resources in that dataset are collected at once. Activate or deactivate aggregation at the dataset level, not for individual resources.
Under Dataset Management > Resources, the Starburst source lists each object type that is aggregated. Select a resource to review its schema on the Schema tab.
| Resource name | Resource type | Resource ID | Identity attribute | Display attribute |
|---|---|---|---|---|
| Service Account | Account |
starburst:service-account |
serviceAccountId |
userName |
| Cross-Account IAM Role | Account |
starburst:cross-account-iam-role |
roleArn |
roleName |
| SQL Job | Application |
starburst:sql-job |
sqlJobId |
name |
For the complete resource schemas, refer to Service Account Attributes, Cross-Account IAM Role Attributes, and SQL Job Attributes.
Owner Correlation
These resources do not define owner correlation. For more information, refer to Owner Correlation.
Service Account Attributes
Use the following schema attributes for the Service Account resource in the Service Account dataset. Service accounts are machine logins used by tools and automation. Each one has a primary role, optional additional roles, and one or more passwords.
| Attribute Name | Description |
|---|---|
| serviceAccountId | Unique identifier of the service account. Resource ID. |
| userName | Username of the service account. Display attribute. |
| roleId | Primary role assigned to the service account. |
| additionalRoleIds | Additional roles assigned to the service account. |
| syncToken | Synchronization token for the service account. |
| classification | Account classification. Service accounts are machine accounts. |
| serviceAccountPasswordIds | Identifiers of the service account's passwords. |
| passwordPrefixes | Prefixes of the service account's passwords. |
| passwordDescriptions | Descriptions of the service account's passwords. |
| passwordCreatedDates | Date and time when each password was created. |
Cross-Account IAM Role Attributes
Use the following schema attributes for the Cross-Account IAM Role resource in the Cross-Account IAM Role dataset. Cross-account IAM roles are AWS roles that Galaxy assumes to reach object storage and other AWS data sources for catalogs.
| Attribute Name | Description |
|---|---|
| roleArn | Amazon Resource Name (ARN) of the cross-account IAM role. Resource ID. |
| roleName | Name of the cross-account IAM role. Display attribute. |
| roleAlias | Alias of the cross-account IAM role in Galaxy. |
| status | Current status of the role. |
| validationMessage | Details about the role's validation status. |
| classification | Account classification. Cross-account IAM roles are machine accounts. |
| createdAt | Date and time when the role was created. |
| updatedAt | Date and time when the role was last updated. |
SQL Job Attributes
Use the following schema attributes for the SQL Job resource in the SQL Job dataset. SQL jobs are scheduled queries. Each one runs under a role on a cron schedule.
| Attribute Name | Description |
|---|---|
| sqlJobId | Unique identifier of the SQL job. Resource ID. |
| name | Name of the SQL job. Display attribute. |
| cronExpression | Cron schedule for the SQL job. |
| timezone | Time zone in which the schedule is evaluated. |
| roleId | Role the SQL job runs under. |
| syncToken | Synchronization token for the SQL job. |