Skip to content

Loading Account Data

IdentityNow needs to collect information about your users' system accounts and their associated access so they can be governed. To load or aggregate account data into IdentityNow, you'll first need to create one or more sources.

Configuring a Source

A source is the IdentityNow representation of a third-party application, database, or directory management system that maintains its own set of user accounts or personnel records. IdentityNow collects data from these sources. A source can be added to IdentityNow through a direct connection or a flat file feed.

  • A direct connection is a method of communicating directly between a source server and IdentityNow. To set up a direct connection, you'll provide essential connection information specific to the source.
  • A flat file feed is a .csv file that contains all relevant information about the accounts you want to load into IdentityNow. When you create a flat file source, you load the account information by importing the file.

To create a source:

  1. Go to Admin > Connections > Sources.

  2. Select Create New.

  3. Search for a source type and select Configure.

    For descriptions of the available source types, refer to the IdentityNow Connectors documentation.

  4. Enter a unique name and description for the source to help admins differentiate it from others.

  5. Select a source owner who will be responsible for the system.
  6. Choose whether you will be connecting IdentityNow directly to the source system or loading its data from a file-based representation of its data.
  7. For direct connection sources, select a virtual appliance cluster with connectivity to the source.
  8. (Optional) Select a governance group to grant its members source or role sub-admin level oversight of the source and its access.
  9. Select Continue to go to the source configuration page.

The remaining source configuration details depend on the source type and connection type selected:

  • To access the documentation for a specific supported direct connect source, refer to the IdentityNow Connectors documentation.

  • If you choose a flat file connection type for any source type, you do not have to specify connection parameters. Instead, you will use a file import to load the source's data. In addition, IdentityNow offers two connectors for loading flat files when there is not a predefined connector for the source: Delimited File and Generic.

Additional Configurations

  • If the source is an authoritative source containing records that represent your organization's personnel, you must create identity profiles for the source to create identities from the source data.

  • If the source represents a system where your personnel have user accounts and access rights, ensure that its correlation logic will match the source accounts to the identities they belong to.

After you complete and save your source configuration, you can manually aggregate account information as needed or schedule account aggregation from any direct connect source on a regular basis.

Scheduling Aggregations for Direct Connect Sources

You can schedule aggregations to automatically load new data on the source into IdentityNow on a regular basis.

  1. Go to Admin > Connections > Sources and select the source you want to schedule aggregations for.

  2. Go to Import Data > Account Aggregation and select the Enable Schedule checkbox.

  3. Choose how often the aggregation runs and the time you want each aggregation to start.

    Notes

    • If you select Daily, you can use the Recurring Every dropdown list to set an interval within the day. The start time and interval will determine the number of aggregations that will be performed each day. For example, if the time is set to 2 PM and the recurrence is set to 8 hours, the aggregation will occur twice each day. To aggregate once per day, set it to 24 hours.
    • Recurrences stop at midnight. If you select a start time and interval that would result in a single run for the day, the Recurring Every dropdown automatically changes to 24 hours when the page reloads. For example, if the start time is set to 1 PM and the recurrence is set to 12 hours, a single aggregation will run at 1 PM that day. The next time the user loads the page, the Recurring Every dropdown will be set to 24 hours.
    • The time zone (GMT offset) for the account aggregation schedule is determined by the time zone set for the connected virtual appliance cluster. Since time zones are set as an offset of GMT, standard time for the selected time zone is always used; schedules do not shift for daylight savings time.
    • The aggregation is added to the processing queue at the defined time. Other queued or in-progress operations may delay the job start.
  4. (Optional) Select the Disable Account Deletion checkbox to ensure no accounts are deleted, or set the percentage of allowed deleted accounts per aggregation in the Account Delete Threshold section. Select Save to save your changes.

    Note

    The percentage must be an integer between 1 and 100. If the deletions exceed this value, IdentityNow will not delete any accounts. SailPoint recommends using this option to avoid removing user data in the event of a misconfiguration.

    Account Deletion Limitations
    • If a source has 10 or fewer accounts, setting this value to 4 percent or less will cause IdentityNow to round it to 1 percent to prevent all accounts from being deleted.
    • If a source has 11 - 20 accounts, setting this value to 2 percent or less will cause IdentityNow to round it 1 percent to prevent all accounts from being deleted.
  5. Select Save to schedule aggregations for the source.

If you need to cancel an account aggregation after it has started, you can do so from the Aggregation Activity Log.

Aggregation Best Practices

SailPoint recommends following these best practices to help your aggregations run more efficiently.

  • Aggregate as infrequently as possible based on your business needs. It is almost never necessary to aggregate more than once a day for most non-authoritative sources.
  • When possible, aggregate outside of business hours.
  • Only aggregate account data relevant to identity governance processes. Use filtering on the connector if applicable to aggregate only the most recent records for active users.
  • If the source supports delta aggregation, use this option to lower your aggregation times and minimize load on the system.

Manually Aggregating Information from a Direct Connect Source

You can manually aggregate accounts from the source from the Source Aggregation tab.

  1. Go to Admin > Connections > Sources and select a source.

  2. Select the Import Data tab and then select Account Aggregation. To aggregate entitlements, refer to Loading Entitlements for a Direct Connect Source.

  3. (Optional) Select the Disable Account Deletion checkbox to ensure no accounts are deleted, or set the percentage of allowed deleted accounts per aggregation in the Account Delete Threshold section. Select Save to save your changes.

    Note

    The percentage must be an integer between 1 and 100. If the deletions exceed this value, IdentityNow will not delete any accounts. SailPoint recommends using this option to avoid removing user data in the event of a misconfiguration.

    Account Deletion Limitations
    • If a source has 10 or fewer accounts, setting this value to 4 percent or less will cause IdentityNow to round it to 1 percent to prevent all accounts from being deleted.
    • If a source has 11 - 20 accounts, setting this value to 2 percent or less will cause IdentityNow to round it 1 percent to prevent all accounts from being deleted.
  4. Select the Start button next to Manual Aggregation.

The aggregation is added to the Aggregation Activity Log, where you can view the status of aggregations and the date and time the aggregation was started.

While an account aggregation is running, the Start button will be disabled. If you need to cancel an account aggregation after it has started, you can do so from the Aggregation Activity Log.

Manually Aggregating from a Flat File

If you choose a flat file connection type for any source type, you must use a file import to manually aggregate the source's data. After you close out of the source configuration, you will be taken to the source’s details page.

  1. Go to Admin > Connections > Sources and select a source.
  2. Select the Import Data tab.
  3. Select Import Accounts.

  4. (Optional) If you haven't created a .csv file of the account data, select Download to download a mapping template that can be used for the file import. You can include any set of columns.

  5. Select the Disable Account Deletion checkbox to ensure no accounts deleted, or set the percentage of allowed deleted accounts per aggregation in the Account Delete Threshold section. Select Save to save your changes.

    Note

    This percentage must be an integer between 1 and 100. If the expected deletions exceed this value, IdentityNow will not delete any accounts. SailPoint recommends using this option to avoid removing user data in the event of a misconfiguration.

    Account Deletion Limitations
    • If a source has 10 or fewer accounts, setting this value to 4 percent or less will cause IdentityNow to round it to 1 percent to prevent all accounts from being deleted.
    • If a source has 11 - 20 accounts, setting this value to 2 percent or less will cause IdentityNow to round it 1 percent to prevent all accounts from being deleted.
  6. Select Import.

  7. Select the file you want to import.

Configuring Delta Aggregations for Supported Sources

When IdentityNow performs a delta aggregation on a supported source, it only loads the accounts that have been created or changed since the most recent aggregation. For example, if one user's group memberships change in Active Directory, you can aggregate account information for that user only instead of all accounts on the source.

Notes

  • Delta aggregation does not support OU moves. If you perform any type of OU move, you need to perform full account aggregations to avoid unexpected behavior such as the creation of duplicate user accounts.
  • Delta aggregation does not support account deletion.

Supported Sources

The following sources support delta aggregation:

Microsoft Active Directory IBM Tivoli Directory Server SAP Direct
Microsoft Entra ID JDBC SAP HR/HCM
Microsoft Lightweight Directory Services Okta SCIM 2.0
Google Workspace Oracle SunOne ServiceNow Identity Governance
HCL Domino Oracle Fusion HCM Workday

To configure delta aggregations on a supported source:

  1. Go to Admin > Connections > Sources.

  2. In Cards view, select Edit for an existing source or create a new source.

    Alternatively, in Table view, select Actions > Edit for an existing source.

  3. In the source configuration pages, go to Additional Settings and select the Delta Aggregation checkbox.

    Note

    For JDBC, delta aggregation requires some addition configurations. Refer to the JDBC connector documentation for details.

Aggregating Account Information on a Direct Connect Source Using APIs

If you prefer to aggregate a direct connect source using the available REST API, you can do so using the loadAccounts API. In some cases, this is necessary, such as when you need to force reprocessing of all source accounts by disabling aggregation optimization.

Note

To use APIs, you'll need to use one of our supported authentication methods. For details, refer to the supported Authentication Methods in our API documentation.

  1. Go to Admin > Connections > Sources and select the source you want to aggregate.

    The source ID is displayed at the end of the URL in your browser address.

  2. Make note of this number, as you'll need to refer to in the next step.

  3. Use your preferred tool to call the following API:

    POST <api-url>/cc/api/source/loadAccounts/<source_id>

    where

    <api-url> is the URL to the API on the IdentityNow org (e.g., https://org-name.api.identitynow.com).

    <source_id> is the ID of the direct connect source you want to aggregate.

To ensure that every account is scanned when you aggregate the source:

If you have changed the correlation configuration for the direct connect source, you can disable optimization by including form-data: disableOptimization=true in the Body of the call. This allows IdentityNow to reprocess every record whether or not the data has changed.

Note

This parameter setting applies to the current aggregation only. Subsequent aggregations performed using the API will have optimization enabled unless you set this flag to true each time. Scheduled or manual aggregations performed using the UI will always run with optimization enabled.

Viewing the Aggregation Activity Log

When a source's accounts are being aggregated, an entry is added to the Aggregation Activity Log. This log includes information like the name of the admin who started it (for manual aggregations), the date and time the aggregation occurred, and its current status.

  1. Go to Admin > Connections > Sources and select the source you want to view aggregation activity for.

  2. For flat file sources, select Import Data > Import Accounts.

    For a direct connect sources, select Import Data > Account Aggregation.

    The Aggregation Activity Log appears as a table at the bottom of these pages.

  3. In the Aggregation Activity Log, select the Info icon to display detailed status information, including how long it took for the aggregation to complete and the number of retries that were attempted before the aggregation completed successfully.

    In the case of an unsuccessful aggregation, information that might help you diagnose the issue is provided.

  4. Select Close to close the window.

Canceling an Aggregation

To cancel an aggregation that is in progress, select the X next to it in the Aggregation Activity Log and choose Yes in the confirmation message.

Aggregation Activity Log entry showing the user, date, accounts scanned, and pending status.

This changes the status of the aggregation to Terminated.

Select the Info icon to view statistics and messages about the job execution and data aggregated before the job was terminated. This includes the aggregation’s status, starting date and time, duration, accounts scanned, optimization (enabled), and accounts deleted for the termination. You can also view the name of the user who terminated the aggregation in the warning message.

Aggregating Data for a Single Account

If you are a Helpdesk admin or an administrator, you may need to reload or aggregate the data from a single account to have the most current information. For example, if a user calls the Helpdesk for help unlocking their account, their locked status may not display immediately. Aggregating that person's account updates it data and status without taking the time and resources to update the source's entire contents.

  1. Go to Admin > Identities > Identity List.

  2. Select the name of the user whose account you want to aggregate.

  3. Select Accounts.

  4. Select the Actions icon on the desired account and choose Aggregate Account.

This action updates the related identity's account data, but it does not synchronize the new data with the identity attributes. It also does not have any impact on provisioning. It only loads any changes from the account source data.

Note

Account deletions are only processed during full source aggregations. If the account is not found on the source during single-account aggregation, it will remain unchanged in IdentityNow.

Deleting Authoritative Accounts and Identities

When an identity’s account on its authoritative source is deleted, aggregating that source deletes that authoritative account from IdentityNow, removing the account from the identity. The identity is then analyzed to determine if it should be reassigned to another identity profile, deleted, or hidden in IdentityNow.

  1. If the identity has an account on another, lower priority authoritative source, it will be reassigned to that source’s identity profile, and its identity attributes will be recalculated based on that identity profile’s mappings.

  2. If the identity does not belong to another authoritative source, IdentityNow will attempt to delete the identity. This happens overnight, in a nightly job that runs automatically.

  3. However, deletion can be blocked if the identity:

    • has accounts from any other sources correlated to it.
    • is the owner of any role, access profile, application, or source.
    • is the owner or requester of an active work item, such as an access request approval.
    • is designated as another identity’s manager.
    • has elevated IdentityNow capabilities, such as role admin, org admin, or helpdesk.
    • is marked as a protected identity. (This is an API-level configuration.)

    If any of these conditions prevent the identity from being deleted, it will remain in IdentityNow as a hidden identity.

    When an identity is hidden:

    • Since it is not connected to an identity profile, its last name, email, and UID values are no longer mapped, and the identity does not appear in the Identity List.
    • All accounts previously correlated to the identity remain attached to the hidden identity, but they are treated as uncorrelated accounts since their associated identity is not an authoritative identity. In an uncorrelated accounts certification that includes the accounts' sources, these accounts will be grouped together under a shared uncorrelated identity.

Errors and Retries

Identity Security Cloud automatically recognizes some aggregation error messages from source connectors, such as "Connection Reset" and "Failed to connect to server", as retryable errors. When Identity Security Cloud receives a retryable error during aggregation, it will retry the action. You can configure additional retry errors for a source by using a PATCH request to add the aggregationRetryErrors field to a source within its connectorAttributes. You can also add the retryWaitTime and maxRetryCount fields to configure the retry process.

You can use these three fields to configure a source's retry errors:

  • aggregationRetryErrors: The errors to retry.

  • retryWaitTime: The time span to wait before retrying. 5000 milliseconds is the default and recommended value. This value must be entered in milliseconds.

  • MaxRetryCount: The maximum number of retries. Three is the default and recommended value.

Some connectors do not follow these retryable error settings. For more information, refer to the connector’s documentation in IdentityNow Connector Guides.

Troubleshooting Common Aggregations Issues

The following list describes common aggregation issues that may occur and their typical causes:

When I import a delimited file, only a single entitlement is aggregated for each account.

Ensure that the delimited file is sorted by account ID.

Accounts that contain emojis within their names are not aggregating.

IdentityNow supports the UTF8MB3 character set, which excludes many emojis and some special characters. Refer to the Basic Multilingual Plane for more information.

When I attempted to aggregate a delimited source, I received a ‘123456789’ is not a valid phone number error message.

Ensure a country code is specified for the phone number.

An uncorrelated account did not correlate after running an aggregation, even after the correlation logic had been updated. After running a non-optimized aggregation, the account correlated correctly.

This is expected behavior. An unoptimized aggregation does not reevaluate correlation logic for accounts with no data changes. By running a non-optimized aggregation, correlation logic was reevaluated, and the account was correlated once more.