Skip to content

Loading Account Data

Loading account data into IdentityNow aggregates source accounts and their associated access into the system so they can be governed. To load or aggregate account data into IdentityNow, you need to create one or more sources, which can be either direct connections or flat file feeds.

Account data for flat file sources can be manually uploaded at any time to aggregate changes to those accounts. While you can't load flat file feeds for multiple sources simultaneously, you can load them consecutively.

If you start aggregations for multiple direct connect sources, IdentityNow will place them in a queue and process them in the order that you load the files or start the aggregations. You can monitor the aggregation progress on the System Activity panel of the Admin Dashboard.

You can also configure supported sources to use delta aggregation, which updates only those accounts on the source that have changed. For more information, read Configuring Delta Aggregations for Supported Sources.

Configuring a Source

Choose the type of source you will use as a source of account data and refer to the documentation specific to that type for detailed instructions on how to configure it in IdentityNow:

  • direct connection is a method of communicating directly between a source server and IdentityNow. To set up a direct connection between IdentityNow and a source server, you'll provide essential connection information specific to the source.

    To access the documentation for a specific supported direct connect source, refer to the list of Supported Connectors for IdentityNow.

  • flat file feed is a CSV file that contains all relevant information about the accounts you want to load into IdentityNow. When you create a flat file source, you load the account information by importing the file.

    For more information about loading data into IdentityNow from a flat file, refer to the Flat File Source Configuration Guide.

Note

To enable others in your organization to create and manage sources, you can grant individual users either the source admin or source sub-admin user level.

Source admins can create, configure, manage, and edit sources. Source sub-admins can perform these source actions only on the sources associated with the governance group they are members of.

Next Steps

  1. After you've finished initially loading existing account data into IdentityNow, you must create identity profiles for the source to create IdentityNow accounts from the source data.

  2. After you've created at least one identity profile, you can change the pre-set or default correlation for the source to meet your needs. For more information, refer to Configuring Account Correlation.

After the initial automatic aggregation that IdentityNow performs to load account data from a configured source, you can manually aggregate account information as needed or schedule IdentityNow to aggregate data from any direct connect source on a regular basis.

Scheduling Aggregations for Direct Connect Sources

After IdentityNow performs an initial aggregation to load account data from a direct connect source, you have the option of scheduling future aggregations to automatically load new data on the source into IdentityNow on a regular basis.

You can also manually aggregate account information from a direct connect source as needed.

Aggregation Best Practices

When scheduling aggregations, we recommend you follow these best practices to help your aggregations run more quickly, with minimal impact to your organization.

  • Aggregate as infrequently as possible based on your business needs. It is almost never necessary to aggregate more often than once a day for most non-authoritative sources.
  • When possible, aggregate outside of business hours. Usually, this is at night.
  • Aggregate only what you deem is necessary and important. Use filtering on the connector if applicable to aggregate only the most recent records for active users.
  • If the source supports delta aggregation, use this option to lower your aggregation times and minimize load on the system.

Note

Using emojis in IdentityNow is not supported. Including emojis in account or entitlement aggregations can cause aggregations to fail.

To schedule aggregations:

  1. In the Admin interface, go to Connections > Sources and select the source you want to schedule aggregations for.

  2. Go to Import Data > Account Aggregation and select the Enable Schedule checkbox.

  3. Use the available options to schedule how often you want the aggregation to run and the time you want to start each aggregation.

    Note

    If you select Daily, you can use the Recurring Every dropdown menu to set an interval within the day. The start time and interval will determine the number of aggregations that will be performed each day. For example, if the Time is set to 2 pm and the recurrence is set to 8 hours, aggregation will occur twice each day. To aggregate once per day, set it to 24 hours.

  4. (Optional) Set the percentage of allowed deleted accounts per aggregation in the Account Delete Threshold section. If the expected deletions exceed this value, IdentityNow will not delete any accounts. We recommend you use this option to avoid removing user data in the event of a misconfiguration.

    Important

    Zero percent and 100 percent are equivalent and indicate that there is no delete threshold. Aggregations that delete large numbers of accounts from this source will complete without any warnings.

    Account Deletion Limitations
    • If a source has 10 or fewer accounts, setting this value to 4 percent or less will cause IdentityNow to round it to 1 percent to prevent all accounts from being deleted.
    • If a source has 11 - 20 accounts, setting this value to 2 percent or less will cause IdentityNow to round it 1 percent to prevent all accounts from being deleted.
  5. Select Save to schedule aggregations for the source.

    Note

    If you need to cancel an aggregation after it has started, you can do so from the Aggregation Activity Log. This is helpful if you encounter a problem during an aggregation that might take several hours to complete.

Manually Aggregating Information from a Direct Connect Source

The first time you load account or entitlement information from a direct connect source, that information is aggregated automatically. After the initial automatic aggregation, you can manually aggregate accounts and entitlements from the source as many times as you need, using the Source Aggregation tab.

Note

  • If you are running a manual aggregation on entitlements due to an error with unrecognized entitlements in a certification campaign, the manual aggregation alone may not fix the error, but can tell you what the error is so that you can take steps to address it.

    Alternately, you may need to generate another campaign after remediation to confirm whether or not the error with entitlements was resolved. If it wasn't resolved, the entitlements will continue to show up in campaigns.

  • Entitlements can be deleted based on whether or not they are present in aggregations. Visit Deleting Entitlements for more information.

To manually aggregate account information:

  1. In the Admin interface, go to Connections > Sources and select a source.

  2. Select Import Data > Entitlement Aggregation or Account Aggregation based on the type of information you want to aggregate from the source.

    Important

    If the Enable Schedule option is checked, we recommend you make note of the current settings as they will be reset when you uncheck this option to perform a manual aggregation.

  3. (Optional) Set the percentage of allowed deleted accounts per aggregation in the Account Delete Threshold section. If the expected deletions exceed this value, IdentityNow will not delete any accounts. We recommend you use this option to avoid removing user data in the event of a misconfiguration.

    Important

    Zero percent and 100 percent are equivalent and indicate that there is no delete threshold. Aggregations that delete large numbers of accounts from this source will complete without any warnings.

    Account Deletion Limitations
    • If a source has 10 or fewer accounts, setting this value to 4 percent or less will cause IdentityNow to round it to 1 percent to prevent all accounts from being deleted.
    • If a source has 11 - 20 accounts, setting this value to 2 percent or less will cause IdentityNow to round it 1 percent to prevent all accounts from being deleted.
  4. Select the Start button next to Manual Aggregation.

An entry is added to the Aggregation Activity Log with a status of Pending. The Date column indicates the date and time the aggregation was started. When the aggregation is complete, the Status column will indicate if the aggregation completed successfully.

Note

Aggregations can take two hours or longer depending on the size of the source. During this time, the Start button will be disabled. If you need to cancel an aggregation after it has started, you can do so from the Aggregation Activity Log.

Configuring Delta Aggregations for Supported Sources

When IdentityNow performs a delta aggregation on a supported source, it only loads the accounts that have been created or changed since the most recent aggregation. For example, if one of your users changes their phone number in Active Directory, you can aggregate account information for that user only — instead of all of the accounts on the source. This can speed up the process of loading changes from your source.

Account deletions will be handled during the next full aggregation.

Note

Delta aggregation does not support OU moves. If you perform any type of OU move, you need to perform full account aggregations to avoid unexpected behavior such as the creation of duplicate accounts for the same user.

Supported Sources

The following sources support delta aggregation:

Microsoft Active Directory IBM Tivoli Directory Server SAP Direct
Microsoft Azure Active Directory JDBC SAP HR/HCM
Microsoft Lightweight Directory Services Okta SCIM 2.0
Google Workspace Oracle SunOne ServiceNow Identity Governance
HCL Domino Oracle Fusion HCM Workday

To configure delta aggregations on a supported source:

  1. In the Admin interface, select Connections > Sources.

  2. In the Config tab, select Advanced Options.

  3. Under Delta Aggregation Mode, select the Enable option.

    Note

    For JDBC, you must select both of these options: Delta Aggregation Mode and Enable Account Delta Aggregation.

Aggregating Account Information on a Direct Connect Source Using APIs

If you prefer to aggregate a direct connect source using the available REST API, you can do so using the loadAccounts API.

Note

To use APIs, you'll need to use one of our supported authentication methods. As a best practice, SailPoint recommends using OAuth 2. Also, the endpoint may vary slightly based on the authentication method. For details, refer to the supported Authentication Methods.

  1. Sign in to your org as an administrator.

  2. In the Admin interface, go to Connections > Sources and select the source you want to aggregate.

    The source ID is displayed at the end of the URL in your browser address.

  3. Make note of this number, as you'll need to refer to in the next step.

  4. Use your preferred tool to call the following API:

    POST <api-url>/cc/api/source/loadAccounts/<source_id>

    where

    <api-url> is the URL to the API on the IdentityNow org (e.g., https://org-name.api.identitynow.com).

    <source_id> is the ID of the direct connect source you want to aggregate.

To ensure that every account is scanned when you aggregate the source :

If you have changed the correlation configuration for the direct connect source, you can disable optimization by including the following key-value pair in the Body of the call, as form-data: disableOptimization=true.

Note

This parameter setting applies to the current aggregation only. Subsequent aggregations performed using the API will have optimization enabled unless you set this flag to true each time. Scheduled or manual aggregations performed using the UI will always run with optimization enabled.

For more information on using IdentityNow REST APIs, refer to this Wiki article and the Developer portal.

Viewing the Aggregation Activity Log

Whenever a source is being aggregated, a new entry is added to the Aggregation Activity Log, along with the name of the admin user who started it (for manual aggregations), the date and time the aggregation occurred, and its current status. This is helpful for tracking the progress of an aggregation that is still processing and viewing detailed information about the aggregation.

Note

You can also cancel an aggregation that is in progress from the Aggregation Activity Log.

  1. In the Admin interface, go to Connections > Sources and select the source you want to view aggregation activity for.

  2. If it's a flat file source, select Import Data.

    If it's a direct connect source, select Import Data > Account Aggregation.

    Select the frequency (daily, weekly, or monthly), and starting and recurring times, or select Start to run a manual aggregation.

  3. In the Aggregation Activity Log, select the Info icon to display detailed status information, such as how long it took for the aggregation to complete and the number of retries that were attempted before the aggregation completed successfully.



    In the case of an unsuccessful aggregation, information that might help you diagnose the issue that caused the aggregation to fail is provided.

  4. Select Close when you are done viewing the detailed information for the aggregation.

Canceling an Aggregation

To cancel an aggregation that has already started, select the X next to its Pending status, then select Yes in the confirmation message that's displayed.

Aggregation Activity Log entry showing the user, date, accounts scanned, and pending status.

The status of the aggregation changes to Terminated.

Select the Info icon for the aggregation to view the status, starting date and time, duration, accounts scanned, optimization (enabled), and accounts deleted for the termination. You can also view the name of the user who terminated the aggregation in the warning message.

Canceled aggregation info. A warning states a user has terminated this task.

Aggregating Data for a Single Account

If you are a Helpdesk admin or an administrator, you may need to reload or aggregate the data from a single account to have the most current information. For example, if a user calls the Helpdesk for help unlocking their account, that status may not display immediately. Aggregating that person's account gives you access without taking the time and resources to update the source's entire contents.

  1. In the Admin interface, select Identities > Identity List.

  2. Select the name of the user whose account you want to aggregate.

  3. Select Accounts.

  4. Select the Actions icon on the account you want to unlock and choose Aggregate Account.

This action initiates a refresh of the related identity's data, but it does not synchronize the new data with the identity. It also does not have any impact on provisioning. It only loads any changes from the account source data.

For more information about the identity refresh process, refer to Updating Identity Data.