Identity Refresh

Refresh identity tasks scan all identities to ensure that all identity information is up-to-date and accurate. Refresh identity scans are also used to detect and report on policy violations and launch event certifications.

Incremental identity refresh can be configured to only refresh those identities on which information has changed since the last refresh was performed, to increase performance.

Note

Partitioning is disabled if you enable Mark dormant scopes after refresh or Refresh the group scorecards options.

Note

The Number of Refresh Threads option is not supported when partitioning is enabled.

Partitioning is available to speed the processing time for identity refresh tasks and level the load on the machines running these tasks. Partitioning is used to break operations into multiple pieces, or partitions. Each partition is then placed in a global queue, and machines, or hosts, in a cluster compete to execute the partitions in the queue. Machines are added or removed from the cluster dynamically with automatic balancing. If a machine fails or is taken down while processing a partition, the partition is placed back into the queue and reassigned to a different machine. A single result object is shared by all partitions and is continually updated so you can monitor the overall progress of the partitioned operation. When all partitions have finished executing the result is marked complete. See Partitioning.

The information scanned and updated is determined by the following criteria when the task is created or edited. You can use any combination of options to build a task.

To reduce or eliminate the possibility of getting an ObjectAlreadyLocked exception, there are additional parameters available on the IdentityIQ debug pages. enableTriggerIdentityQueue, set to true to enable the queuing feature, and triggerIdentityQueueSize, to specify the number of triggers to queue prior to processing, without this setting, the default is 10.

Option	Description
Optional filter string to constrain the identities refreshed	A filtering string used to limit the number of identity cubes updated by this task. For example you can limit the refresh to one department within your enterprise, such as Finance, by entering: department == "Finance"
Optional list of group or population names to constrain the identities refreshed	A filtering string used to limit the number of identity cubes updated by this task. For example you can limit the refresh to one group or population within your enterprise.
Refresh identities whose last refresh date is before this date	Refresh any identities not refreshed since the date entered. Enter and date manually or click the [...] icon to display the calendar view. Use this to recover from a refresh that ended abnormally. For example, you start a refresh task and it runs for a day before stopping abnormally. After resolving the issue with the task, instead of repeating the refresh of all the identities that completed before the task stopped, you can only refresh the ones that were missed on the last refresh. Enter the approximate date the last refresh stopped and only refresh the remainder.
Refresh identities whose last refresh date is at least this number of hours ago	Enter the number of hours manually. Use this option to refresh identities that have not been refreshed recently. The time is in this option is relative rather than absolute. Instead of remembering a specific task launch date and typing that in each time you run the refresh task you can have just one task and run that repeatedly. For example you can run it for every thing more than an hour old.
Refresh identities whose last refresh date is within this number of hours	Enter the number of hours manually. Use this option to refresh identities that were refreshed recently. The primary use case for this is to refresh things that were recently touched by aggregation. For example, if you have several aggregation sources but those sources tend to touch different subsets of all identities, and you would rather not refresh the identities that were not touch be the last aggregation.
Include modified identities in the refresh window	Refresh any identities modified within the specified time frames. There are two dates stored on each Identity, the date of last refresh and the date of last modification. The last refresh date is set whenever you run the refresh or aggregation tasks and the identity is changed in some way. The last modification date is set whenever you edit the identity in some way outside of a refresh or aggregation task, for example from a Lifecycle Manager workflow or a custom task. Use this option to refresh identities that were edited within a period of time, but not necessarily by the refresh task. For example, you might do a full refresh once a week but during the week people were adding or removing roles, changing extended identity attributes, doing manual correlation, or changing identities in some other way. Most of those cases have options to do a targeted refresh immediately after the change happens but this is not always the case and sometimes it is better to batch up a number of refreshes rather than have hundreds of individual refreshes occurring concurrently. If you ran the refresh task with one of the date-based options you would not necessarily pick up identities that were manually edited. If you want to include those select this option.
Refresh only identities marked as needing refresh during aggregation	Only refresh identities marked as needing refresh during the most recent aggregation task. For more information on using this option to optimize performance, see Refreshing Changed Identities Only (Delta Identity Refresh).
Do not reset the needing refresh marker after refresh	Do not clear the needing refresh marker set during aggregation. Use this option if you have multiple refresh tasks scheduled, such as entitlement and risk refresh. Then you can set the final refresh to clear the markers.
Exclude identities marked inactive	Exclude inactive identities from the refresh.
Refresh identity attributes	Update Identity Cubes with any changes made to the attributes used to define identities.
Refresh Identity Entitlements for all links	Refresh any account attribute mark as an entitlement in the application schema. This process is resource intensive as it refreshes all entitlement values for all links.
Refresh manager status	Update all Identity Cubes in which the manager status has changed. For example, if a user was promoted to manager in their department, their Identity Cube would be updated by this task.
Refresh assigned and detected roles and promote additional entitlements	Update any assigned or detected role assignments that have change since the last time this task was run. Any additional entitlements found in this refresh are promoted during this task.
Provision assignments	Provision any assigned roles and entitlements detected since the last time this task was run.
Disable deprovisioning of deassigned roles	Prevents assigned roles from being deprovisioned after they have been deassigned.
Refresh role metadata for each identity	Update information about the identity's relationship to their role. For example, information regarding whether or not an identity has all the roles required by the given role. Note: This option must be selected in order to generate Role Statistics.
Enable manual account selection	Sent Account Selection Notification emails to users with more than one account on any application where the system cannot determine the provisioning account. By default, no provisioning is done in this case.
Synchronize Attributes	Provision identity mapping targets if their value has changed.
Refresh the identity risk scorecards	Update Identity Risk Scores with any information discovered by the scan performed by this task.
Maintain identity histories	Update the identity history by creating a snapshot of any identities with information that has changed since the last refresh.
Refresh the group scorecards	Update Group Risk Scores with any information discovered by the scan performed by this task. Partitioning is disabled if you select this option.
Clean up groups definitions that are no longer referenced	Delete unreferenced group definitions. This option is only supported if it is selected in conjunction with the Refresh the group score card option and they run in the same task.
Check active policies	Scan for active policies and apply those policies to the identities included in the task.
Keep previous violations	Maintain a history of violations that are no longer active.
A comma separated list of specific policy names. When set this overrides the default policies	Scan for and apply only those policies included in this list to the identities included in the task.
Refresh assigned scope	Refresh assigned scope based on changes discovered.
Disable auto creation of scopes	Do not automatically assign scope to identities as part of this task.
Mark dormant scopes after refresh	Mark scopes that are not assigned to any identities as dormant. Partitioning is disabled if you select this option.
Process Events	Enable event certifications. Use the snapshots created during aggregation to approximate the previous state of the identities at the beginning of the refresh. This copied identity is compared to the updated identity to determine if event certifications are launched.
Disable identity processing threshold	Identity processing thresholds let you stop lifecycle events before they are fully processed to prevent any dangerous workflows from accidentally being triggered. They can ben enabled in Rapid Setup events and in Lifecycle and Certification events. If identty processing threholds are enabled, use this field to disable the identity processing threshold for this task. See Using Identity Processing Thresholds for Error Prevention.
Refresh logical application links	Scan for changes to composite applications and refresh the link information.
Promote managed attributes	When enabled, any values for entitlement or permissions encountered while running the task automatically get promoted as managed attributes
Number of Refresh Threads	Specify the number of concurrent threads used during task processing. The number of threads should not exceed 10. This option is not supported with partitioning enabled.
Always launch the workflow (even if the usual triggers do not apply)	Launch a workflow for each identity even if no identity triggers or provisioning policy questions apply.
Enable the generation of work items for unmanaged parts of the provisioning plan	Create work items for role entitlements that are not managed by available connectors or provisioning integration modules so the appropriate action can be taken.
Disable connector lookup of managers that do not correlate	Disable the default MANAGER_LOOKUP feature and stop the automatic lookup/bootstrap of the manager account at the connector level.
Enable partitioning	Enable partitioning of this task across multiple hosts. Partitioning must be configured globally before this option can be used. See Partitioning.
Number of partitions	Specify a number of partitions. If no number is specified, IdentityIQ calculates an optimal number based on available request servers.
Loss Limit	The loss limit sets the maximum number of identities that will be reprocessed in case of a sudden termination of a partitioned refresh. This option is used only when partitioning is enabled. See Loss Limits.
Do not schedule retry requests during application maintenance windows	Disables the scheduling of provisioning retry requests, when provisioning fails due to an application being within a maintenance window. Application maintenance windows can be set for each application. See Using the Edit Application Page.

Refreshing Changed Identities Only (Delta Identity Refresh)

A "delta" identity refresh lets you update only those Identity Cubes that have changed since your last aggregation(s), rather than updating all identities. This can result in a significant reduction in refresh time, and can remove or reduce the need to partition your identities into subsets for efficient refresh processing.

In most cases, identities which have had no changes to their attributes or accounts ("link" objects) as a result of aggregations are not likely to have new policy violations or need new workflows launched to handle state changes. These identities can therefore be skipped or not processed by the Identity Refresh task. In contrast, identities that have undergone some kind of change, referred to as some kind of "delta," should be processed by the Identity Refresh task.

IdentityIQ lets you set up your tasks to refresh only the changed or delta identities; this is a two-step process:

1. Configure and run an aggregation task to mark identities as changed when attribute or account data on the identities has been modified.
2. Configure and run an Identity Refresh task to perform their functions only on the marked identities.

Marking Identities as Changed

During an aggregation, details of some identities are changed, while some others may not be. IdentityIQ's aggregation tasks include a setting that lets the task flag any identities updated by the aggregation as needing a refresh. This lets you single out only updated identities for a refresh when the Refresh Identity Cube task is run. The default behavior of aggregation task is to set this flag; if you don't want an aggregation task to flag identities that need a refresh, you can turn this option off.

During the aggregation task, IdentityIQ marks the identities that have changed by setting the attribute needsRefreshto trueon the changed identities as they are updated. This is a default operation performed in all aggregation tasks, although it can be turned off with an option on each of the aggregation tasks if desired.

This needsRefresh flag can then be used by the Identity Refresh tasks to target only those identities with accounts that were modified in a recent aggregation. The refresh tasks can then reset that flag to false when they are done with the identities so that subsequent aggregations can set the flag anew, and subsequent refresh cycles will again only pick up changed identities.

If you want to use this delta identity refresh feature, you should carefully consider which attributes you choose to aggregate from your applications into IdentityIQ. Aggregating attributes such as last login date, for example, would likely cause IdentityIQ to reflect changes to identities more frequently than choosing to aggregate only more static data fields, and would therefore flag more identities for delta refresh.

Note that aggregation is the only process which automatically sets this needsRefreshflag on identities. If other processes (such as Lifecycle Manager requests) make attribute or account changes to an identity which would affect identity refresh functionality, a full refresh that does not rely on this flag would be required to process those other identities' changes. Alternatively, the Lifecycle Manager workflows also include an optional targeted identity refresh step which could refresh the single changed identity immediately, and could be configured either to clear or not clear that identity's current needsRefreshflag value at that time.

To set up an aggregation task that will mark identities that have changed:

1. Click Setup > Tasks.

2. Choose the aggregation task to edit.

3. Uncheck the Disable marking the identity as needing a refresh option.

4. Save the task.

Refreshing Only Identities Marked As Changed

In the Refresh Identity Cube task (or other refresh tasks), select the option to Refresh only identities marked as needing refresh during aggregation. It is important to note that this operation is disabled by default; that is, default behavior for Identity Refresh tasks is to ignore the needsRefresh flag that was set by the aggregation task. If you want to use the delta identity refresh feature, you have to explicitly set this option in your refresh task(s):

1. Select Setup > Tasks.

2. Choose the refresh task to edit.

3. Check the Refresh only identities marked as needing refresh during aggregation option.

4. Save the task.

When the refresh task runs, it resets the needsRefresh flag to false for every identity it processes. This way, IdentityIQ knows that the identity has been refreshed already and so will not refresh it again until the next aggregation. However, you can change this behavior if you want. Depending on how you run refresh tasks, you may or may not want to reset this flag.

For example, if you aggregate and refresh infrequently, it can be a good practice to have the refresh task clear the needsRefresh tag, to avoid needlessly repeating refreshes on an identity that has just been refreshed. However, if you segment the refresh task to, for example, split out the refreshing of entitlements, attributes, and policies, you would not want to clear the needsRefresh tag. Leaving the needsRefresh tag in place as you iterate through all the refresh segments lets you avoid a situation where an identity is updated only for one segment of the full refresh process, rather than all segments that might apply.

To configure the refresh task so that it does not clear the needsRefresh flag from an Identity Cube when it runs:

1. Select Setup > Tasks.

2. Choose the refresh task to edit.

3. Check the Do not reset the needing refresh marker after refresh option.

4. Save the task.

Best Practices for Delta Identity Refresh

Delta identity refresh offers flexibility for managing the important identity refresh functions more efficiently. Here are some recommended best practices for using this functionality.

Interspersing Delta and Full Refreshes

Delta identity refresh can be a helpful performance boost for IdentityIQ, as it streamlines time-intensive identity refresh processes. It is still strongly recommended, however, that customers regularly run a full refresh to ensure that all identities get updated and processed following other changes which may occur outside the data flows which would trigger identities to be marked as needing refresh. For example, Lifecycle Manager request workflows may or may not refresh identities after entitlement and role assignments, depending on options configured in the workflows, and they do not set the needsRefresh flag by default; a full refresh on a periodic basis would catch those identities and update them as needed.

Different customers run refreshes on different schedules, often depending on the volume of ongoing changes they anticipate or experience in their environment. For example, customers who run an hourly delta refresh should plan to run a daily full refresh, while customers with lower change volumes may run daily delta refreshes with weekly full refreshes.

Timed Triggers

It is common for installations to define workflow triggers (IdentityTrigger objects) which are set to run based on the calendar date when the trigger executes. For example, a "pre-offboarding" workflow could be configured to run the day before a user's termination date or a "pre-onboarding" workflow could run a day or a week before the user's start date, with those dates being specified through attributes in an HR or authoritative system feed. The Identity Refresh task that launches those workflow triggers (through its "Process Events" option) should be configured as a full refresh which processes all identities, not a delta refresh handling only identities that need refreshing; the Identity's attributes or accounts might not change in any aggregation run on the day these events need to execute, so this workflow needs to evaluate all identities, not just recently changed ones.

Delta Identity Refresh and Logical Applications

Logical applications in IdentityIQ are applications which are defined based on accounts, entitlements, and / or permissions granted through one or more other applications. For example, Application X, for which access is controlled by membership in the AppX group in LDAP, could be configured as a logical application in IdentityIQ.

Logical application "aggregation" is actually performed by processing information already stored in IdentityIQ for other applications (for example, to identify Application X accounts and entitlements, IdentityIQ looks at the LDAP application accounts with the AppX group), rather than by reading data from an external source like traditional application aggregations do. Standard applications, with a data source external to IdentityIQ, support an optimized aggregation process in which IdentityIQ skips the bulk of processing on any records it finds to be unchanged as it reads the source. Because the data source for logical applications is existing identity data which must be dynamically examined and assessed for logical accounts based on the logical application definition, there is no built-in option for optimizing aggregation from logical applications.

Delta identity refresh offers functionality which can be used to approximate an "optimized refresh," so it is possible to use this feature to manage a more optimized aggregation of logical applications. These are the steps to implement that option:

1. Run an optimized aggregation from the tier applications in the logical application definition. This aggregation will then mark only the changed identities from those applications with the needsRefreshflag. For best results, this should be run at a time when there are no identities previous marked as needing refresh, that is, a full or delta refresh process has reset the needsRefresh flag on all marked identities prior to this aggregation.)

2. Run a delta identity refresh task to recalculate the logical application accounts. Since it will only look at the identities whose accounts changed in the optimized aggregation, it will only recalculate the logical accounts for those identities. This refresh should have these options set:

   • Refresh only identities marked as needing refresh during aggregation -- enable this option to have the task process only identities which were marked as changed in the optimized aggregations in step 1.

   • Refresh logical application links -- enable this option to use this refresh for logical application account calculation.

   • Edit the task's definition in the Debug pages to add a **compositeApplications**key, set to a comma-separated list of logical application names which this refresh should process; this can be a single logical application at a time or a limited list so the task only processes the logical application(s) which are based on the tier applications just aggregated