Aggregation, Filter, and Partitioning Settings

This section contains the configuration parameters for aggregation, filter, and partitioning settings.

For more information about aggregation, refer to Loading Account Data.

  1. Select the Aggregate All Groups checkbox to aggregate Security, Microsoft 365, Distribution List, and Mail Enabled Security Groups. If this option is cleared, only Security Groups are aggregated.

  2. Select the Delta Aggregation checkbox to aggregate delta changes for accounts and groups.

  3. (Applicable for B2C tenant only) Select the Aggregate B2C Membership checkbox to fetch B2C user memberships for all users.

  4. Enter the value of Page Size for the number of objects to be fetched in a single page when iterating over large data sets.

When configuring the filters, consider that the connector prioritizes account filters over group filters during aggregation. For example, the connector aggregates groups, which fall outside of the group filter, if the group is associated with an account included within the account filter.

Note

  • If the filter (either Group Filter or User Filter) uses any advanced query filter (per the Microsoft documentation), then Advanced User Filter and Advanced Group Filter should also be selected.

  • The connector supports advanced query filters like endsWith, NOT, and NE during aggregation. The Azure API does not support the advanced query filters while also using an expanded attribute such as manager in the URL. When using the advanced filters, ensure that you remove the manager attribute from the account schema and remove the owners attribute from the group schema.

  • If you're using the advanced filters you must add the supportsAdvancedAccountFilter attribute to the source XML using the Identity Security Cloud REST API. The connector will then automatically add the required header (ConsistencyLevel:eventual) in the header and add &$count=true

    key: supportsAdvancedAccountFilter

    value: true

  1. Set the User Filters to define the scope of Accounts applied during account aggregation. For example, to aggregate those Microsoft Entra ID users who are active, use accountEnabled eq true. For more information on filters, refer to the Microsoft documentation.

  2. Select Advanced User Filter to include advanced filter queries such as endsWith, NOT, and NE during aggregation processes.

  3. Set the Group Filters to define the scope of Groups applied during group aggregation. Group filters apply during entitlement aggregation. For example, to only aggregate Microsoft Entra ID groups whose display name is starting with letter A, use startswith(displayName,'A'). For more information on filters, refer to the Microsoft documentation.

  4. Select Advanced Group Filter to include advanced filter queries such as endsWith, NOT, and NE during group aggregation processes.

  5. In the Directory Roles Filter field, when the entitlement schema "roles" is present in the source, enter filter statements to ensure that only the specified roles are aggregated during entitlement aggregation. For example, isBuiltIn eq true. For more information on filtering conditions and values, refer to the Microsoft documentation.

  6. In the Application Role Filter field, enter any filter statements to apply during entitlement aggregation. An application role is an entitlement object that captures the details related to the AppRoles defined within the scope of enterprise applications. For example:

    • For a filter to match the Enterprise Application default view on the Azure portal use the following:

      tags/Any(x: x eq 'WindowsAzureActiveDirectoryIntegratedApp')

    • For a filter to exclude Microsoft's built-in service principals use the following:

      appOwnerOrganizationId ne f8cdef31-a31e-4b4a-93e4-5f571e91255a&$top=100

  7. Select Save.

  1. Select Enable Partitioning if you want to process data in parallel across multiple threads.

  2. In the Partitioning Scheme field, enter user filters to define your aggregation partitions. Enter a supported user filter and then press Enter. You can add multiple filters using this method. The filter you define in Partitioning Scheme should evenly distribute all users to be managed across partitions.

    This is a multi-valued attribute. The value consists of different search filters for the attributes that can be filtered (for example, accountEnabled, city, displayName, mail, usageLocation, etc.). The partitioning scheme is a list of user filters where each user filter represents one user partition, and the partitioning scheme defines the scope of accounts applied during account aggregation. For more information, refer to the Microsoft Documentation on using query parameters.

    The following are some example Partitioning Scheme entries based on how you want to configure the partitions:

    • The following examples show you configurations based off of alphabetical inputs: 

      • startswith(displayName, 'A')

      • startswith(displayName, 'B')

      • startswith(displayName, 'C')

      • startswith(givenName, 'Smith')

    • You can also define schemes using other attributes:

      • accountEnabled eq true

      • userPrincipalName eq 'Paul@contoso.onmicrosoft.com'

  3. Enter the Number of Partitioning Threads the connector should use to concurrently execute your defined partitions. By default, it is set to 2 partitioning threads. If the VA is configured for multiple CPU cores, you can configure a higher number of partition threads to increase the performance.

    Note
    The Number of Partitioning Threads should equal the number of CPU cores on VA*2.

  4. Select Save.