Aggregation Out of Memory Error and Resolution

If your JDBC source has to aggregate a large amount of data, it may encounter an out of memory error. To mitigate this issue and aggregate the data, you can perform one of the following solutions:

  • Use the TYPE_FORWARD_ONLY ResultSet type for JDBC drivers that support it.
  • Increase RAM of the Java heap size (memory) in the Java virtual machine for JDBC drivers that do not support the TYPE_FORWARD_ONLY ResultSet type.

JDBC drivers that support the TYPE_FORWARD_ONLY ResultSet type read the data in the forward direction only which reduces data caching and results in lower memory consumption. Use the SailPoint REST API to update JDBC drivers that support the TYPE_FORWARD_ONLY ResultSet type.

Note
For more information on SailPoint's REST APIs, refer to Best Practices: REST API Authentication and REST API - Update Source (Partial) in the SailPoint Developer Community.

Use the following attribute keys:

Attribute Keys

Suggested Values

resultSetFetchSize

1000

statementFetchSize

1000

resultSetType

TYPE_FORWARD_ONLY

For JDBC drivers that don't support the TYPE_FORWARD_ONLY ResultSet type, increase the RAM allocated for the Java heap size (memory) in the Java virtual machine. To increase the allocated RAM:

  1. Create a text file and name it java_overwrites.yaml. Copy the following parameters into the file.

    Copy 
    #Xmx: 912m
    #DbsfPoolSize: 5

    Currently, the virtual appliance only supports the following CCG JAVA HEAP options:

    • Xmx

    • DbsfPoolSize

  2. Uncomment the two lines #Xmx: & #DbsfPoolSize by removing the #. Then configure your required values and save the file.

    Note
    The default value for Xmx , 912m, represents 912 MB. This parameter can only be configured in megabytes. For example, if you want to configure 24 GB, set Xmx to 24000m.

    For example:

    Xmx: 24000m

    DbsfPoolSize: 5

    Caution
    Always leave at least 3GB of RAM on the virtual appliance to run non-CCG processes.

  3. Copy the java_overwrites.yaml file to the virtual appliance in the following directory:

    /home/sailpoint/ccg/

    Note

    On newer CCG images, CCG is owned by root. To add the java_overwrites.yaml file, you may need to run the following command:

    sudo /usr/bin/chown sailpoint /home/sailpoint/ccg

  4. Restart CCG by running the following command:

    sudo systemctl restart ccg

  5. Confirm the ccg.log, found at /home/sailpoint/log/ccg.log does not contain errors related to the new configuration.