Partitioning Aggregation

The JDBC connector supports the manual partitioning through configured SQL statements.

The Partitioning Enabled configuration parameter must be selected and the list of SQL statements/parameterized stored procedures must be entered in the Partitioning Statements field.

For example, if there is an employee dataset that has 100,000 rows with a sequential employeeId field, the partitioning statements that can be used are as follows:

Copy 
select x,y,z from a where employeeId <= 10000;
select x,y,z from a where employeeId > 10000 AND employeeId <= 20000;
select x,y,z from a where employeeId > 20000 AND employeeId < =30000;
…
select x,y,z from a where employeeId > 90000;

The above example has 10 partitions, handling approximately 10,000 accounts. The last row (with employeeId > 90000) handles a larger number of accounts depending on the total number of employees in the system.

Note
An additional aggregation option, noAttributePromotion, has been added. If this attribute is set to true, the attribute promotion is skipped during aggregation.