Looking for help managing rollups in my org!

[GithubUser1892]

Hello,

I've recently started using Apex Rollup and would like some guidance on the best way to maintain these in my org.

I have set up two scheduled flows, one at 1 AM and another at 4:30 AM. I made sure not to choose an object on the scheduled flows and am using the Full Recalc CMDT-driven Invocable action and a text template with the list of rollups.

I need some rollups to be done before the second set are rolled up, which is why I set them to run at different times.

The issue I'm running into is:

Failed to process batch for class 'RollupFullBatchRecalculator' for job id '707Mm000003pKfo'

caused by: System.LimitException: Apex heap size too large: 12230504

External entry point
Class.RollupLogger.log: line 44, column 1
Class.RollupLogger.CombinedLogger.log: line 186, column 1
Class.RollupAsyncProcessor.execute: line 231, column 1

I looked at similar posts and lowered the Batch Chunk Size to 200. There were 21,163 total batches and 6,964 batches processed.

The idea is that the first set of rollups would be finished before the second set ran, but since the first set started at 1 AM, I had to abort it 11 hours later as it was still running. The two scheduled flows would be ideally ran every night.

Thanks for your help!

[jamessimone]

Hi there - thanks for reaching out. I guess my first question would be "why do you need to do full recalcs every day?"

The intent of the full recalculation is to get data prepped in the org so that flow/Apex can then be responsible for maintaining the state of rollups in a much, much less process-intensive way. While I have tried to support the possibility of performing full recalculations across a broad swathe of different functionality, the reality is that for orgs with LDV and many rollups (which it sounds like you may have both, given the heap size issue with a batch size of 500), it simply takes time for batch processes to run through all of the chunks in general.

As well - I would recommend disabling logging in production. It is the single largest use of resources when it comes to Apex Rollup, and especially for full recalcs, it's simply not an efficient usage of your org's limits. I would guess that with logging disabled, you'll see things run much more smoothly, and almost certainly would be able to re-up the Batch Chunk Size as well.

Lastly - are you using the latest version of Apex Rollup? Knowing which version you're on will help immensely; I've made some major improvements as of late to the full recalculation code paths, and if you're on an earlier version, upgrading will certainly help.

[GithubUser1892]

I've made that update! Any luck on the Apex heap size too large exception?

Or do you think if I manually run the recalculations through the Recalculate Rollups tab that should work for the one-time setup?

[jamessimone]

That change will remove the Apex heap size being too large exception entirely because the RollupParentResetProcessor won't be in use - I haven't had the chance yet to take a look through the latest logs you've provided, but I am hopeful to be able to do so later today or this weekend.

[jamessimone]

@GithubUser1892 in the event that updating the flag doesn't completely solve the heap size issue, I've identified some additional heap savings that are (presumably) related to the error you were getting, which I intend to ship in #431 (hoping to release that today)

[GithubUser1892]

That's great! I'll update to the newest release as well!

Another question I had, is it okay to include rollups from different objects in the same Full Recalc CMDT-driven Invocable?

[jamessimone]

Yes, it's perfectly okay. Depending on the number of children records for each full recalc you're using, if Apex Rollup sees that the total number of children is over the Max Lookup Rows Prior To Batching number on the org default Rollup Control record, a batch job will be spawned per child object type. These batch jobs are processed serially - that is, one at a time - so that's just something to be aware of if you're using the Apex Jobs page in setup to track their progress.