MongoDB Error - Command aggregate failed: $addToSet used too much memory and cannot spill to disk

nayan · December 5, 2023, 5:25pm

Problem

If you use ZappySys MongoDB Source to call MongoDB Aggregate Pipeline Query then possible that you might get an error like below if the query needs more memory to process.

Command aggregate failed: $addToSet used too much memory and cannot spill to disk. Memory limit: 104857600 bytes

Possible Solution

To fix this issue add AllowDiskUse:true option as below

{Table: ZipCodes, Method: aggregate, AllowDiskUse:true}
{
  pipeline: 
  [
    {$match : { status: "A" } } ,
    {$group : {_id :  "$state", TotalPopulation: { $sum: "$pop" }}},
    {$match: { TotalPopulation: { $gt: 25 } } },
    {$project : { StateAbbr: "$_id" , _id: 0, TotalPopulation:1 } },
    {$sort : {TotalPopulation : -1}}
  ]
}