Error on server console: UnhandledPromiseRejectionWarning: RangeError: Maximum call stack size exceeded
Meteor 1.8.1
Project worked fine at 1.6.x, MongoDB 3.2.x
mlab pushed me to mongo 3.6.x and now some of my charts (chart.js) and tables (aslagle:reactive-table) aren’t working.
Both of these are generated via aggregate/pipeline (example below).
Long story short, I’ve neglected this project for several months and just getting back to it, ran into this issue, and tried updating to 1.8.1. Which has me at mongo 4.0.6
Charts and tables still not working and I think it’s because I’m missing some update to the syntax/structure in the aggregates and/or pipelines. I could very well be wrong here as I’m rusty, not a dev by trade and as mentioned haven’t touched this in a few months.
Here’s one failing aggregate/pipeline that’s used to draw a line chart (Chart.js):
pointTrendsSprt: function(uid, sp){
var pipeline = [
{ $match: {"userId": uid, "sport":sp, "slate": {$exists: true} } },
{ $project: {slate:1, sport:1, pointsWon:1, createdAt:1} },
{ $group: {
_id: "$slate",
sport: {"$addToSet": "$sport"},
pointsWon: {"$sum": "$pointsWon"},
createdAt: {"$max": "$createdAt"}
}
},
{ $sort: { "createdAt":1 } }
];
return Results.aggregate(pipeline);
},
Here’s another that’s used in a aslagle:reactive-table:
allXteam: function(uid){
var pipeline = [
{ $match: {"userId": uid, "win": "true"} },
{ $project: {team:1, match:1, play:1, Count: "$count"} },
{ $group:
{_id: "$team",
play: {"$addToSet": "$play"},
count:{$sum:1}}
}
];
return Picks.aggregate( pipeline );
},
Both of these are in a methods.js file in the server folder. I use Meteor.call on the client side js file to set session vars where the chart and tables pull from.
As mentioned, I think I need to fix something in my aggregations/pipelines but tried adding in “cursor” and “explain” and am either not doing that right or just wrong in that assumption.
Any suggestions/guidance would be greatly appreciated.
I found that this solution worked:
Collection.rawCollection().aggregate().toArray()
Here's my final re-factoring (although I will have to look into that 'Meteor.wrapAsync' form:
allXteam: async function(uid){
var pipeline = [
{ $match: {"userId": uid, "win": "true"} },
{ $project: {team:1, match:1, play:1, Count: "$count"} },
{ $group:
{_id: "$team",
play: {"$addToSet": "$play"},
count:{$sum:1}}
}
];
const aXt = await Picks.rawCollection().aggregate( pipeline ).toArray();
return aXt;
Related
Hello guys I'm writing a Message Application with Node.js and Mongoose. I keep datas in mongodb like that:
I want to list users who messaged before so I need to filter my 'Messages' collection but I can't do what exactly I want. If he sent a message to a person I need to take persons name but, if he take a message from a person I need to take persons name however in first situation person name in reciever, in second situation person name in sender. I made a table for explain more easily. I have left table and I need 3 name like second table.(Need to eliminate one John's name)
Sorry, if this problem asked before but I don't know how can I search this problem.
I tried this but it take user name who logged in and duplicate some names.
Message.find({$or: [{sender: req.user.username}, {reciever: req.user.username}]})
One option is to use an aggregation pipeline to create two sets and simply union them:
db.collection.aggregate([
{$match: {$or: [{sender: req.user.username}, {reciever: req.user.username}]}},
{$group: {
_id: 0,
recievers: {$addToSet: "$reciever"},
senders: {$addToSet: "$sender"}
}},
{$project: {
_id: req.user.username,
previousChats: {"$setDifference":
[
{$setUnion: ["$recievers", "$senders"]},
[req.user.username]
]
}
}}
])
See how it works on the playground example
This is a tricky one, but can be solved with a fairly simple aggregation pipeline.
Explanation
On our first stage of the pipeline, we will want to get all the messages sent or received by the user (in our case David), for that we will use a $match stage:
{
$match: {
$or: [
{sender: 'David'},
{receiver: 'David'}
]
}
}
After we found all the messages from or to David, we can start collecting the people he talks to, for that we will use a $group stage and use 2 operations that will help us to achieve this:
$addToSet - This will add all the names to a set. Sets only contain one instance of the same value and ignore any other instance trying to be added to the set of the same value.
$cond - This will be used to add either the receiver or the sender, depending which one of them is David.
The stage will look like this:
{
$group: {
_id: null,
chats: {$addToSet: {$cond: {
if: {$eq: ['$sender', 'David']},
then: '$receiver',
else: '$sender'
}}}
}
}
Combining these 2 stages together will give us the expected result, one document looking like this:
{
"_id": null, // We don't care about this
"chats": [
"John",
"James",
"Daniel"
]
}
Final Solution
Message.aggregate([{
$match: {
$or: [
{
sender: req.user.username
},
{
receiver: req.user.username
}
]
}
}, {
$group: {
_id: null,
chats: {
$addToSet: {
$cond: {
'if': {
$eq: [
'$sender',
req.user.username
]
},
then: '$receiver',
'else': '$sender'
}
}
}
}
}])
Sources
Aggregation
$match aggregation stage
$group aggregation stage
$addToSet operation
$cond operation
I am trying to get randomly one item from an array using mongoose, I use .aggregate:
const winner = await gSchema.aggregate(
[
{ "$unwind": "$Users" },
{ "$sample": { "size": 1 } }
]
)
I console.log(winner) I get:
[
{
_id: new ObjectId("62c0943a789817d59c19bfa4"),
Guild: '1234567889',
Host: '1234567889',
Channel: '1234567889',
MessageID: '1234567889',
Time: '86400000',
Date: 2022-07-02T18:53:46.981Z,
Users: '1234567889',
__v: 0
}
]
Instead, I want to only get the value of Users like: 1234567889 in my console, not the whole Schema, any idea how to achieve that?
Also is there a way to use filter when using aggregate?
In order to get only the Users data add a projection step:
const winner = await gSchema.aggregate(
[
{$unwind: "$Users"},
{$sample: {size: 1}},
{$project: {Users: 1, _id:0}}
]
)
In order to filter, add a $match step.
Quick update about the issue, using console.log(winner[0].Users) solved my problem
I have a collection that is made up of companies. Each company has a "number_of_employees" as well as a subdocument of "offices" which includes "state_code" and "country_code". For example:
{
'_id': ObjectId('52cdef7c4bab8bd675297da5'),
'name': 'Technorati',
'number_of_employees': 35,
'offices': [
{'description': '',
'address1': '360 Post St. Ste. 1100',
'address2': '',
'zip_code': '94108',
'city': 'San Francisco',
'state_code': 'CA',
'country_code': 'USA',
'latitude': 37.779558,
'longitude': -122.393041}
]
}
I'm trying to get the number of employees per state across all companies. My latest attempt looks like:
db.research.aggregate([
{ "$match": {"offices.country_code": "USA" } },
{ "$unwind": "$offices" },
{ "$project": { "_id": 1, "number_of_employees": 1, "offices.state_code": 1 } }
])
But now I'm stuck on how to do the $group. Because the num_of_employees is at the company level and not the office level I want to split them evenly across the offices. For example, if Technorati has 5 offices in 5 different states then each state would be allocated 7 employees.
In SQL I could do this easily enough using a windowed function to get average employees across offices by company and then summing those while grouping by state. I can't seem to find any clear examples of similar functionality in MongoDB though.
Note, this is for a school assignment, so the use of third-party libraries isn't feasible. Also, I'm hoping that this can all be done in a simple snippet of code, possibly even one call. I could certainly create new intermediate collections or do this in Python and process data there, but that's probably outside of the scope of the homework.
Anything to point me in the right direction would be greatly appreciated!
You are actually on the right track. You just need to derive an extra field numOfEmpPerOffice by using $divide and $sum it when $group by state.
db.collection.aggregate([
{
"$match": {
"offices.country_code": "USA"
}
},
{
"$addFields": {
"numOfEmpPerOffice": {
"$divide": [
"$number_of_employees",
{
"$size": "$offices"
}
]
}
}
},
{
"$unwind": "$offices"
},
{
$group: {
_id: "$offices.state_code",
totalEmp: {
$sum: "$numOfEmpPerOffice"
}
}
}
])
Here is the Mongo playground for your reference.
So I mad the mistake and saved a lot of doduments twice because I messed up my document id. Because I did a Insert, i multiplied my documents everytime I saved them. So I want to delete all duplicates except the first one, that i wrote. Luckilly the documents have an implicit unique key (match._id) and I should be able to tell what the first one was, because I am using the object id.
The documents look like this:
{
_id: "5e8e2d28ca6e660006f263e6"
match : {
_id: 2345
...
}
...
}
So, right now I have a aggregation that tells me what elements are duplicated and stores them in a collection. There is for sure a more elegant way, but I am still learning.
[{$sort: {"$_id": 1},
{$group: {
_id: "$match._id",
duplicateIds: {$push: "$_id"},
count: {$sum: 1}
}},
{$match: {
count: { $gt: 1 }
}}, {$addFields: {
deletableIds: { $slice: ["$duplicateIds", 1, 1000 ] }
}},
{$out: 'DeleteableIds'}]
Now I do not know how to proceed further, as it does not seem to have a "delete" operation in aggregations and I do not want to write those temp data to a db just so I can write a delete command with that, as I want to delete them in one go. Is there any other way to do this? I am still learning with mongodb and feel a little bit overwhelmed :/
Rather than doing all of those you can just pick first document in group for each _id: "$match._id" & make it as root document. Also, I don't think you need to do sorting in your case :
db.collection.aggregate([
{
$group: {
_id: "$match._id",
doc: {
$first: "$$ROOT"
}
}
},
{
$replaceRoot: {
newRoot: "$doc"
}
}, {$out: 'DeleteableIds'}
])
Test : MongoDB-Playground
I think you're on the right track, however, to delete the duplicates you've found you can use a bulk write on the collection.
So if we imagine you aggregation query saved the following in the the DeleteableIds collection
> db.DeleteableIds.insertMany([
... {deletableIds: [1,2,3,4]},
... {deletableIds: [103,35,12]},
... {deletableIds: [345,311,232,500]}
... ]);
We can now take them and write a bulk write command:
const bulkwrite = db.DeleteableIds.find().map(x => ({ deleteMany : { filter: { _id: { $in: x.deletableIds } } } }))
then we can execute that against the database.
> db.collection1.bulkWrite(bulkwrite)
this will then delete all the duplicates.
Edit: Solved below. A note for the repo maintainers that I know browse here: I notice that many of the offending users are very old. They are from when we forked the database ages ago. I'm not sure what changed, but something did, without requiring an npm update, that caused this error to start appearing. These users have been duplicate within our database for years, yet we just started getting this exception yesterday. Not sure if that's an "issue" worth looking into, or if it's the result of an issue being resolved.
I found some similar questions but they are due to authentication errors, which I'm not experiencing.
I hit the above error message for both email and username, which isn't surprising because I use the user's e-mail as their username. What is surprising is that this error message starting occurring mid-day yesterday after no changes I could pin to be the source of the issue. There were no new users created that had a duplicate e-mail / username, and I was unable to change a username or e-mail through parse-dashboard or the API to be a duplicate.
More oddly, this is occurring on both production and test servers, both having started at the same time. One was a duplicate of the other at one point, but the fork was three years ago.
Any ideas where I can start to try to solve this? It doesn't seem to stop the server from functioning but I'm getting an uncaught exception breakpoint triggering every time I start my debug server.
I was looking into mongo's aggregate kinda of using this guide as a basis:
https://www.compose.com/articles/finding-duplicate-documents-in-mongodb/
However, nothing comes up when I try db.User.aggregate([{$group: { _id: {username: "$email"}, uniqueIds: {$addToSet: "$_id"}, count: {$sum: 1}}}, {$match: { count: {"$gt": 1}}}]);, and if I do _User (the actual name of the collection) I get the error E QUERY TypeError: Cannot call method 'aggregate' of undefined
For clearer formatting:
db._User.aggregate(
[
{
$group:{
_id: {
username: "$email"
}, uniqueIds: {
$addToSet: "$_id"
}, count: {
$sum: 1
}
}
}, {
$match: {
count: {
"$gt": 1
}
}
}
]
);
Anybody have tips on how I can figure out where these duplicates are or why I'm just now getting this error?
The Mongo shell doesn't like all collection names, so they're not dot accessible. I just had to use .getCollection() instead of .collection, and it worked perfectly!
db.getCollection("_User").aggregate(
[
{
$group:{
_id: {
username: "$email"
}, uniqueIds: {
$addToSet: "$_id"
}, count: {
$sum: 1
}
}
}, {
$match: {
count: {
"$gt": 1
}
}
}
]
);