I am fairly new to MongoDB and I am facing a strange issue.
I've been able to build an aggregation pipeline using a compass.
The query produces the expected result in the GUI but it returns nothing in mongo shell.
Here is the query :
db.searchQueryCollection.aggregate([{
$match: {
time_stamp: {
$gte: ISODate("2018-08-13 17:33:20.000"),
$lt: ISODate("2018-08-13 19:33:20.000")
}
}
}, {
$group: {
_id: "$purpose",
groupedPurpose: {$push: "$purpose"}
}
}, {$project: {number_of_results: {$size: "$groupedPurpose"}}}])
Here is the output in compass :
Any idea on what's going wrong?
In mongo shell, you should use as below:
db.getCollection("YOUR_COLLECTION_NAME").aggregate([{
$match: {
time_stamp: {
$gte: ISODate("2018-08-13 17:33:20.000"),
$lt: ISODate("2018-08-13 19:33:20.000")
}
}
}, {
$group: {
_id: "$purpose",
groupedPurpose: {$push: "$purpose"}
}
}, {$project: {number_of_results: {$size: "$groupedPurpose"}}}])
If none of your aggregation queries return a result, you may be connected to the wrong database.
If you're connecting using a database driver, check your connection string. In the following example, the database name is at the end of the connection string.
MONGODB_URI=mongodb+srv://username:password#cluster0.abcdefg.mongodb.net/databasename
If you're connecting using MongoDB shell, run the following two commands:
show dbs;
admin 288.00 KiB
config 328.00 KiB
local 62.66 MiB
mydatabase 1.24 MiB
use mydatabase;
Related
I am running this query to add new field in my document, and it run successfully but does not update the document in mongodb.(movies is the collection name, mongodb version - 4.2.15 Community)
db.movies.aggregate([{
$addFields: {
totalHomework: "123312" ,
totalQuiz: "12312"
}}])
update
db.collection.update({},
{
$set: {
totalHomework: "123312",
totalQuiz: "12312"
}
})
mongoplayground
So I mad the mistake and saved a lot of doduments twice because I messed up my document id. Because I did a Insert, i multiplied my documents everytime I saved them. So I want to delete all duplicates except the first one, that i wrote. Luckilly the documents have an implicit unique key (match._id) and I should be able to tell what the first one was, because I am using the object id.
The documents look like this:
{
_id: "5e8e2d28ca6e660006f263e6"
match : {
_id: 2345
...
}
...
}
So, right now I have a aggregation that tells me what elements are duplicated and stores them in a collection. There is for sure a more elegant way, but I am still learning.
[{$sort: {"$_id": 1},
{$group: {
_id: "$match._id",
duplicateIds: {$push: "$_id"},
count: {$sum: 1}
}},
{$match: {
count: { $gt: 1 }
}}, {$addFields: {
deletableIds: { $slice: ["$duplicateIds", 1, 1000 ] }
}},
{$out: 'DeleteableIds'}]
Now I do not know how to proceed further, as it does not seem to have a "delete" operation in aggregations and I do not want to write those temp data to a db just so I can write a delete command with that, as I want to delete them in one go. Is there any other way to do this? I am still learning with mongodb and feel a little bit overwhelmed :/
Rather than doing all of those you can just pick first document in group for each _id: "$match._id" & make it as root document. Also, I don't think you need to do sorting in your case :
db.collection.aggregate([
{
$group: {
_id: "$match._id",
doc: {
$first: "$$ROOT"
}
}
},
{
$replaceRoot: {
newRoot: "$doc"
}
}, {$out: 'DeleteableIds'}
])
Test : MongoDB-Playground
I think you're on the right track, however, to delete the duplicates you've found you can use a bulk write on the collection.
So if we imagine you aggregation query saved the following in the the DeleteableIds collection
> db.DeleteableIds.insertMany([
... {deletableIds: [1,2,3,4]},
... {deletableIds: [103,35,12]},
... {deletableIds: [345,311,232,500]}
... ]);
We can now take them and write a bulk write command:
const bulkwrite = db.DeleteableIds.find().map(x => ({ deleteMany : { filter: { _id: { $in: x.deletableIds } } } }))
then we can execute that against the database.
> db.collection1.bulkWrite(bulkwrite)
this will then delete all the duplicates.
Error on server console: UnhandledPromiseRejectionWarning: RangeError: Maximum call stack size exceeded
Meteor 1.8.1
Project worked fine at 1.6.x, MongoDB 3.2.x
mlab pushed me to mongo 3.6.x and now some of my charts (chart.js) and tables (aslagle:reactive-table) aren’t working.
Both of these are generated via aggregate/pipeline (example below).
Long story short, I’ve neglected this project for several months and just getting back to it, ran into this issue, and tried updating to 1.8.1. Which has me at mongo 4.0.6
Charts and tables still not working and I think it’s because I’m missing some update to the syntax/structure in the aggregates and/or pipelines. I could very well be wrong here as I’m rusty, not a dev by trade and as mentioned haven’t touched this in a few months.
Here’s one failing aggregate/pipeline that’s used to draw a line chart (Chart.js):
pointTrendsSprt: function(uid, sp){
var pipeline = [
{ $match: {"userId": uid, "sport":sp, "slate": {$exists: true} } },
{ $project: {slate:1, sport:1, pointsWon:1, createdAt:1} },
{ $group: {
_id: "$slate",
sport: {"$addToSet": "$sport"},
pointsWon: {"$sum": "$pointsWon"},
createdAt: {"$max": "$createdAt"}
}
},
{ $sort: { "createdAt":1 } }
];
return Results.aggregate(pipeline);
},
Here’s another that’s used in a aslagle:reactive-table:
allXteam: function(uid){
var pipeline = [
{ $match: {"userId": uid, "win": "true"} },
{ $project: {team:1, match:1, play:1, Count: "$count"} },
{ $group:
{_id: "$team",
play: {"$addToSet": "$play"},
count:{$sum:1}}
}
];
return Picks.aggregate( pipeline );
},
Both of these are in a methods.js file in the server folder. I use Meteor.call on the client side js file to set session vars where the chart and tables pull from.
As mentioned, I think I need to fix something in my aggregations/pipelines but tried adding in “cursor” and “explain” and am either not doing that right or just wrong in that assumption.
Any suggestions/guidance would be greatly appreciated.
I found that this solution worked:
Collection.rawCollection().aggregate().toArray()
Here's my final re-factoring (although I will have to look into that 'Meteor.wrapAsync' form:
allXteam: async function(uid){
var pipeline = [
{ $match: {"userId": uid, "win": "true"} },
{ $project: {team:1, match:1, play:1, Count: "$count"} },
{ $group:
{_id: "$team",
play: {"$addToSet": "$play"},
count:{$sum:1}}
}
];
const aXt = await Picks.rawCollection().aggregate( pipeline ).toArray();
return aXt;
I currently have a mongo shell command to run a duplicate search in a mailing list collection:
var duplicates = [];
db.mailing_entries.aggregate([
{ $group: {
_id: { full_name: "$full_name", business: "$business", address_line_1: "$address_line_1", postal_code: "$postal_code" },
dups: { $addToSet: "$_id" },
count: { $sum: 1 }
}},
{ $match: {
count: { $gt: 1 }
}}
])
.result
.forEach(function(doc) {
doc.dups.shift();
doc.dups.forEach( function(dupId){
duplicates.push(dupId);
}
)
});
printjson(duplicates);
The shell code is working perfectly for me, however, after much searching, I cannot find a way to properly translate to Doctrine Mongo ODM using a map reduce function, or any other method.
I'm currently using the Doctrine Mongo ODM module integrated with Zend Framework 2.
I have searched far and wide but to no avail.
Underlying doctrine/mongodb package introduced aggregation builder in version 1.2 and while they're no docs written as of yet you can take a look at examples in linked pull request and take advantage of your IDE's autocompletion.
We aim to allow hydrating results of aggregate results into documents in ODM but feature is not ready yet and for now you need to obtain builder like this (supposing MailingEntry is your mapped document)
$builder = $documentManager->getDocumentCollection(MailingEntry::class)->createAggregationBuilder();
I've coded an aggregation I want to run on a collection in the mongo shell. If I paste it directly into the shell, I have to do it line by line which is tedious and slow, especially when it doesn't work and I have to do it over. Therefore, I put it in a .js document and loaded it like this load("myaggregation.js") but that only returns true if the code is valid. How do I run the aggregation after loading it? I can't find any documentation about this. Do I have to use the nodejs driver?
Put your aggregation code in a function:
function my_aggregate() {
return db.foo.aggregate( [ ... ] );
}
and store that in your .js file.
Then run load to load it. You can also pass the filename on the commandline using the --shell commandline flag which will start the interactive shell after running any specified .js files.
Once the file is run your new function will be available to execute. Simply type
my_aggregate()
Update You do have to use an explicit return which I failed to mention. So in your case you would want something like:
function my_aggregate() {
return db.zips.aggregate([{ $match: { state: { $in: [ "CA", "NY" ] } }},{ $group:{ _id : { "state" : "$state","city" : "$city" }, pop : { $sum: "$pop"}}},{ $match: { pop: { $gt: 25000 } }},{$group:{ _id : { "city" : "$city"}, pop : { $avg: "$pop"}}}]);
}
Have you seen http://docs.mongodb.org/manual/tutorial/write-scripts-for-the-mongo-shell/?
Something like: mongo localhost:27017/test myjsfile.js
It also says that "You can execute a .js file from within the mongo shell, using the load() function".
I know this is a late answer but I found a nice way to handle this:
Create a js file containing your aggregate query:
use blog;
db.posts.aggregate([
... my query ...
]);
then you can simply cat and pipe the file to mongo client:
cat myfile.js | mongo
script.js file content:
aggrQuery = db.foo.aggregate([
{$group: {_id: '$bar', pop: {$max: '$buz'}}}
])
in the mongo shell:
> load('/file/location/script.js');
true
>aggrQuery
here will be output