I have searched a lot to find a way for aggregation using loopback mongodb, unfortunately no perfect solution found. One of them is here
But can't implement this, any one to help me solve this problem, with any new solution, or describing above link.
Loopback doesn't provide a way to do an aggregation query, but you can find another solution in: https://github.com/strongloop/loopback/issues/890
//Using the datasource we are making a direct request to MongoDB instead of use the PersistedModel of Loopback
var bookCollection = Book.getDataSource().connector.collection(Book.modelName);
bookCollection.aggregate({
$group: {
_id: { category: "$category", author: "$author" },
total: { $sum: 1 }
}
}, function(err, groupByRecords) {
if(err) {
next(err);
} else {
next();
}
});
Related
So I am writing code where I want to insert many articles to the MongoDB, but I want to check if there are no articles with the same ID and skip them if there are. I can not find an implementation of this logic online can anyone help me with a solution?
collection.InsertMany works fine, but it does not check for existing documents.
You can use the '$setOnInsert'.
Like this :
db.products.update(
{ },
{
$set: { _id: 1, item: "apple" },
$setOnInsert: { defaultQty: 100 }
},
{ upsert: true }
)
And the document is in this link:
https://docs.mongodb.com/manual/reference/operator/update/setOnInsert/
In the Mongo docs on materialized views i saw this example:
updateMonthlySales = function(startDate) {
db.bakesales.aggregate( [
{ $match: { date: { $gte: startDate } } },
{ $group: { _id: { $dateToString: { format: "%Y-%m", date: "$date" } }, sales_quantity: { $sum: "$quantity"}, sales_amount: { $sum: "$amount" } } },
{ $merge: { into: "monthlybakesales", whenMatched: "replace" } }
] );
};
I wanted to understand
where do I write this function,
and how I can invoke it
It would be extra amazing if anyone has a solution from a Play-Framework/Reactive Mongo pov. Reactive-Mongo's way of writing aggregation is quite difficult imo, and I wanted to find a different way to trigger an aggregation, which. updates a view, which will be read by my application.
Any help would be very much appreciated :)
The page uses interactive shell to explain how materialised views work.
In this example they define the function in the mongo shell and then call it in the same shell https://docs.mongodb.com/manual/core/materialized-views/#perform-initial-run
This is for documentation purposes only. You need to run the function to actualize the view and they do it by invoking the same function in the same shell https://docs.mongodb.com/manual/core/materialized-views/#refresh-materialized-view
You won't do it manually in prod. It must be a daemon to run the pipeline by schedule, event, or on-demand. Depending on language you are using to write the daemon the syntax may be completely different. The essential part is it must be an aggregation pipeline with the $merge stage at the end.
I am a mongoDB atlas cloud user. I need to update more than 10K documents remotely, all documents in a colleciton, at once.
Currently I tried ".forEach" to self-referencing, but it is too slow and i wonder there is better way.
myCollection.find({})
.forEach(function(o){
const totalPoint = o.gamepoint + 10
return myCollection.update(
{_id: o._id}
, {
$set: {
total = totalPoint
}
}
);
});
there is no problem! but very slow.... Is there no way to improve the perfromance?
If upgrading the hardware is the only way, what should be upgraded between app hosting server and mongodb server?
Update: My apology. the former code sample is not suit for explaining this problem. this is corrected one.
myCollection.find({})
.forEach(function(o){
const totalPoint = o.gamepoint + o.gamepoint
return myCollection.update(
{_id: o._id}
, {
$set: {
total = totalPoint
}
}
);
});
You can use mongo db operator to get desire outcome. Go here.
Collection.update({},{$inc: { total: 10}},{multi: true } , function(err, updated){
console.log('updated')// this will update all documents.
})
the following update query will get you there but you will need mongodb server v4.2 for it to work.
db.myCollection.update({}, [{
$set: {
total: {
$add: ["$gamepoint", "$gamepoint"]
}
}
}], {
multi: true
});
reference: Update with Aggregation Pipeline
I'm trying to aggregate data from my Mongo collection to produce some statistics for FreeCodeCamp by making a large json file of the data to use later.
I'm running into the error in the title. There doesn't seem to be a lot of information about this, and the other posts here on SO don't have an answer. I'm using the latest version of MongoDB and drivers.
I suspect there is probably a better way to run this aggregation, but it runs fine on a subset of my collection. My full collection is ~7GB.
I'm running the script via node aggScript.js > ~/Desktop/output.json
Here is the relevant code:
MongoClient.connect(secrets.db, function(err, database) {
if (err) {
throw err;
}
database.collection('user').aggregate([
{
$match: {
'completedChallenges': {
$exists: true
}
}
},
{
$match: {
'completedChallenges': {
$ne: ''
}
}
},
{
$match: {
'completedChallenges': {
$ne: null
}
}
},
{
$group: {
'_id': 1, 'completedChallenges': {
$addToSet: '$completedChallenges'
}
}
}
], {
allowDiskUse: true
}, function(err, results) {
if (err) { throw err; }
var aggData = results.map(function(camper) {
return _.flatten(camper.completedChallenges.map(function(challenges) {
return challenges.map(function(challenge) {
return {
name: challenge.name,
completedDate: challenge.completedDate,
solution: challenge.solution
};
});
}), true);
});
console.log(JSON.stringify(aggData));
process.exit(0);
});
});
Aggregate returns a single document containing all the result data, which limits how much data can be returned to the maximum BSON document size.
Assuming that you do actually want all this data, there are two options:
Use aggregateCursor instead of aggregate. This returns a cursor rather than a single document, which you can then iterate over
add a $out stage as the last stage of your pipeline. This tells mongodb to write your aggregation data to the specified collection. The aggregate command itself returns no data and you then query that collection as you would any other.
It just means that the result object you are building became too large. This kind of issue should not be impacted by the version. The fix implemented for 2.5.0 only prevents the crash from occurring.
You need to filter ($match) properly to have the data which you need in result. Also group with proper fields. The results are put into buffer of 64MB. So reduce your data. $project only the columns you require in result. Not whole documents.
You can combine your 3 $match objects to single to reduce pipelines.
{
$match: {
'completedChallenges': {
$exists: true,
$ne: null,
$ne: ""
}
}
}
I had this issue and I couldn't debug the problem so I ended up abandoning the aggregation approach. Instead I just iterated through each entry and created a new collection. Here's a stripped down shell script which might help you see what I mean:
db.new_collection.ensureIndex({my_key:1}); //for performance, not a necessity
db.old_collection.find({}).noCursorTimeout().forEach(function(doc) {
db.new_collection.update(
{ my_key: doc.my_key },
{
$push: { stuff: doc.stuff, other_stuff: doc.other_stuff},
$inc: { thing: doc.thing},
},
{ upsert: true }
);
});
I don't imagine that this approach would suit everyone, but hopefully that helps anyone who was in my particular situation.
I have a query as mentioned below.
var projstat = ['A' , 'B'];
Post.native(function(err, collection) {
if (err)
console.log(err);
collection.find({
'status': {
"$in": projstat
}
}, {multi: true}, function(err, result) {
console.log(result);
if (req.isSocket) {
return res.json(result);
}
});
});
Please correct me if i am wrong as it doesnot return any results. Please help.
You're not using the native find correctly; rather than using a callback as an argument (like Waterline does), you chain the call to toArray and use the callback as its argument:
collection.find({
'status': {
"$in": projstat
}
}).toArray(function(err, results) {...});
Docs for the native Mongo driver are here.
However, the more important point in this case is that you don't need native at all. You can use the regular Waterline find, which automatically does an in query when an attribute is set to an array:
Post.find({status: projstat}).exec(function(err, results) {...});