I am new to Mongodb, and NoSQL in general and I am trying to use mongodbs aggregate function to aggregate data from one collection to be inserted into another. An example of the original collection would be this:
Original Collection
{
supplier: 'aldi',
timestamp: '1492807458',
user: 'eddardstark#gmail.com',
hasBeenAggregated:false,
items:[{
name: 'butter',
supplier: 'aldi',
expiry: '1492807458',
amount: 454,
measureSymbol: 'g',
cost: 2.19
},{
name: 'milk',
supplier: 'aldi',
expiry: '1492807458',
amount: 2000,
measureSymbol: 'ml',
cost: 1.49
}]
}
An example of the output I am trying to achieve would be:
New Collection
{
user:'eddardstark#gmail.com',
amount: 3.68,
isIncome: false,
title: 'food_shopping',
timestamp: '1492807458'
}
The aggregation function that I am using is:
Aggregation
var result = db.runCommand({
aggregate: 'food_transactions',
pipeline: [
{$match: {hasBeenAggregated: false}},
{$unwind: '$items'},
{$group:{_id: '$_id',amount:{$sum: '$items.cost'}}},
{$project: {
_id:0,
user:1,
amount:1,
isIncome: {$literal: false},
title:{$literal: 'food_shopping'},
timestamp:1
}}
]
});
printjson(result)
This aggregation function does not return the user or timestamp fields. Instead, I get the following output:
Output
{
"amount" : 3.6799999999999997,
"isIncome" : false,
"title" : "food_shopping"
}
If I don't group the results and perform the calculations in the $project stage, the fields are all projected correctly, but obviously, there is a new document created for each sub-document in the items array and that rather defeats the purpose of the aggregation.
What am I doing wrong?
Update your $group pipeline to include all the fields you wish to project further down the pipeline.
To include user field you can use $first
{$group:{_id: '$_id', user:{$first:'$user`}, amount:{$sum: '$items.cost'}}},
Additionally, if you are 3.4 version you can simplify your aggregation to below.
Use $reduce to sum all the item's cost in a single document. For all documents you can add $group after $reduce.
db.collection.aggregate([
{$match: {hasBeenAggregated: false}},
{$project: {
_id:0,
user:1,
amount: {
$reduce: {
input: "$items",
initialValue: 0,
in: { $add : ["$$value", "$$this.cost"] }
}
},
isIncome: {$literal: false},
title:{$literal: 'food_shopping'},
timestamp:1
}}
])
Related
I'd like to sort a collection, then add a virtual property to the result which is their numerical order in which the results where displayed as.
So for example, we have a collection called calls, and we'd like to ascertain the current call queue priority as a number so it can be synced to our CRM via reverse ETL.
We have to do this inside of the query itself because we don't have an intermediary step where we can introduce any logic to determine this logic.
So my current query is
db.getCollection('callqueues').aggregate([
{
$match: {
'invalidated': false,
'assigned_agent': null
}
},
{ $sort: {
score: -1, _id: -1
} },
{
$addFields: {
order: "<NEW ORDER PROPERTY HERE>",
}
},
])
So I was wondering how would I insert as a virtual property their order, where the first element after the sort should be 1, second 2, etc
One option (since mongoDB version 5.0) is to use $setWindowFields for this:
db.collection.aggregate([
{$match: {invalidated: false, assigned_agent: null}},
{$setWindowFields: {
sortBy: {score: -1, _id: -1},
output: {
order: {
$sum: 1,
window: {documents: ["unbounded", "current"]}
}
}
}}
])
See how it works on the playground example
EDIT: If your mongoDB version is earlier than 5.0, you can use a less efficient query, involving $group and $unwind:
db.collection.aggregate([
{$match: {invalidated: false, assigned_agent: null}},
{$sort: {score: -1, _id: -1}},
{$group: {_id: 0, data: {$push: "$$ROOT"}}},
{$unwind: {path: "$data", includeArrayIndex: "order"}},
{$replaceRoot: {newRoot: {$mergeObjects: ["$data", {order: {$add: ["$order", 1]}}]}}}
])
See how it works on the playground example < 5.0
for example
animals = ['cat','mat','rat'];
collection contains only 'cat' and 'mat'
I want the query to return 'rat' which is not there in collection..
collection contains
[
{
_id:objectid,
animal:'cat'
},
{
_id:objectid,
animal:'mat'
}
]
db.collection.find({'animal':{$nin:animals}})
(or)
db.collection.find({'animal':{$nin:['cat','mat','rat']}})
EDIT:
One option is:
Use $facet to $group all existing values to a set. using $facet allows to continue even if the db is empty, as #leoll2 mentioned.
$project with $cond to handle both cases: with or without data.
Find the set difference
db.collection.aggregate([
{$facet: {data: [{$group: {_id: 0, animals: {$addToSet: "$animal"}}}]}},
{$project: {
data: {
$cond: [{$gt: [{$size: "$data"}, 0]}, {$first: "$data"}, {animals: []}]
}
}},
{$project: {data: "$data.animals"}},
{$project: {_id: 0, missing: {$setDifference: [animals, "$data"]}}}
])
See how it works on the playground example - with data or playground example - without data
I have a table Thread:
{
userId: String
messageId: String
}
Now I have an array of userIds, I need to query 20 messageIds for each of them, I can do it with a loop:
const messageIds = {}
for (const userId of userIds) {
const results = await Thread.find({ userId }).sort({ _id: -1 }).limit(20).exec()
messageIds[userId] = results.map(result => result.messageId)
}
But of course this doesn't perform well. Is there a better solution?
The problem with your approach is that you are issuing multiple separate queries to MongoDB.
The simplest workaround to this is using the $push and $slice approach. But this has the problem that the intermediate step would creating an array of huge size.
Another way could be to use $facet as part of aggregation query.
So you need a $facet step in the aggregation like -
[
{$facet: {
'userId1': [
{$match: {userId: 'userId1'} },
{$limit: 20},
{$group: {_id: '', msg: {$push: '$messageId'} } }
],
'userId2': [
{$match: {userId: 'userId2'} },
{$limit: 20},
{$group: {_id: '', msg: {$push: '$messageId'} } }
],
.... (for each userId in array)
}}
]
You can easily just generate this query by iterating over the list of users and adding keys for each user.
So you end up with an object where key is the userId and the value is the array of messages (obj[userId].msg)
You can use aggregation to group threads by userId, and return the top 20:
db.threads.aggregate([
{$match: {userId:{$in: userIds}}},
{$sort: {_id: -1}},
{$group: {_id: "$userId", threads: {$push: "$$ROOT"}}},
{$project: {_id:0, userId:"$_id", threads: {$slice:["$threads", 20]}}}
])
I have a list of items, and I want mongoDB return the result of the sum of their price*quantity, in other words, the total value of my items.
Schema = {
_id: ObjectId,
price: Number,
quantity: Number
}
I'm trying using the aggregation framework, or map reduce, but I can't figure out how correctly use it.
Here an there is an example for finding the sum of prices,
db.items.aggregate([
{$group: {
_id: null,
prices: {$sum: "$price"}
}}
])
Here is what I would like to obtain:
db.items.aggregate([
{$group: {
_id: null,
prices: {$sum: "$price"*"$quantity"}
}}
])
You don't need to use map-reduce for this. You can use aggregation framework and combine multiple aggregation operators. You almost got it you were just missing the final piece - $multiply operator:
db.items.aggregate([{
"$group" : {
"_id" : null,
"prices" : {
"$sum" : {
"$multiply" : ["$price", "$quantity"]
}
}
}
}]);
I have a document collection that look like the following:
{
name : "tester"
, activity: [
{
gear: "glasses"
where: "outside"
}
, {
gear: "hat"
, where: "inside"
}
, {
gear: "glasses"
, where: "car"
}
]
}
How do I query the collection to return only documents with multiple activities that contain the value of "gear":"glasses"?
Thanks!
I think it's possible to do without aggregation framework, if you need full document filtered by your condition:
db.collection.find({
"activity": {$elemMatch: {gear:"glasses"}},
"activity.1" : {$exists: 1}
})
This is going to be ugly with aggregation framework, but it can be done:
db.collection.aggregate(
{$match: {"activity.gear": "glasses"}},
{$unwind: "$activity"},
{$group: {
_id: {_id: "$_id", name: "$name"},
_count: {$sum: {$cond: [{$eq: ["glasses", "$activity.gear"]}, 1, 0]}}
}},
{$match: {_count: {$gt: 1}}}
)
When analyzing the above query, I would recommend walking through step. Start with just the "$match", the the "$match" and "$unwind". And so one. You will see how each step works.
The response is not the full document. If you are looking for the full document, include a $project step that passes through a dummy activity, and reconstruct the full document on the output.
You can also try this:
db.collection.find( { activity: { $elemMatch: { gear: "glasses" } } )