I've been cracking my head over the addition of every 'other' columns together during aggregation in Mongo.
A sample of my data:
[
{'item': 'X',
'USA': 3,
'CAN': 1,
'CHN': 1,
'IDN': 1,
:
:
:
},
{'item': 'R',
'USA': 2,
'CAN': 2,
'CHN': 1,
'IDN': 2,
:
:
:
}
]
At the aggregate stage, I would like to have a new field called 'OTHER', which is the resultant of the summation of all the fields that are not specified.
My desired result is this:
[
{'item': 'X',
'NAM': 79,
'IDN': 51,
'OTHER': 32
},
{'item': 'R',
'NAM': 42,
'IDN': 11,
'OTHER': 20
}
]
So far, the closest I could get is using this:
mycoll.aggregate([
{'$addFields':{
'NAM': {'$add':[{'$ifNull':['$CAN', 0]},{'$ifNull':['$USA', 0]}]},
'INDIA': {'$ifNull':['$IDN', 0]},
'OTHER': /* $add all the fields that are not $USA, $CAN, $IDN*/
}},
])
Mongo gurus, please enlighten this poor soul. Deeply appreciate it. Thanks!
In general the idea is converting your document to an array so we could iterate over it while ignoring unwanted fields.
{
'$addFields': {
'NAM': {'$add': [{'$ifNull': ['$CAN', 0]}, {'$ifNull': ['$USA', 0]}]},
'INDIA': {'$ifNull': ['$IDN', 0]},
"OTHER": {
$reduce:
{
input: {"$objectToArray": "$$ROOT"},
initialValue: {sum: 0},
in: {
sum: {
$cond: {
if: {$in: ["$$this.k", ['_id', "item", "CAN", "USA", "IDN"]]},
then: "$$value.sum",
else: {$add: ["$$value.sum", "$$this.v"]}
}
}
}
}
}
}
}
Obivously you should also add any other fields that you have in your document that you do not want to sum up / are not of type number.
Related
I want to get an aggregation/count of the occurrence of all items in an array across all documents. I've tried looking up examples but none of them seem to cover this scenario exactly or go about it in a very obtuse way.
Here's a simple idea of the document model i'm working with. The itemIds array within each object is always unique (no repeated values):
[{
_id:1,
itemIds:[3, 4, 6, 12]
},
{
_id:2,
itemIds:[4, 12]
},
{
_id:3,
itemIds:[3, 4, 8, 9, 12]
}]
I need the counts of each of these summed up (doesn't have to be this exact format but just giving a general idea of what I need):
{
itemsCount:[
{
itemId:3,
count:2
},
{
itemId:4,
count:3
},
{
itemId:6,
count:1
},
{
itemId:8,
count:1
},
{
itemId:9,
count:1
},
{
itemId:12,
count:3
}
]
}
Please try this :
db.yourCollection.aggregate([
{$project : {'itemIds' : 1, _id :0}},
{$unwind : '$itemIds'},
{$group : {'_id': '$itemIds', count :{$sum :1}}}
])
I got a problem when I use db.collection.aggregate in MongoDB.
I have a data structure like:
_id:...
Segment:{
"S1":1,
"S2":5,
...
"Sn":10
}
It means the following in Segment: I might have several sub attributes with numeric values. I'd like to sum them up as 1 + 5 + .. + 10
The problem is: I'm not sure about the sub attributes names since for each document the segment numbers are different. So I cannot list each segment name. I just want to use something like a for loop to sum all values together.
I tried queries like:
db.collection.aggregate([
{$group:{
_id:"$Account",
total:{$sum:"$Segment.$"}
])
but it doesn't work.
You have made the classical mistake to have arbitrary field names. MongoDB is "schema-free", but it doesn't mean you don't need to think about your schema. Key names should be descriptive, and in your case, f.e. "S2" does not really mean anything. In order to do most kinds of queries and operations, you will need to redesign you schema to store your data like this:
_id:...
Segment:[
{ field: "S1", value: 1 },
{ field: "S2", value: 5 },
{ field: "Sn", value: 10 },
]
You can then run your query like:
db.collection.aggregate( [
{ $unwind: "$Segment" },
{ $group: {
_id: '$_id',
sum: { $sum: '$Segment.value' }
} }
] );
Which then results into something like this (with the only document from your question):
{
"result" : [
{
"_id" : ObjectId("51e4772e13573be11ac2ca6f"),
"sum" : 16
}
],
"ok" : 1
}
Starting Mongo 3.4, this can be achieved by applying inline operations and thus avoid expensive operations such as $group:
// { _id: "xx", segments: { s1: 1, s2: 3, s3: 18, s4: 20 } }
db.collection.aggregate([
{ $addFields: {
total: { $sum: {
$map: { input: { $objectToArray: "$segments" }, as: "kv", in: "$$kv.v" }
}}
}}
])
// { _id: "xx", total: 42, segments: { s1: 1, s2: 3, s3: 18, s4: 20 } }
The idea is to transform the object (containing the numbers to sum) as an array. This is the role of $objectToArray, which starting Mongo 3.4.4, transforms { s1: 1, s2: 3, ... } into [ { k: "s1", v: 1 }, { k: "s2", v: 3 }, ... ]. This way, we don't need to care about the field names since we can access values through their "v" fields.
Having an array rather than an object is a first step towards being able to sum its elements. But the elements obtained with $objectToArray are objects and not simple integers. We can get passed this by mapping (the $map operation) these array elements to extract the value of their "v" field. Which in our case results in creating this kind of array: [1, 3, 18, 42].
Finally, it's a simple matter of summing elements within this array, using the $sum operation.
Segment: {s1: 10, s2: 4, s3: 12}
{$set: {"new_array":{$objectToArray: "$Segment"}}}, //makes field names all "k" or "v"
{$project: {_id:0, total:{$sum: "$new_array.v"}}}
"total" will be 26.
$set replaces $addFields in newer versions of mongo. (I'm using 4.2.)
"new_array": [
{
"k": "s1",
"v": 10
},
{
"k": "s2",
"v": 4
},
{
"k": "s3",
"v": 12
}
]
You can also use regular expressions. Eg. /^s/i for words starting with "s".
I have one document like this:
-document: users-
{
"name": "x", password: "x" recipes:
[{title: eggs, dificult: 1},{title: "pizza" dificult: 2}],
name: "y", password: "y" recipes: [{title: "oil", dificult: 2},{title: "eggandpotatoes" dificult: 2}]
}
I want to get all recipes filtering by title and dificult
I have tried some like this
db.users.find({$and: [{"recipes.title": /.*egg.*/},{"recipes.dificult": 2}]},
{"recipes.title": 1,"recipes.dificult": 1}).toArray();
this should return
{title: "eggandpotatoes" dificult: 2}
but return
{title: eggs, dificult: 1}
{title: "eggandpotatoes" dificult: 2}
I would like once the filter works, limit the result with start: 2 and end: 5
returning from the 2 result the next 5.
Thanks in advance.
You can use the $filter aggregation to filter the recipes and $slice to limit the results. It will give you everything you are looking expect but regex search which is not possible as of now.
db.users.aggregate([{
$project: {
_id: 0,
recipes: {
$slice: [{
$filter: {
input: "$recipes",
as: "recipe",
"cond": {
$and: [{
$eq: ["$$recipe.title", "eggandpotatoes"]
}, {
$eq: ["$$recipe.dificult", 2]
}]
}
}
}, 2, 3]
}
}
}]);
If you need to find out regular expression in title then use $elemMatch as below :
db.users.find({"recipes":{"$elemMatch":{"title": /.*egg.*/,"dificult": 2}}}, {"recipes.$":1})
One thing I mentioned here if your recipes array contains some thing like this
"recipes" : [ { "title" : "egg", "dificult" : 2 }, { "title" : "eggand", "dificult" : 2 } ]
then $ projection return only first matched array objcect in above case it will be
"recipes" : [ { "title" : "egg", "dificult" : 2 } ]
If you need this both array then go to #Sagar answer using aggregation
I have the following type of documents stored in a collection.
{
"_id" : "318036:2014010100",
"data": [
{"flow": [6, 10, 12], "occupancy": [0.0356, 0.06, 0.0856], time: 0},
{"flow": [2, 1, 4], "occupancy": [0.01, 0.0056, 0.0422], time: 30},
...
]
}
I want to compute an aggregated value from the first, second, ..., nth value in the flow and occupancy arrays. The order within the array should be preserved. Assuming I want compute the sum. The result should look like the following:
{
"_id" : "318036:2014010100",
"data": [
{"flow": [6, 10, 12], "occupancy": [0.0356, 0.06, 0.0856], sum: [6.0356, 10.006, 12.00856], time: 0},
{"flow": [2, 1, 4], "occupancy": [0.01, 0.0056, 0.0422], sum: [2.01, 1.0056, 4.0422], time: 30},
...
]
}
I tried to solve this by using the aggregation framework but my current approach does not preserve the ordering and produces to much sums.
db.sens.aggregate([
{$match: {"_id":/^318036:/}},
{$limit: 1},
{$unwind: "$data"},
{$unwind: "$data.flow"},
{$unwind: "$data.occupancy"},
{
$group: {
_id: {id: "$_id", time: "$data.time", o: "$data.occupancy", f: "$data.flow", s: {$add: ["$data.occupancy", "$data.flow"]}}
}
},
{
$group: {
_id: {id: "$_id.id", time: "$_id.time"}, occ: { $addToSet: "$_id.o"}, flow: {$addToSet: "$_id.f"}, speed: {$addToSet: "$_id.s"}
}
}
])
I am not sure if it is possible to solve this problem with the aggregation framework, so a solution using MapReduce would also be fine. How can I produce the desired result?
An alternative solution with neither aggregation framework nor map/reduce:
db.sens.find().forEach(function (doc) {
doc.data.forEach(function(dataElement) {
var sumArray = [];
for (var j = 0; j < dataElement.flow.length; j++) {
sumArray[j] = dataElement.flow[j] + dataElement.occupancy[j];
}
dataElement.sum = sumArray;
collection.save(doc);
});
});
I have a collection structured thusly:
{
_id: 1,
score: [
{
foo: 'a',
bar: 0,
user: {user1: 0, user2: 7}
}
]
}
I need to find all documents that have at least one 'score' (element in score array) that has a certain value of 'bar' and a non-empty 'user' sub-document.
This is what I came up with (and it seemed like it should work):
db.col.find({score: {"$elemMatch": {bar:0, user: {"$not":{}} }}})
But, I get this error:
error: { "$err" : "$not cannot be empty", "code" : 13030 }
Any other way to do this?
Figured it out: { 'score.user': { "$gt": {} } } will match non-empty docs.
I'm not sure I quite understand your schema, but perhaps the most straight forward way would be to not have an "empty" value for score.user ?
Instead purposely not have that field in your document if it has no content?
Then your query could be something like ...
> db.test.find({ "score" : { "$elemMatch" : { bar : 0, "user" : {"$exists": true }}}})
i.e. looking for a value in score.bar that you want (0 in this case) checking for the mear existence ($exists, see docs) of score.user (and if it has a value, then it'll exist?)
editied: oops I missed the $elemMatch you had ...
You probably want to add an auxiliary array that keeps track of the users in the user document:
{
_id: 1,
score: [
{
foo: 'a',
bar: 0,
users: ["user1", "user2"],
user: {user1: 0, user2: 7}
}
]
}
Then you can add new users atomically:
> db.test.update({_id: 1, score: { $elemMatch: {bar: 0}}},
... {$set: {'score.$.user.user3': 10}, $addToSet: {'score.$.users': "user3"}})
Remove users:
> db.test.update({_id: 1, score: { $elemMatch: {bar: 0}}},
... {$unset: {'score.$.user.user3': 1}, $pop: {'score.$.users': "user3"}})
Query scores:
> db.test.find({_id: 1, score: {$elemMatch: {bar: 0, users: {$not: {$size: 0}}}}})
If you know you'll only be adding non-existent users and removing existent users from the user document, you can simplify users to a counter instead of an array, but the above is more resilient.
Look at the $size operator for checking array sizes.
$group: {
_id: '$_id',
tasks: {
$addToSet: {
$cond: {
if: {
$eq: [
{
$ifNull: ['$tasks.id', ''],
},
'',
],
},
then: '$$REMOVE',
else: {
id: '$tasks.id',
description: '$tasks.description',
assignee: {
$cond: {
if: {
$eq: [
{
$ifNull: ['$tasks.assignee._id', ''],
},
'',
],
},
then: undefined,
else: {
id: '$tasks.assignee._id',
name: '$tasks.assignee.name',
},
},
},
},
},
},
},