How do I get MongoDB to calculate the sum of array values when the array field may be missing completely (as is the case for month 10)?
For example:
> db.month.save({MonthNum: 10,
... NumWeekdays: 23});
> db.month.save({MonthNum: 11,
... NumWeekdays: 21,
... Holidays: [ {Description: "Thanksgiving", NumDays: 2} ] });
> db.month.save({MonthNum: 12,
... NumWeekdays: 22,
... Holidays: [ {Description: "Christmas", NumDays: 6},
... {Description: "New Year's Eve", NumDays: 1} ] });
> db.month.aggregate( { $unwind: "$Holidays" },
... { $group: { _id: "$MonthNum",
... total: { $sum: "$Holidays.NumDays" } } });
{
"result" : [
{
"_id" : 12,
"total" : 7
},
{
"_id" : 11,
"total" : 2
}
],
"ok" : 1
}
How do I get month 10 to show up in the above results (showing "total" as 0)?
Bonus: How do I get the above to show the available weekdays (the NumWeekdays minus the sum of the Holidays)?
I've tried $project to get the data into a canonical format first but without success so far... thanks!
$unwind isn't passing along your document with MonthNum 10 because your Holidays array is empty on that document (see the note at the bottom of the $unwind docs). Assuming that Holidays is always either an array containing at least one item or completely absent from a document, you can use the $ifNull operator inside of $project to add a "Holiday" document that just has NumDays = 0 to your Holidays is null:
db.month.aggregate([
// Make "Holidays" = [{NumDays:0}] if "Holidays" is null for this document (i.e. absent)
{$project:{NumWeekDays:1, MonthNum:1, Holidays:{$ifNull:["$Holidays", [{"NumDays":0}]]}}},
// Now you can unwind + group as normal
{$unwind:"$Holidays"},
{$group:{_id:"$MonthNum", NumWeekDays:{$first:"$NumWeekDays"}, "total":{$sum:"$Holidays.NumDays"}}},
// This should take care of "available weekdays"
{$project:{total:1, available:{$subtract:["$NumWeekDays", "$total"]}}}
]);
Note that $ifNull won't work if for some of your documents Holidays is an empty array; it has to be absent completely.
Related
I would like to ranked in descending order a list of documents in array names via their number value.
Here's the structure part of my collection :
_id: ObjectId("W")
var1: "X",
var2: "Y",
var3: "Z",
comments: {
names: [
{
number: 1;
},
{
number: 3;
},
{
number: 2;
}
],
field: Y;
}
but all my request with db.collection.find().sort( { "comments.names.number": -1 } ) doesn't work.
the desired output sort is :
{ "_id" : ObjectId("W"), "var1" : "X", "var3" : "Z", "comments" : { [ { "number" : 3 }, { "number" : 2 },{ "number" : 1 } ], "field": "Y" } }
Can you help me?
You need to aggregate the result, as below:
Unwind the names array.
Sort the records based on comments.names.number in descending
order.
Group the records based on the _id field.
project the required structure.
Code:
db.collection.aggregate([
{$unwind:"$comments.names"},
{$sort:{"comments.names.number":-1}},
{$group:{"_id":"$_id",
"var1":{$first:"$var1"},
"var2":{$first:"$var2"},
"var3":{$first:"$var3"},
"field":{$first:"$comments.field"},
"names":{$push:"$comments.names"}}},
{$project:{"comments":{"names":"$names","field":"$field"},"var1":1,
"var2":1,"var3":1}}
],{"allowDiskUse":true})
If your collection is large, you might want to add a $match criteria in the beginning of the aggregation pipeline to filter records or use (allowDiskUse:true), to facilitate sorting large number of records.
db.collection.aggregate([
{$match:{"_id":someId}},
{$unwind:"$comments.names"},
{$sort:{"comments.names.number":-1}},
{$group:{"_id":"$_id",
"var1":{$first:"$var1"},
"var2":{$first:"$var2"},
"var3":{$first:"$var3"},
"field":{$first:"$comments.field"},
"names":{$push:"$comments.names"}}},
{$project:{"comments":{"names":"$names","field":"$field"},"var1":1,
"var2":1,"var3":1}}
])
What The below query does:
db.collection.find().sort( { "comments.names.number": -1 } )
is to find all the documents, then sort those documents based on the number field in descending order. What this actually does is for each document get the comments.names.number field value which is the largest, for each document. And then sort the parent documents based on this number. It doesn't manipulate the names array inside each parent document.
You need update document for sort an array.
db.collection.update(
{ _id: 1 },
{
$push: {
comments.names: {
$each: [ ],
$sort: { number: -1 }
}
}
}
)
check documentation here:
http://docs.mongodb.org/manual/reference/operator/update/sort/#use-sort-with-other-push-modifiers
MongoDB queries sort the result documents based on the collection of fields specified in the sort. They do not sort arrays within a document. If you want the array sorted, you need to sort it yourself after you retrieve the document, or store the array in sorted order. See this old SO answer from Stennie.
The following query in mongo, behaves strange :
db.items.findOne({},{ "List": { "$slice": [ skip, 3 ] }})
First:
Instead of returning one object with ["_id","List"] keys only, it returns a full object.
Second:
if skip is negative and |skip| is higher than list.length then it returns the first three elements as though skip==0
I would expect for:
{
"_id" : ObjectId("542babf265f5de9a0d5c2928"),
"List" : [
1,
2,
3,
4,
5
]
"other" : "not_important"
}
query:
db.items.findOne({},{ "List": { "$slice": [-10, 3 ] }})
to get:
{
"_id" : ObjectId("542babf265f5de9a0d5c2928"),
"List" : []
}
instead, I get:
{
"_id" : ObjectId("542babf265f5de9a0d5c2928"),
"List" : [
1,
2,
3
]
"other" : "not_important"
}
Why?
I use mongoDB 2.4.10
Second: if skip is negative and |skip| is higher than list.length then it returns the first three elements as though skip==0
Yes. That is how the javascript Array.prototype.slice() method works, which is internally used by mongodb.
According to the ECMAScript® Language Specification,
If relativeStart is negative, let k be max((len + relativeStart),0);
else let k be min(relativeStart, len).
In your case relativeStart is -10,
k = max((-10+5),0), k = 0; (where, 5 is the length of your array).
Hence k or skip will always be 0, in these cases.
First: Instead of returning one object with ["_id","List"] keys only, it returns a full object.
Yes, the projection operator works that way. Unless a inclusion or exclusion is explicitly specified in the projection parameter, the whole document is retrieved with the projection operators such as $slice,$elemmatch being applied.
db.items.findOne({},{"_id":1,"List": { "$slice": [-10, 3 ] }})
would return:
{ "_id" : ObjectId("542babf265f5de9a0d5c2928"), "List" : [ 1, 2, 3 ] }
The second parameter to the findOne() method is not only for simple projection purpose, fields are not projected, only if any one of the field names have a value of 0 or 1 against them. If not the whole document is returned. If any field has a projection operator to be applied, it would be applied and projected.
The projection mechanism seems to happen in the below manner, whenever the $slice operator is involved.
By default all the fields would be included for projection.
By Default all the fields whose values are derived based on the projection operator, $slice, if truthy, are always displayed, irrespective of the below.
Steps taking place for exclusion or inclusion.
The list of fields specified in the projection parameter are accumulated in their specified order.
For only the first field encountered with value '0' or '1':
If the
field has a value '0' - then it is excluded, and all the remaining
fields are marked to be included.
If a field has '1' - then it is included, and all the remaining fields
are marked to be excluded.
For all the subsequent fields, they are excluded or included based on
their values.
Whilst this behavior is by design for the $slice operator, it is possible since MongoDB 3.2 to evaluate this and alter the result with the aggregation operator for $slice:
Given the example documents:
{ "_id" : ObjectId("5922846dbcf60428d0f69f6e"), "a" : [ 1, 2, 3, 4 ] }
{ "_id" : ObjectId("5922847cbcf60428d0f69f6f"), "a" : [ 5, 6 ] }
If given a conditional expression to test against the length of the array with $size and only perform the $slice when the reverse index was greater than or equal to that length, or otherwise return an empty array:
db.collection.aggregate([
{ "$project": {
"a": {
"$cond": {
"if": { "$gte": [ { "$size": "$a" }, 4 ] },
"then": { "$slice": [ "$a", -4, 2 ] },
"else": { "$literal": [] },
}
}
}}
])
Then of course you get:
{ "_id" : ObjectId("5922846dbcf60428d0f69f6e"), "a" : [ 1, 2 ] }
{ "_id" : ObjectId("5922847cbcf60428d0f69f6f"), "a" : [ ] }
So that is how you could get MongoDB to return a "slice" that acts in this way.
Say I have the following four documents in a collection called "Store":
{ item: 'chair', modelNum: 1154, votes: 75 }
{ item: 'chair', modelNum: 1152, votes: 16 }
{ item: 'table', modelNum: 1017, votes: 24 }
{ item: 'table', modelNum: 1097, votes: 52 }
I would like to find only the documents with the highest number of votes for each item type.
The result of this simple example would return modelNum: 1154 and modelNum: 1097. Showing me the most popular model of chair and table, based on the customer inputed vote score.
What is the best way write this query and sort them by vote in descending order? I'm developing using meteor, but I don't think that should have an impact.
Store.find({????}).sort({votes: -1});
You can use $first or $last aggregation operators to achieve what you want. These operators are only useful when $group follows $sort. An example using $first:
db.collection.aggregate([
// Sort by "item" ASC, "votes" DESC
{"$sort" : {item : 1, votes : -1}},
// Group by "item" and pick the first "modelNum" (which will have the highest votes)
{"$group" : {_id : "$item", modelNum : {"$first" : "$modelNum"}}}
])
Here's the output:
{
"result" : [
{
"_id" : "table",
"modelNum" : 1097
},
{
"_id" : "chair",
"modelNum" : 1154
}
],
"ok" : 1
}
If you are looking to do this in Meteor and on the client I would just use an each loop and basic find. Minimongo keeps the data in memory so I don't think additional find calls are expensive.
like this:
Template.itemsList.helpers({
items: function(){
var itemNames = Store.find({}, {fields: {item: 1}}).map(
function( item ) { return item.item; }
);
var itemsMostVotes = _.uniq( itemNames ).map(
function( item ) {
return Store.findOne({item: item}, {sort: {votes: -1}});
}
);
return itemsMostVotes;
}
});
I have switched to findOne so this returns an array of objects rather than a cursor as find would. If you really want the cursor then you could query minimongo with the _ids from itemMostVotes.
You could also use the underscore groupBy and sortBy functions to do this.
You would need to use the aggregation framework.
So
db.Store.aggregate(
{$group:{_id:"$item", "maxVotes": {$max:"$votes"}}}
);
I have a collections of objects with structure like this:
{
"_id" : ObjectId("5233a700bc7b9f31580a9de0"),
"id" : "3df7ce4cc2586c37607a8266093617da",
"published_at" : ISODate("2013-09-13T23:59:59Z"),
...
"topic_id" : [
284,
9741
],
...
"date" : NumberLong("1379116800055")
}
I'm trying to use the following query:
db.collection.find({"topic_id": { $in: [ 9723, 9953, 9558, 9982, 9833, 301, ... 9356, 9990, 9497, 9724] }, "date": { $gte: 1378944001000, $lte: 1378954799000 }, "_id": { $gt: ObjectId('523104ddbc7b9f023700193c') }}).sort({ "_id": 1 }).limit(1000)
The above query uses topic_id, date index but then it does not keep the order of returned results.
Forcing it to use hint({_id:1}) makes the results ordered, but the nscanned is 1 million documents even though limit(1000) is specified.
What am I missing?
I have some documents that look like this:
{
(...stuff...)
something: {
first: {
0: 3,
1: 5,
2: 2
},
second: {
0: 1,
1: 9,
2: 7
}
}
}
For the sake of simplicity in this question, I'll assume that my $match only hits this one document. What I'd like to do is, in an aggregate command, add up the 0s, and add up the 1s, and add up the 2s, so that I can produce something like this:
something.0: 4 (something.first.0 + something.second.0)
something.1: 14 (something.first.1 + something.second.1)
something.2: 9 (something.first.2 + something.second.2)
Is this something that can be done, or do I need to change my document schema to rearrange the nested documents such that all the 0s are together, etc?
If you know the names of all the arrays you want to sum, you can use the Aggregation Framework in MongoDB 2.2+ and a projection with the $add operator:
db.something.aggregate(
{ $project: {
something: {
0: { $add: [ "$something.first.0", "$something.second.0" ] },
1: { $add: [ "$something.first.1", "$something.second.1" ] },
2: { $add: [ "$something.first.2", "$something.second.2" ] }
}
}}
)
Sample output:
{
"result" : [
{
"_id" : ObjectId("517ae4914bc9ade96ca6402d"),
"something" : {
"0" : 4,
"1" : 14,
"2" : 9
}
}
],
"ok" : 1
}
If you don't know the names (i.e. you are using trying to add the 0th element of every array element in a document) you will need to use MapReduce or implement the logic in your application code instead.