How to retrieve a sub document array in MongoDB - mongodb

I have a collection in mongodb like this:
db.country_list.find().pretty()
{
"_id" : ObjectId("53917321ccbc96175d7a808b"),
"countries" : [
{
"countryName" : "Afghanistan",
"iso3" : "AFG",
"callingCode" : "93"
},
{
"countryName" : "Aland Islands",
"iso3" : "ALA",
"callingCode" : "358"
},
{
"countryName" : "Albania",
"iso3" : "ALB",
"callingCode" : "355"
}
]
}
like that i have 100 country details
i want to retrieve a country name where the calling code is 355.
I have tried like this
db.country_list.find({countries: {$elemMatch :{ 'callingCode':'355'} } } )
and like this
db.country_list.find({'countries.callingCode':'355'}).pretty()
but i am getting all records.How to get a specific record .Thanks in advance

What you want is the positional $ operator:
db.country_list.find(
{ "countries": { "$elemMatch" :{ "callingCode":"355"} } }
{ "countries.$": 1 }
)
Or even with the other syntax you tried:
db.country_list.find(
{ "countries.callingCode": "355"}
{ "countries.$": 1 }
)
This is because a "query" matches documents and is not a filter for the array contained in those documents. So the second argument there projects the field with the "position" that was matched on the query side.
If you need to match more than one array element, then you use the aggregation framework which has more flexibility:
db.country_list.aggregate([
// Matches the documents that "contain" the match
{ "$match": {
"countries.callingCode": "355"
}},
// Unwind the array to de-normalize as documents
{ "$unwind": "$countries" },
// Match to "filter" the array content
{ "$match": {
"countries.callingCode": "355"
}},
// Group back if you want an array
{ "$group": {
"_id": "$_id",
"countries": { "$push": "$countries" }
}}
])
Or with MongoDB 2.6 or greater you can do this without the $unwind and $group:
db.country_list.aggregate([
// Matches the documents that "contain" the match
{ "$match": {
"countries.callingCode": "355"
}},
// Project with "$map" to filter
{ "$project": {
"countries": {
"$setDifference": [
{ "$map": {
"input": "$countries",
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el.callingCode", "355" ] }
"$$el",
false
]
}
}},
[false]
]
}
}}
])

Related

$match in date after $project and $subtract returns empty result in MongoDB

The documents follow this structure:
{
"_id" : ObjectId("5a01b474d88dc4001e684c97"),
"created_time" : ISODate("2017-11-07T11:26:12.563+0000"),
"posts" : [
{
"story" : "Test",
"created_time" : ISODate("2017-11-06T17:38:02.000+0000"),
"id" : "769055806629274_768721009996087",
"_id" : ObjectId("5a01b498d88dc4001e68553c")
},
{
"story" : "Test",
"created_time" : ISODate("2017-11-05T12:00:00.000+0000"),
"id" : "1637086293239159_2011737915773993",
"_id" : ObjectId("5a01b498d88dc4001e68553d")
}
[...]
]
}
I want to filter the posts collection by created_time. Each post needs to have created_time greater than created_time of the document. In other words, I want to get posts only for the last month based on the document.
I'm trying this aggregation:
db.collection.aggregate([
{
$project: {
"past_month": {
$subtract: ["$created_time", 2629746000] //a month
},
"created_time": "$created_time",
"posts": "$posts"
}
}, {
$unwind: '$posts'
}, {
$match: {
"posts.created_time": {
$gte: "$past_month"
}
}
}, {
"$group": {
"_id": "$_id",
"posts": {
"$push": "$posts"
}
}
}
])
But the result is always empty. If I change $gte: "$past_month" to $gte: ISODate("2017-10-08T00:57:06.563+0000") to test, the results is not empty.
For the requirement:
Each post needs to have created_time greater than created_time of the
document
to be satisfied, with MongoDB Server version 3.4 and above, use the $addFields pipeline in conjunction with the $filter operator to filter the posts as:
db.collection.aggregate([
{
"$addFields": {
"posts": {
"$filter": {
"input": "$posts",
"as": "post",
"cond": {
"$gt": [
"$$post.created_time",
{ "$subtract": ["$created_time", 2629746000] }
]
}
}
}
}
}
])
The $addFields will replace the posts array with the filtered one in the expression above.
For MongoDB 3.2 you can still use $filter not within $addFields pipeline as it's not supported but with $project instead.
For MongoDB 3.0 use a combination of $setDifference and $map operators to filter the posts array as
db.collection.aggregate([
{
"$project": {
"created_time": 1,
"posts": {
"$setDifference": [
{
"$map": {
"input": "$posts",
"as": "post",
"in": {
"$cond": [
{
"$gt": [
"$$post.created_time",
{ "$subtract": ["$created_time", 2629746000] }
]
},
{
"story" : "$$post.story",
"created_time" : "$$post.created_time",
"id" : "$$post.id",
"_id" : "$$post._id",
},
false
]
}
}
},
[false]
]
}
}
}
])
You can do this in simple and clean way with the help of moment.js library -
var checkForDate = moment().subtract(1, 'months');
var startDate= moment(checkForDate).startOf('month');
var endDate= moment(checkForDate).endOf('month');
db.collection.find({
"posts.created_time":{
$lt:endDate,
$gt:startDate
}
})
you can achieve your desired result with this without using aggregate chain mechanism

Querying MongoDB array of similar objects

I've to work with old MongoDB where objects in one collection are structured like this.
{
"_id": ObjectId("57fdfcc7a7c81fde38b79a3d"),
"parameters": [
{
"key": "key1",
"value": "value1"
},
{
"key": "key2",
"value": "value2"
}
]
}
The problem is that parameters is an array of objects, which makes efficient querying difficult. There can be about 50 different objects, which all have "key" and "value" properties. Is it possible to make a query, where the query targets "key" and "value" inside one object? I've tried
db.collection.find({$and:[{"parameters.key":"value"}, {"parameters.value":"another value"}]})
but this query hits all the objects in parameters array.
EDIT. Nikhil Jagtiani found solution to my original question, but actually I should be able query to target multiple objects inside parameters array. E.g. check keys and values in two different objects in parameters array.
Please refer below mongo shell aggregate query :
db.collection.aggregate([
{
$unwind:"$parameters"
},
{
$match:
{
"parameters.key":"key1",
"parameters.value":"value1"
}
}
])
1) Stage 1 - Unwind : Deconstructs an array field from the input documents to output a document for each element. Each output document is the input document with the value of the array field replaced by the element.
2) Stage 2 - Match : Filters the documents to pass only the documents that match the specified condition(s) to the next pipeline stage.
Without aggregation, queries will return the entire document even if one subdocument matches. This pipeline will only return the required subdocuments.
Edit: If you need to specify multiple key value pairs, what we need is $in for parameters field.
db.collection.aggregate([{$unwind:"$parameters"},{$match:{"parameters":{$in:[{ "key" : "key1", "value" : "value1"},{ "key" : "key2", "value" : "value2" }]}}}])
will match the following two pairs of key-values as subdocuments:
1) { "key" : "key1", "value" : "value1" }
2) { "key" : "key2", "value" : "value2" }
There is a $filter operator in the aggregation framework which is perfect for such queries. A bit verbose but very efficient, you can use it as follows:
db.surveys.aggregate([
{ "$match": {
"$and": [
{
"parameters.key": "key1",
"parameters.value": "val1"
},
{
"parameters.key": "key2",
"parameters.value": "val2"
}
]
}},
{
"$project": {
"parameters": {
"$filter": {
"input": "$parameters",
"as": "item",
"cond": {
"$or": [
{
"$and" : [
{ "$eq": ["$$item.key", "key1"] },
{ "$eq": ["$$item.value", "val1"] }
]
},
{
"$and" : [
{ "$eq": ["$$item.key", "key2"] },
{ "$eq": ["$$item.value", "val2"] }
]
}
]
}
}
}
}
}
])
You can also do this with more set operators in MongoDB 2.6 without using $unwind:
db.surveys.aggregate([
{ "$match": {
"$and": [
{
"parameters.key": "key1",
"parameters.value": "val1"
},
{
"parameters.key": "key2",
"parameters.value": "val2"
}
]
}},
{
"$project": {
"parameters": {
"$setDifference": [
{ "$map": {
"input": "$parameters",
"as": "item",
"in": {
"$cond": [
{ "$or": [
{
"$and" : [
{ "$eq": ["$$item.key", "key1"] },
{ "$eq": ["$$item.value", "val1"] }
]
},
{
"$and" : [
{ "$eq": ["$$item.key", "key2"] },
{ "$eq": ["$$item.value", "val2"] }
]
}
]},
"$$item",
false
]
}
}},
[false]
]
}
}
}
])
For a solution with MongoDB 2.4, you would need to use the $unwind operator unfortunately:
db.surveys.aggregate([
{ "$match": {
"$and": [
{
"parameters.key": "key1",
"parameters.value": "val1"
},
{
"parameters.key": "key2",
"parameters.value": "val2"
}
]
}},
{ "$unwind": "$parameters" },
{ "$match": {
"$and": [
{
"parameters.key": "key1",
"parameters.value": "val1"
},
{
"parameters.key": "key2",
"parameters.value": "val2"
}
]
}},
{
"$group": {
"_id": "$_id",
"parameters": { "$push": "$parameters" }
}
}
]);
Is it possible to make a query, where the query targets "key" and
"value" inside one object?
This is possible if you know which object(id) you are going to query upfront(to be given as input parameter in the find query). If that is not possible then we can try on the below approach for efficient querying.
Build an index on the parameters.key and if needed also on parameters.value. This would considerably improve the query performance.
Please see
https://docs.mongodb.com/manual/indexes/
https://docs.mongodb.com/manual/core/index-multikey/

Mongodb count all array elements in all objects matching by criteria

I have a collection that is log of activity on objects like this:
{
"_id" : ObjectId("55e3fd1d7cb5ac9a458b4567"),
"object_id" : "1",
"activity" : [
{
"action" : "test_action",
"time" : ISODate("2015-08-31T00:00:00.000Z")
},
{
"action" : "test_action",
"time" : ISODate("2015-08-31T00:00:22.000Z")
}
]
}
{
"_id" : ObjectId("55e3fd127cb5ac77478b4567"),
"object_id" : "2",
"activity" : [
{
"action" : "test_action",
"time" : ISODate("2015-08-31T00:00:00.000Z")
}
]
}
{
"_id" : ObjectId("55e3fd0f7cb5ac9f458b4567"),
"object_id" : "1",
"activity" : [
{
"action" : "test_action",
"time" : ISODate("2015-08-30T00:00:00.000Z")
}
]
}
If i do followoing query:
db.objects.find({
"createddate": {$gte : ISODate("2015-08-30T00:00:00.000Z")},
"activity.action" : "test_action"}
}).count()
it returns count of documents containing "test_action" (3 in this set), but i need to get count of all test_actions (4 on this set). How do i do that?
The most "performant" way to do this is to skip the $unwind altogther and simply $group to count. Essentially "filter" arrays get the $size of the results to $sum:
db.objects.aggregate([
{ "$match": {
"createddate": {
"$gte": ISODate("2015-08-30T00:00:00.000Z")
},
"activity.action": "test_action"
}},
{ "$group": {
"_id": null,
"count": {
"$sum": {
"$size": {
"$setDifference": [
{ "$map": {
"input": "$activity",
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el.action", "test_action" ] },
"$$el",
false
]
}
}},
[false]
]
}
}
}
}}
])
Since MongoDB version 3.2 we can use $filter, which makes this much more simple:
db.objects.aggregate([
{ "$match": {
"createddate": {
"$gte": ISODate("2015-08-30T00:00:00.000Z")
},
"activity.action": "test_action"
}},
{ "$group": {
"_id": null,
"count": {
"$sum": {
"$size": {
"$filter": {
"input": "$activity",
"as": "el",
"cond": {
"$eq": [ "$$el.action", "test_action" ]
}
}
}
}
}
}}
])
Using $unwind causes the documents to de-normalize and effectively creates a copy per array entry. Where possible you should avoid this due the the often extreme cost. Filtering and counting array entries per document is much faster by comparison. As is a simple $match and $group pipeline compared to many stages.
You can do so by using aggregation:
db.objects.aggregate([
{$match: {"createddate": {$gte : ISODate("2015-08-30T00:00:00.000Z")}, {"activity.action" : "test_action"}}},
{$unwind: "$activity"},
{$match: {"activity.action" : "test_action"}}},
{$group: {_id: null, count: {$sum: 1}}}
])
This will produce a result like:
{
count: 4
}

How can I select document with array items containing in values array?

I have collection in mongodb (3.0):
{
_id: 1,
m: [{_id:11, _t: 'type1'},
{_id:12, _t: 'type2'},
{_id:13, _t: 'type3'}]
},
{
_id: 2,
m: [{_id:21, _t: 'type1'},
{_id:22, _t: 'type21'},
{_id:23, _t: 'type3'}]
}
I want to find documents with m attributes where m._t containing ['type1', 'type2'].
Like this:
{
_id: 1,
m: [{_id:11, _t: 'type1'},
{_id:12, _t: 'type2'}]
},
{
_id: 2,
m: [{_id:21, _t: 'type1'}]
}
I tried to use $ and $elemMatch, but couldn't get required result.
How to do it, using find()?
Help me, please! Thanks!
Because the $elemMatch operator limits the contents of the m array field from the query results to contain only the first element matching the $elemMatch condition, the following will only return the an array with the first matching elements
{
"_id" : 11,
"_t" : "type1"
}
and
{
"_id" : 21,
"_t" : "type1"
}
Query using $elemMatch projection:
db.collection.find(
{
"m._t": {
"$in": ["type1", "type2"]
}
},
{
"m": {
"$elemMatch": {
"_t": {
"$in": ["type1", "type2"]
}
}
}
}
)
Result:
/* 0 */
{
"_id" : 1,
"m" : [
{
"_id" : 11,
"_t" : "type1"
}
]
}
/* 1 */
{
"_id" : 2,
"m" : [
{
"_id" : 21,
"_t" : "type1"
}
]
}
One approach you can take is the aggregation framework, where your pipeline would consist of a $match operator, similar to the find query above to filter the initial stream of documents. The next pipeline step would be the crucial $unwind operator that "splits" the array elements to be further streamlined with another $match operator and then the final $group pipeline to restore the original data structure by using the accumulator operator $push.
The following illustrates this path:
db.collection.aggregate([
{
"$match": {
"m._t": {
"$in": ["type1", "type2"]
}
}
},
{
"$unwind": "$m"
},
{
"$match": {
"m._t": {
"$in": ["type1", "type2"]
}
}
},
{
"$group": {
"_id": "$_id",
"m": {
"$push": "$m"
}
}
}
])
Sample Output:
/* 0 */
{
"result" : [
{
"_id" : 2,
"m" : [
{
"_id" : 21,
"_t" : "type1"
}
]
},
{
"_id" : 1,
"m" : [
{
"_id" : 11,
"_t" : "type1"
},
{
"_id" : 12,
"_t" : "type2"
}
]
}
],
"ok" : 1
}
To get your "filtered" result, the $redact with the aggregation pipeline is the fastest way:
db.junk.aggregate([
{ "$match": { "m._t": { "$in": ["type1", "type2"] } } },
{ "$redact": {
"$cond": {
"if": {
"$or": [
{ "$eq": [ { "$ifNull": ["$_t", "type1"] }, "type1" ] },
{ "$eq": [ { "$ifNull": ["$_t", "type2"] }, "type2" ] }
],
},
"then": "$$DESCEND",
"else": "$$PRUNE"
}
}}
])
The $redact operator sets up a logical filter for the document that can also traverse into the array levels. Note that this is matching on _t at all levels of the document, so make sure there are no other elements sharing this name.
The query uses $in for selection just as the logical filter uses $or. Anything that does not match, gets "pruned".
{
"_id" : 1,
"m" : [
{
"_id" : 11,
"_t" : "type1"
},
{
"_id" : 12,
"_t" : "type2"
}
]
}
{
"_id" : 2,
"m" : [ { "_id" : 21, "_t" : "type1" } ]
}
Short and sweet and simple.
A bit more cumbersome, but a reasonably safer is to use this construct with $map and $setDifference to filter results:
db.junk.aggregate([
{ "$match": { "m._t": { "$in": ["type1", "type2"] } } },
{ "$project": {
"m": {
"$setDifference": [
{ "$map": {
"input": "$m",
"as": "el",
"in": {
"$cond": {
"if": {
"$or": [
{ "$eq": [ "$$el._t", "type1" ] },
{ "$eq": [ "$$el._t", "type2" ] }
]
},
"then": "$$el",
"else": false
}
}
}},
[false]
]
}
}}
])
The $map evaluates the conditions against each element and the $setDifference removes any of those condtions that returned false rather than the array content. Very similar to the $cond in redact above, but it is just working specifically with the one array and not the whole document.
In future MongoDB releases ( currently available in development releases ) there will be the $filter operator, which is very simple to follow:
db.junk.aggregate([
{ "$match": { "m._t": { "$in": ["type1", "type2"] } } },
{ "$project": {
"m": {
"$filter": {
"input": "$m",
"as": "el",
"cond": {
"$or": [
{ "$eq": [ "$$el._t", "type1" ] },
{ "$eq": [ "$$el._t", "type2" ] }
]
}
}
}
}}
])
And that will simply remove any array element that does not match the specified conditions.
If you want to filter array content on the server, the aggregation framework is the way to do it.

How to find document and single subdocument matching given criterias in MongoDB collection

I have collection of products. Each product contains array of items.
> db.products.find().pretty()
{
"_id" : ObjectId("54023e8bcef998273f36041d"),
"shop" : "shop1",
"name" : "product1",
"items" : [
{
"date" : "01.02.2100",
"purchasePrice" : 1,
"sellingPrice" : 10,
"count" : 15
},
{
"date" : "31.08.2014",
"purchasePrice" : 10,
"sellingPrice" : 1,
"count" : 5
}
]
}
So, can you please give me an advice, how I can query MongoDB to retrieve all products with only single item which date is equals to the date I pass to query as parameter.
The result for "31.08.2014" must be:
{
"_id" : ObjectId("54023e8bcef998273f36041d"),
"shop" : "shop1",
"name" : "product1",
"items" : [
{
"date" : "31.08.2014",
"purchasePrice" : 10,
"sellingPrice" : 1,
"count" : 5
}
]
}
What you are looking for is the positional $ operator and "projection". For a single field you need to match the required array element using "dot notation", for more than one field use $elemMatch:
db.products.find(
{ "items.date": "31.08.2014" },
{ "shop": 1, "name":1, "items.$": 1 }
)
Or the $elemMatch for more than one matching field:
db.products.find(
{ "items": {
"$elemMatch": { "date": "31.08.2014", "purchasePrice": 1 }
}},
{ "shop": 1, "name":1, "items.$": 1 }
)
These work for a single array element only though and only one will be returned. If you want more than one array element to be returned from your conditions then you need more advanced handling with the aggregation framework.
db.products.aggregate([
{ "$match": { "items.date": "31.08.2014" } },
{ "$unwind": "$items" },
{ "$match": { "items.date": "31.08.2014" } },
{ "$group": {
"_id": "$_id",
"shop": { "$first": "$shop" },
"name": { "$first": "$name" },
"items": { "$push": "$items" }
}}
])
Or possibly in shorter/faster form since MongoDB 2.6 where your array of items contains unique entries:
db.products.aggregate([
{ "$match": { "items.date": "31.08.2014" } },
{ "$project": {
"shop": 1,
"name": 1,
"items": {
"$setDifference": [
{ "$map": {
"input": "$items",
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el.date", "31.08.2014" ] },
"$$el",
false
]
}
}},
[false]
]
}
}}
])
Or possibly with $redact, but a little contrived:
db.products.aggregate([
{ "$match": { "items.date": "31.08.2014" } },
{ "$redact": {
"$cond": [
{ "$eq": [ { "$ifNull": [ "$date", "31.08.2014" ] }, "31.08.2014" ] },
"$$DESCEND",
"$$PRUNE"
]
}}
])
More modern, you would use $filter:
db.products.aggregate([
{ "$match": { "items.date": "31.08.2014" } },
{ "$addFields": {
"items": {
"input": "$items",
"cond": { "$eq": [ "$$this.date", "31.08.2014" ] }
}
}}
])
And with multiple conditions, the $elemMatch and $and within the $filter:
db.products.aggregate([
{ "$match": {
"$elemMatch": { "date": "31.08.2014", "purchasePrice": 1 }
}},
{ "$addFields": {
"items": {
"input": "$items",
"cond": {
"$and": [
{ "$eq": [ "$$this.date", "31.08.2014" ] },
{ "$eq": [ "$$this.purchasePrice", 1 ] }
]
}
}
}}
])
So it just depends on whether you always expect a single element to match or multiple elements, and then which approach is better. But where possible the .find() method will generally be faster since it lacks the overhead of the other operations, which in those last to forms does not lag that far behind at all.
As a side note, your "dates" are represented as strings which is not a very good idea going forward. Consider changing these to proper Date object types, which will greatly help you in the future.
Based on Neil Lunn's code I work with this solution, it includes automatically all first level keys (but you could also exclude keys if you want):
db.products.find(
{ "items.date": "31.08.2014" },
{ "shop": 1, "name":1, "items.$": 1 }
{ items: { $elemMatch: { date: "31.08.2014" } } },
)
With multiple requirements:
db.products.find(
{ "items": {
"$elemMatch": { "date": "31.08.2014", "purchasePrice": 1 }
}},
{ items: { $elemMatch: { "date": "31.08.2014", "purchasePrice": 1 } } },
)
Mongo supports dot notation for sub-queries.
See: http://docs.mongodb.org/manual/reference/glossary/#term-dot-notation
Depending on your driver, you want something like:
db.products.find({"items.date":"31.08.2014"});
Note that the attribute is in quotes for dot notation, even if usually your driver doesn't require this.