Search date in array Mongodb - mongodb

Question

I have many documents like this one:
{
"_id" : ObjectId("54a94200aa76d3db6cd51977"),
"URL" : "http://...",
"Statistics" : [
{
"Date" : ISODate("2010-05-18T18:07:29.000+0000"),
"Clicks" : NumberInt(250),
},
{
"Date" : ISODate("2010-05-21T12:06:41.000+0000"),
"Clicks" : NumberInt(165),
},
{
"Date" : ISODate("2010-05-30T08:37:50.000+0000"),
"Clicks" : NumberInt(263),
}
]
}
My query looks like this:
db.clicks.aggregate([
{ $match : 'Statistics.Date' : { $gte: new Date("2010-05-18T00:00:00.000Z"), $lte: new Date("2010-05-18T23:59:59.999Z") } },
{ $unwind' => '$Statistics' },
{ $group : { _id : { year : { $year : '$Statistics.Date' }, month : { $month : '$Statistics.Date' }, day : { $dayOfMonth : '$Statistics.Date' } }, Clicks : { $sum : '$Statistics.Clicks' } },
{ $sort : { _id : 1 } }
])
When I try to sum up the clicks from a specific date it gives me all dates, instead of only one. What am I doing wrong? Thanks in advance.
Edit 1:
As there are >80.000 documents in that collection I can't do a $unwind before the $match. Also afaik this would be not a good idea, 'cause that would make the query slower than necessary.
The huge amount of documents and data in it is the reason why I have to use $sum. The document I made above is just an example and only the structure is the same as in my project.
The above query gives me back smth like this:
{
"_id" : [
{
"year" : 2010,
"month" : 5,
"day" : 18
}
],
"Clicks" : 250
},
{
"_id" : [
{
"year" : 2010,
"month" : 4,
"day" : 21
}
],
"Clicks" : 165
},
{
"_id" : [
{
"year" : 2010,
"month" : 5,
"day" : 30
}
],
"Clicks" : 263
}
If I don't use $group I also have to use $limit as the query would exceed 16MB otherwise:
db.clicks.aggregate([
{ $match : 'Statistics.Date' : { $gte: new Date("2010-05-18T00:00:00.000Z"), $lte: new Date("2010-05-18T23:59:59.999Z") } },
{ $unwind' : '$Statistics' },
{ $limit : 1 }
])
This result:
{
"_id" : ObjectId("54a94200aa76d3db6cd51977"),
"URL" : "http://...",
"Statistics" : {
"Date" : {
"sec" : 1274166878,
"usec" : 0
},
"Clicks" : 250
}
}
Due to performance reasons I have to use $group and not using it is not an option.
As I have all done in PHP there may be some errors in the document, queries and results I mentioned. Hopefully this won't be a problem. I still haven't figured out what's causing my problem. Can anyone help me?
Edit 2:
As this seems to be an performance issue which can't be solved I'm migrating all the data from the 'Statistics' array into an own collection. Thx to anyone for your help.

Answer 1

You need to run your $match twice, both before and after the $unwind:
db.clicks.aggregate([
{ $match : { 'Statistics.Date' : {
$gte: new ISODate("2010-05-18T00:00:00.000Z"),
$lte: new ISODate("2010-05-18T23:59:59.999Z") } } },
{ $unwind: '$Statistics' },
{ $match : { 'Statistics.Date' : {
$gte: new ISODate("2010-05-18T00:00:00.000Z"),
$lte: new ISODate("2010-05-18T23:59:59.999Z") } } },
{ $group : {
_id : { year : { $year : '$Statistics.Date' },
month : { $month : '$Statistics.Date' },
day : { $dayOfMonth : '$Statistics.Date' } },
Clicks : { $sum : '$Statistics.Clicks' } } },
{ $sort : { _id : 1 } }
])
The first $match is used to select the documents with at least one Statistics element in the right date range. The second one is used to filter out the other Statistics elements of those docs that aren't in the right date range.

Answer 2

Things may have been solved but posting answer for ones who are seeking help from this question
{ $match : 'Statistics.Date' : { $gte: new Date("2010-05-18T00:00:00.000Z"),
enter code here$lte: new Date("2010-05-18T23:59:59.999Z") } }
this match will filter main documents. What you want is to filter the documents inside Statistics array.
Now documents filtered by $match will contain full Statistic array. And unwinding after filtering may have Sub-document of Statistic whose sibling document(document that are in the same array) have passed $match condition.
Note: simple find projection:
db.col_name.find({},{"Statistics.$":1}) will filter array too but
$project in aggregation is not helping in filtering array of documents.