How to query for a nested date range in Mongoid - mongodb

Question

Ok, here's my data:
"stats" : [
{
"campaign_id" : "some_id",
"log_id" : "some_id",
"agent" : "some_id",
"office" : "some_id",
"hq" : "some_name",
"seller" : "some_name",
"status" : "live",
"phases" : [
{
"phase" : "main_phase",
"banners" : [
{
"banner_id" : "some_id_same_as_below",
"split_var" : "light",
"reports" : [
{
"date" : "2016-11-25",
"banner" : "some_id_same_as_above",
"cost" : "0.231",
"impressions" : 14,
"clicks" : 0
},
...
And I need to query the database for all reports:
"reports" : [
{
"date" : "2016-11-25",
"banner" : "some_id_same_as_above",
"cost" : "0.231",
"impressions" : 14,
"clicks" : 0
},
For the "date" : "2016-11-25" within a date range. For the date range I have this:
start_month = DateTime.current.beginning_of_month - 1.month
end_month = DateTime.current.end_of_month - 1.month
Which gives me start and end of the previous month, which is right. How can I search for all documents that have reports (the nested values inside stats, phases, etc) that falls within this range?
Any ideas?
EDIT
It has been suggested to change the way the data is inserted into the db, but unfortunately I have no control on how the data is inserted (done by a third party service/API).

Answer 1

You can store the dates as standard Date objects instead of formatted strings, which MongoDB stores in ISODate format, for instance:
db.collection.insert({date: new Date()});
will have a field like:
{ "date" : ISODate("2016-11-15T15:50:15.167Z") }
Then you can query by date range (Use the $and operator if you need to query between two ranges or the second statement may override the first)
Such as:
// Return all documents in collection with a date between 11-1-2016 and 12-1-2016
db.collection.find({
$and: [
{ date: { $gte: ISODate("2016-11-01T00:00:00.000Z") } },
{ date: { $lt: ISODate("2016-12-01T00:00:00.000Z") } }
]
})
EDIT: In case you cannot modify your collections, you could do a regex style search...
For instance:
db.collection.find({
"stats.phases.banners.reports.date": /2016-11/
});
will return all documents for November 2016 since it matches on all strings containing "2016-11"
EDIT AGAIN:
Here is a solution using the aggregation framework to return the documents in the format you mentioned above i.e.
{
"reports" : {
"date" : "2016-11-23",
"banner" : ObjectId("58404a9450b5412e92ebbb97"),
"cost" : "0.231",
"impressions" : 14,
"clicks" : 0
}
},
{
"reports" : {
"date" : "2016-11-25",
"banner" : ObjectId("58404a9450b5412e92ebbb97"),
"cost" : "0.231",
"impressions" : 14,
"clicks" : 0
}
}
Note you have to do a lot of unwinds due to your heavily nested array structure...
db.collection.aggregate([
{ $unwind: "$stats" },
{ $unwind: "$stats.phases" },
{ $unwind: "$stats.phases.banners" },
{ $unwind: "$stats.phases.banners.reports" },
{ $match: { "stats.phases.banners.reports.date": /2016-11/ } },
{ $project: { _id: 0, reports: "$stats.phases.banners.reports" } }
])

Answer 2

You need to store date fields in as ISODate objects and then you can use comparison operators like $lt, $lte, $gt, $gte, etc.
Here is how to insert data:
db.test.insert({
"stats": [
{
"campaign_id": "some_id",
"log_id": "some_id",
"agent": "some_id",
"office": "some_id",
"hq": "some_name",
"seller": "some_name",
"status": "live",
"phases": [
{
"phase": "main_phase",
"banners": [
{
"banner_id": "some_id_same_as_below",
"split_var": "light",
"reports": [
{
"date": ISODate("2016-11-25T00:00:00.0Z"),
"banner": "some_id_same_as_above",
"cost": "0.231",
"impressions": 14,
"clicks": 0
}
]
}
]
}
]
}
]
})
Following is the query to find documents whose stats.phases.banners.reports.date is between 25 Nov 2016 to 15 Dec 2016.
db.test.find({"stats.phases.banners.reports.date": {$lt: ISODate("2016-11-25T00:00:00.0Z"), $gt: ISODate("2016-12-25T00:00:00.0Z")}})