{
"_id" : ObjectId("568b650543712795bf864a45")
"companyId" : "55e2d7cfdc8f74d14f5c900f",
"timeStamp" : ISODate("2014-12-02T18:30:00.000Z")
},
{
"_id" : ObjectId("568b650543712795bf864a46")
"companyId" : "55e2d7cfdc8f74d14f5c900f",
"timeStamp" : ISODate("2014-12-03T18:30:00.000Z")
},
{
"_id" : ObjectId("568b650543712795bf864a47")
"companyId" : "55e2d7cfdc8f74d14f5c900f",
"timeStamp" : ISODate("2014-12-04T18:30:00.000Z")
}
retrieve all documents in mondays from timeStamp field from last 7 weeks.
You have to use mongodb aggregation framework to achieve this.
Find date of start (current day - 7 weeks) in whatever programming language you are using.
Then you have to use aggregation operation $dayOfWeek to achieve this
var pipeline = [
{
$match: {timeStamp: {$gte: startDate}}
},
{
$project: {dayOfWeek: {$dayOfWeek: '$timeStamp'}}
},
{
$match: {dayOfWeek: 1}
}
];
db.mycollection.aggreage(pipeline)
In above I have projected only one field, you may do project more fields.
For more information please click $dayOfWeek
Related
Suppose, we have this type of documents :
{
"_id" : ObjectId("4c02c58de500fe1be1000305"),
"date" : ""20090530"
}
{
"_id" : ObjectId("4c02c58de500fe1be1000005"),
"date" : ""20090607"
}
Is it possible to find/group all documents with 7 days between them?
With aggregate?
You can write something like this, I believe you won't be needing aggregation here, if you are doing this for the same collection:
db.collectionName.find({date: { $gte: "20090530", $lte: "20090607" } } );
Reference: $lte and $gte
I have the following documents,
{
"_id" : ObjectId("5b85312981c1634f59751604"),
"date" : "0"
},
{
"_id" : ObjectId("5b85312981c1634f59751604"),
"date" : "20180330"
},
{
"_id" : ObjectId("5b85312981c1634f59751604"),
"date" : "20180402"
},
{
"_id" : ObjectId("5b85312981c1634f59751604"),
"date" : "20180323"
},
I tried to convert date to ISODate using $toDate in aggregation,
db.documents.aggregate( [ { "$addFields": { "received_date": { "$cond": [ {"$ne": ["$date", "0"] }, {"$toDate": "$date"}, new Date("1970-01-01") ] } } } ] )
the query executed fine, but when I
db.documents.find({})
to examine all the documents, nothing changed, I am wondering how to fix it. I am using MongoDB 4.0.6 on Linux Mint 19.1 X64.
As they mentioned in the comments, aggregate doesn't update documents in the database directly (just an output of them).
If you'd like to permanently add a new field to documents via aggregation (aka update the documents in the database), use the following .forEach/.updateOne method:
Your example:
db.documents
.aggregate([{"$addFields":{"received_date":{"$cond":[{"$ne":["$date","0"]}, {"$toDate": "$date"}, new Date("1970-01-01")]}}}])
.forEach(function (x){db.documents.updateOne({_id: x._id}, {$set: {"received_date": x.received_date}})})
Since _id's value is an ObjectID(), there may be a slight modification you need to do to {_id:x._id}. If there is, let me know and I'll update it!
Another example:
db.users.find().pretty()
{ "_id" : ObjectId("5acb81b53306361018814849"), "name" : "A", "age" : 1 }
{ "_id" : ObjectId("5acb81b5330636101881484a"), "name" : "B", "age" : 2 }
{ "_id" : ObjectId("5acb81b5330636101881484b"), "name" : "C", "age" : 3 }
db.users
.aggregate([{$addFields:{totalAge:{$sum:"$age"}}}])
.forEach(function (x){db.users.updateOne({name: x.name}, {$set: {totalAge: x.totalAge}})})
Being able to update collections via the aggregation pipeline seems to be quite valuable because of what you have the power to do with aggregation (e.g. what you did in your question, doing calculations based on other fields within the document, etc.). I'm newer to MongoDB so maybe updating collections via aggregation pipeline is "bad practice", but it works and it's been quite valuable for me. I wonder why it isn't more straight-forward to do?
Note: I came up with this method after discovering Nazo's now-deprecated .save() method. Shoutout to Nazo!
This is how my collection structure looks like:
{
"_id" : ObjectId("57589d2a9108dace306602b8"),
"IDproject" : NumberLong(53),
"email" : "john.doe#gmail.com",
"dc" : ISODate("2016-06-06T22:33:13.000Z")
}
{
"_id" : ObjectId("57589d2a9108dace306602b8"),
"IDproject" : NumberLong(53),
"email" : "david.doe#gmail.com",
"dc" : ISODate("2016-06-07T22:33:13.000Z")
}
{
"_id" : ObjectId("57589d2a9108dace306602b8"),
"IDproject" : NumberLong(53),
"email" : "elizabeth.doe#gmail.com",
"dc" : ISODate("2016-06-078T22:33:13.000Z")
}
As you can see, there are two customers added on June 7th and one on June 6th. I would like to group and sum these results for the last 30 days.
It should looks something like this:
{
"dc" : "2016-06-05"
"total" : 0
}
{
"dc" : "2016-06-06"
"total" : 1
}
{
"dc" : "2016-06-07"
"total" : 2
}
As, you can see, there are no records on June 6th, so it's zero. It should be zero for June 5th, etc.
That would be the case #1, and the case #2 are following results:
{
"dc" : "2016-06-05"
"total" : 0
}
{
"dc" : "2016-06-06"
"total" : 1
}
{
"dc" : "2016-06-07"
"total" : 3
}
I've tried this:
db.getCollection('customer').aggregate([
{$match : { IDproject : 53}},
{ $group: { _id: "$dc", total: { $sum: "$dc" } } }, ]);
But seems complicated. I'm first time working with noSQL database.
Thanks.
Here's how you will get daily counts (the common idiom for row count is {$sum: 1}).
However, you cannot obtain zeros for days that are lacking data – because there is no data that would give the grouping key for these days. You must handle these cases in PHP by generating a list of desided dates and then looking if there's data for that each date.
db.getCollection('customer').aggregate([
{$match : { IDproject : 53}},
{$group: {
_id: {year: {$year: "$dc"}, month: {$month: "$dc"}, day: {$dayOfMonth: "$dc"}}},
total: {$sum: 1}
}},
]);
Note that MongoDB only operates in the UTC timezone; there are no aggregation pipeline operators that can convert timestamps to local timezones reliably. The $year, $month and $dayOfMonth operators give the date in UTC which may not be the same as in the local timezone. Solutions include:
saving timestamps in the local timezone (= lying to MongoDB that they are in UTC),
saving the timezone offset with the timestamp,
saving the local year, month and dayOfMonth with the timestamp.
is it possible to query only the first (or last or any single?) day of the month of a mongo date field.
i use the $date aggregation operators regularly but within a $group clause.
basically i have field that is already aggregated (averaged) for each day of the month. i want to select only one of these days (with the value as a representative of the entire month.)
following is a sample of a record set from jan 1, 2014 to feb 1, 2015 with price as the daily price and 28day_avg as the trailing monthly average for 28 days.
{ "date" : ISODate("2014-01-01T00:00:00Z"), "_id" : ObjectId("533b3697574e2fd08f431cff"), "price": 59.23, "28day_avg": 54.21}
{ "date" : ISODate("2014-01-02T00:00:00Z"), "_id" : ObjectId("533b3697574e2fd08f431cff"), "price": 58.75, "28day_avg": 54.15}
...
{ "date" : ISODate("2015-02-01T00:00:00Z"), "_id" : ObjectId("533b3697574e2fd08f431cff"), "price": 123.50, "28day_avg": 122.25}
method 1.
im currently running an aggregation using $month data (and summing the price) but one issue is im seeking to retrieve the underlying date value ISODate("2015-02-01T00:00:00Z") versus the 0,1,2 value that comes with several of the date aggregations (that loop at the first of the week, month, year). mod(28) on a date?
method 2
i'd like to simply pluck out a single record of the 28day_avg as representative of the period. the 1st of the month would be adequate
the desired output is...
_id: ISODate("2015-02-01T00:00:00Z"), value: 122.25,
_id: ISODate("2015-01-01T00:00:00Z"), value: 120.78,
_id: ISODate("2014-12-01T00:00:00Z"), value: 118.71,
...
_id: ISODate("2014-01-01T00:00:00Z"), value: 53.21,
of course, the value will vary from method 1 to method 2 but that is fine. one is 28 days trailing while the other will account for 28, 30, 31 day months...dont care about that so much.
A non-agg is ok but also doesnt work. aka {"date": { "$mod": [ 28, 0 ]} }
To pick the first of the month for each month (method 2), use the following aggregation:
db.test.aggregate([
{ "$project" : { "_id" : "$date", "day" : { "$dayOfMonth" : "$date" }, "28day_avg" : 1 } },
{ "$match" : { "day" : 1 } }
])
You can't use an index for the match, so this is not efficient. I'd suggest adding another field to each document that holds the $dayOfMonth value, so you can index it and do a simple find:
{
"date" : ISODate("2014-01-01T00:00:00Z"),
"price" : 59.23,
"28day_avg" : 54.21,
"dayOfMonth" : 1
}
db.test.ensureIndex({ "dayOfMonth" : 1 })
db.test.find({ "dayOfMonth" : 1 }, { "_id" : 0, "date" : 1, "28day_avg" : 1 })
I'm trying to group by timestamp for the collection named "foo" { _id, TimeStamp }
db.foos.aggregate(
[
{$group : { _id : new Date (Date.UTC({ $year : '$TimeStamp' },{ $month : '$TimeStamp' },{$dayOfMonth : '$TimeStamp'})) }}
])
Expecting many dates but the result is just one date. The data i'm using is correct (has many foo and different dates except 1970). There's some problem in the date parsing but i can not solve yet.
{
"result" : [
{
"_id" : ISODate("1970-01-01T00:00:00.000Z")
}
],
"ok" : 1
}
Tried this One:
db.foos.aggregate(
[
{$group : { _id : { year : { $year : '$TimeStamp' }, month : { $month : '$TimeStamp' }, day : {$dayOfMonth : '$TimeStamp'} }, count : { $sum : 1 } }},
{$project : { parsedDate : new Date('$_id.year', '$_id.month', '$_id.day') , count : 1, _id : 0} }
])
Result :
uncaught exception: aggregate failed: {
"errmsg" : "exception: disallowed field type Date in object expression (at 'parsedDate')",
"code" : 15992,
"ok" : 0
}
And that one:
db.foos.aggregate(
[
{$group : { _id : { year : { $year : '$TimeStamp' }, month : { $month : '$TimeStamp' }, day : {$dayOfMonth : '$TimeStamp'} }, count : { $sum : 1 } }},
{$project : { parsedDate : Date.UTC('$_id.year', '$_id.month', '$_id.day') , count : 1, _id : 0} }
])
Can not see dates in the result
{
"result" : [
{
"count" : 412
},
{
"count" : 1702
},
{
"count" : 422
}
],
"ok" : 1
}
db.foos.aggregate(
[
{ $project : { day : {$substr: ["$TimeStamp", 0, 10] }}},
{ $group : { _id : "$day", number : { $sum : 1 }}},
{ $sort : { _id : 1 }}
]
)
Group by date can be done in two steps in the aggregation framework, an additional third step is needed for sorting the result, if sorting is desired:
$project in combination with $substr takes the first 10 characters (YYYY:MM:DD) of the ISODate object from each document (the result is a collection of documents with the fields "_id" and "day");
$group groups by day, adding (summing) the number 1 for each matching document;
$sort ascending by "_id", which is the day from the previous aggregation step - this is optional if sorted result is desired.
This solution can not take advantage of indexes like db.twitter.ensureIndex( { TimeStamp: 1 } ), because it transforms the ISODate object to a string object on the fly. For large collections (millions of documents) this could be a performance bottleneck and more sophisticated approaches should be used.
It depends on whether you want to have the date as ISODate type in the final output. If so, then you can do one of two things:
Extract $year, $month, $dayOfMonth from your timestamp and then reconstruct a new date out of them (you are already trying to do that, but you're using syntax that doesn't work in aggregation framework).
If the original Timestamp is of type ISODate() then you can do date arithmetic to subtract the hours, minutes, seconds and milliseconds from your timestamp to get a new date that's "rounded" to the day.
There is an example of 2 here.
Here is how you would do 1. I'm making an assumption that all your dates are this year, but you can easily adjust the math to accommodate your oldest date.
project1={$project:{_id:0,
y:{$subtract:[{$year:"$TimeStamp"}, 2013]},
d:{$subtract:[{$dayOfYear:"$TimeStamp"},1]},
TimeStamp:1,
jan1:{$literal:new ISODate("2013-01-01T00:00:00")}
} };
project2={$project:{tsDate:{$add:[
"$jan1",
{$multiply:["$y", 365*24*60*60*1000]},
{$multiply:["$d", 24*60*60*1000]}
] } } };
Sample data:
db.foos.find({},{_id:0,TimeStamp:1})
{ "TimeStamp" : ISODate("2013-11-13T19:15:05.600Z") }
{ "TimeStamp" : ISODate("2014-02-01T10:00:00Z") }
Aggregation result:
> db.foos.aggregate(project1, project2)
{ "tsDate" : ISODate("2013-11-13T00:00:00Z") }
{ "tsDate" : ISODate("2014-02-01T00:00:00Z") }
This is what I use in one of my projects :
collection.aggregate(
// group results by date
{$group : {
_id : { date : "$date" }
// do whatever you want here, like $push, $sum...
}},
// _id is the date
{$sort : { _id : -1}},
{$orderby: { _id : -1 }})
.toArray()
Where $date is a Date object in mongo. I get results indexed by date.