I have a collection with two fields, similar to the one bellow:
{
year: 2017,
month: 04 }
How can i select documents between 2017/07 - 2018/04?
Solved with:
db.collection.aggregate(
// Pipeline
[
// Stage 1
{
$addFields: {
"date": {
"$dateFromParts": {
"year": "$year",
"month": {"$toInt": "$month"}
}
}
}
},
// Stage 2
{
$match: {
"date": {
"$gte": ISODate("2017-07-01T00:00:00.000Z"),
"$lte": ISODate("2018-04-30T00:00:00.000Z")
}
}
},
]
);
Firstly you have to make sure you db is storing date in ISO format ( a format that mongo supports )
You can use following command to find documents :-
model.find({
date:{
$gte:ISODate("2017-04-29T00:00:00.000Z"),
$lte:ISODate("2017-07-29T00:00:00.000Z"),
}
})
Where model is the name of the collection and date is a attribute of
document holding dates in ISO format.
Related
So I try to use MongoDB $match to get data between 2 dates, but it turns out that the data is not returning a.k.a empty here. What it looks like:
db.collection.aggregate([
{
$match: {
date: {
$gte: new Date("2022-10-23"),
$lt: new Date("2022-10-25"),
},
}
},
{
$group: {
_id: "$title",
title: {
$first: "$title"
},
answer: {
$push: {
username: "$username",
date: "$date",
formId: "$formId",
answer: "$answer"
}
}
}
},
])
Here is the data that I try to run on the Mongo playground:
https://mongoplayground.net/p/jKx_5kZnJNz
I think there is no error with my code anymore... but why it gives an empty return.
Migrate the comment to the answer post for the complete explanation.
Issue 1
The document contains the date field as a string type while you are trying to compare with Date which leads to incorrect output.
Ensure that you are comparing both values in the exact type.
Either that migrate the date value to Date type or
converting the date field to Date type in the query via $toDate.
{
$match: {
$expr: {
$and: [
{
$gte: [
{
$toDate: "$date"
},
new Date("2022-10-23")
]
},
{
$lt: [
{
$toDate: "$date"
},
new Date("2022-10-25")
]
}
]
}
}
}
Issue 2
Since you are using $lt ($lt: new Date("2022-10-25")), it won't include the documents with date: new Date("2022-10-25").
For inclusive end date, you shall use $lte.
Demo # Mongo Playground
I have a Mongo database and I have stored dates as strings. Per document I have a field called "creationdate" and a field called "completiondate". The dates format is "YYYY-MM-dd" (ex "2011-12-18"). Even I can execute simple aggregation like greaterThan, greaterThanEqual, I cannot find the difference in dates, which I have to find to calculate the average days difference between completion and creation date.
The above query I have to write it on spring-boot with MongoTemplate if it is possible.
I am trying something like this but it doesn't work.
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(Criteria.where("creationdate").gte(date1).lte(date2).andOperator(Criteria.where("completiondate").ne(""))),
Aggregation.project("servicerequesttype").and(DateOperators.DateFromString.fromStringOf("completiondate").withFormat("%Y-%m-%d")).minus(DateOperators.DateFromString.fromStringOf("creationdate").withFormat("%Y-%m-%d")).as("diff"),
Aggregation.group("servicerequesttype").avg("diff").as("average")
);
date1, date2 are given strings like "2011-01-01"
Is this what you are looking for?
db.collection.aggregate([
{
$project: {
creationdate: {
$dateFromString: {
dateString: "$creationdate",
format: "%Y-%m-%d"
}
},
completiondate: {
$dateFromString: {
dateString: "$completiondate",
format: "%Y-%m-%d"
}
}
}
},
{
$project: {
difference: {
$subtract: [
"$completiondate",
"$creationdate"
]
}
}
},
{
$group: {
_id: null,
average: {
$avg: "$difference"
}
}
},
{
$project: {
_id: 0,
dayAverage: {
$divide: [
"$average",
86400000
]
}
}
}
])
I have created interactive demo here: https://mongoplayground.net/p/wGRw12m3UbB
Hope it helps :)
Spring-Boot
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(Criteria.where("creationdate").gte(date1).lte(date2).andOperator(Criteria.where("completiondate").ne(""))),
Aggregation.project("servicerequesttype").and(DateOperators.DateFromString.fromStringOf("creationdate").withFormat("%Y-%m-%d")).as("creationdate").and(DateOperators.DateFromString.fromStringOf("completiondate").withFormat("%Y-%m-%d")).as("completiondate"),
Aggregation.project("servicerequesttype").and("completiondate").minus("creationdate").as("difference"),
Aggregation.group("servicerequesttype").first("servicerequesttype").as("servicerequesttype").avg("difference").as("temp"),
Aggregation.project("servicerequesttype").and("temp").divide(86400000).as("average")
);
I have a string field in mongodb which should be converted to a date field.
The format of the string is like the following:
2014 - Only year, default month and day are 01 and 01, so it should be converted to date '2014-01-01'
2014-01 - With year and month, which should also be converted to date '2014-01-01'
2014-01-01 - Full date
$dateFromString in the following syntax doesn't seem to work:
$dateFromString: {
dateString: '$order.date',
format: '%Y-%m-%d',
}
How can I make $dateFromString to support multiple format?
What you could do is to add a new field via $addFields and then for its value create few if conditions using the $cond pipeline operator matching each of your date lengths (via $strLenCP) and concatenating the remaining parts (via $concat). Then since all of your date fields will now match the format %Y-%m-%d it should work ... like this:
db.getCollection('<YourCol>').aggregate([{
$addFields: {
dateFixed: {
$cond: {
if: { $eq: [{ $strLenCP: "$date"}, 4] }, // <-- "2011"
then: { $concat: ["$date", "-01-01"] },
else: {
$cond: {
if: { $eq: [{ $strLenCP: "$date" }, 7] }, // <-- "2011-01"
then: { $concat: ["$date", "-01"] },
else: "$date" // <-- "2011-01-01"
}
}
}
}
}
},
{
$project: {
date: {
$dateFromString: {
dateString: 'dateFixed',
format: '%Y-%m-%d'
}
}
}
}
])
You can see it working here
From a MongoDB collection storing data with time stamps I need to return a single record for each hour.
So far I have selected the set of records between two dates successfully, but I cant figure how to build the hourly record I need in the $group clause.
var myName = "CollectionName"
//schema for mongoose
var mySchema = new Schema({
dt: Date,
value: Number
});
var myDB = mongoose.createConnection('mongodb://localhost:27017/MYDB');
myDBObj = myDB.model(myName, evalSchema, myName);
The match in this aggregate call works fine, and the $hour creates a record for each hour in the day.. but I don't know how to recreate the a full date and get an error "unknown group operator $avg" ...
myDBObj.aggregate([
{
$match: { "dt": { $gt: new Date("October 13, 2010 12:00:00"), $lt: new Date("November 13, 2010 12:00:00") } }
},{
$group: {
"_id": { "dt": { "$hour": "$dt" } , "price": { "$avg": "$price" }}
}], function (err, data) { if (err) { return next(err); } res.json(data); });
I think I need to use $dayOfYear so there is different records for each hour of each day, and include a new Date() somewhere ...
Can someone help me do this correctly? any help is appreciated.
The $group pipeline stage works by "grouping" all data by the "key" specified for _id. Other fields you are actually aggregating are separate from the _id value and are their own field properties.
So your $group becomes this instead:
{ "$group": {
"_id": { "$hour": "$dt" },
"price": { "$avg": "$price" }
}}
Or if you want that broken by day then make a compound key:
{ "$group": {
"_id": {
"day": { "$dayOfYear": "$dt" },
"hour": { "$hour": "$dt" }
},
"price": { "$avg": "$price" }
}}
Or just use date math to produce Date objects rounded by hour:
{ "$group": {
"_id": {
"$add": [
{ "$subtract": [
{ "$subtract": [ "$dt", new Date(0) ] },
{ "$mod": [
{ "$subtract": [ "$dt", new Date(0) ] },
1000 * 60 *60
]}
]},
new Date(0)
]
},
"price": { "$avg": "$price" }
}}
Where subrtacting another date object (epoch date) from another prodces a numeric value you can round ( 1000 milliseconds, 60 seconds, 60 minutes = 1 hour ) with the applied math, and adding a number to a date object produces a date corresponding to that value.
So your problem was you had everything in the _id, where the $avg accumulator is not recognised. All accumulators need to be specified outside of the grouping key. That is the intent.
If you want to make an accumulator value part of a grouping key ( does not seem relevant here though ), you instead follow with another group stage, referencing the field that was produced from the former.
I store data about my power consumption, each minute there is a new record, here is an example:
{"date":1393156826114,"id":"5309d4cae4b0fbd904cc00e1","adco":"O","hchc":7267599,"hchp":10805900,"hhphc":"g","ptec":"c","iinst":13,"papp":3010,"imax":58,"optarif":"s","isousc":60,"motdetat":"Á"}
such that I have around 1440 records a day.
How can I get the last record of each day?
Note: I use mongodb in spring java, so I need a query like this:
Example to get all measures :
#Query("{ 'date' : { $gt : ?0 }}")
public List<Mesure> findByDateGreaterThan(Date date, Sort sort);
A bit more modern than the original answer:
db.collection.aggregate([
{ "$sort": { "date": 1 } },
{ "$group": {
"_id": {
"$subtract": ["$date",{"$mod": ["$date",86400000]}]
},
"doc": { "$last": "$$ROOT" }
}},
{ "$replaceRoot": { "newDocument": "$doc" } }
])
The same principle applies that you essentially $sort the collection and then $group on the required grouping key picking up the $last data from the grouping boundary.
Making things a bit clearer since the original writing is that you can use $$ROOT instead of specifying every document property, and of course the $replaceRoot stage allows you to restore that data fully as the original document form.
But the general solution is still $sort first, then $group on the common key that is required and keep the $last or $first depending on sort order occurrences from the grouping boundary for the properties that are required.
Also for BSON Dates as opposed to a timestamp value as in the question, see Group result by 15 minutes time interval in MongoDb for different approaches on how to accumulate for different time intervals actually using and returning BSON Date values.
Not quite sure what you are going for here but you could do this in aggregate if my understanding is right. So to get the last record for each day:
db.collection.aggregate([
// Sort in date order as ascending
{"$sort": { "date": 1 } },
// Date math converts to whole day
{"$project": {
"adco": 1,
"hchc": 1,
"hchp": 1,
"hhphc": 1,
"ptec": 1,
"iinst": 1,
"papp": 1,
"imax": 1,
"optarif": 1,
"isousc": 1,
"motdetat": 1,
"date": 1,
"wholeDay": {"$subtract": ["$date",{"$mod": ["$date",86400000]}]}
}},
// Group on wholeDay ( _id insertion is monotonic )
{"$group":
"_id": "$wholeDay",
"docId": {"$last": "$_id" },
"adco": {"$last": "$adco" },
"hchc": {"$last": "$hchc" },
"hchp": {"$last": "$hchp" },
"hhphc": {"$last": "$hhphc" },
"ptec": {"$last": "$ptec" },
"iinst": {"$last": "$iinst" },
"papp": {"$last": "$papp" },
"imax": {"$last": "$imax" },
"optarif": {"$last": "$optarif",
"isousc": {"$last": "$isouc" },
"motdetat": {"$last": "$motdetat" },
"date": {"$last": "$date" },
}}
])
So the principle here is that given the timestamp value, do the date math to project that as the midnight time at the beginning of each day. Then as the _id key on the document is already monotonic (always increasing), then simply group on the wholeDay value while pulling the $last document from the grouping boundary.
If you don't need all the fields then only project and group on the ones you want.
And yes you can do this in the spring data framework. I'm sure there is a wrapped command in there. But otherwise, the incantation to get to the native command goes something like this:
mongoOps.getCollection("yourCollection").aggregate( ... )
For the record, if you actually had BSON date types rather than a timestamp as a number, then you can skip the date math:
db.collection.aggregate([
{ "$group": {
"_id": {
"year": { "$year": "$date" },
"month": { "$month": "$date" },
"day": { "$dayOfMonth": "$date" }
},
"hchp": { "$last": "$hchp" }
}}
])
It's also possible to format timestamps in the group key as %Y-%m-%d (e.g. 2021-12-05) with dateToString:
// { timestamp: 1638697946000, value: "a" } <= 2021-12-05 9:52:26
// { timestamp: 1638686311000, value: "b" } <= 2021-12-05 6:38:31
// { timestamp: 1638859111000, value: "c" } <= 2021-12-07 6:38:31
db.collection.aggregate([
{ $sort: { timestamp: 1 } },
// { timestamp: 1638686311000, value: "b" }
// { timestamp: 1638697946000, value: "a" }
// { timestamp: 1638859111000, value: "c" }
{ $group: {
_id: { $dateToString: { date: { $toDate: "$timestamp" }, format: "%Y-%m-%d" } },
last: { $last: "$$ROOT" }
}},
// { _id: "2021-12-07", last: { timestamp: 1638859111000, value: "c" } }
// { _id: "2021-12-05", last: { timestamp: 1638697946000, value: "a" } }
{ $replaceWith: "$last" }
])
// { timestamp: 1638697946000, value: "a" } <= 2021-12-05 9:52:26
// { timestamp: 1638859111000, value: "c" } <= 2021-12-07 6:38:31
This:
first $sorts documents by chronological order of timestamps such that we can latter on pick newest documents based on their order.
then $groups documents by their %Y-%m-%d-formatted timestamps:
by first converting the timestamp into a datetime: { $toDate: "$timestamp" }
and then converting the associated datetime into a string only representing the year, month and day: { $dateToString: { date: ..., format: "%Y-%m-%d" } }
such that for each group (i.e. date), we can pick the $last (i.e. newest since chronologically sorted) matching document
and the pick is the whole document as represented by $$ROOT
finally cleans up the group result with a $replaceWith stage (alias for $replaceRoot).