Im trying to get the count of certain items grouped on certain dates.
This is working using the following aggregate query:
// this query works, without matching dates
[
{'$match': {
'some_id': ObjectId('foobar'),
'some_boolean_value': true
}
},
{'$project':
{'day':
{'$substr': ['$some_date', 0, 10]}}
},
{'$group': {_id: '$day', count: { '$sum': 1 }}},
{'$sort': {_id: -1}}
]
The next step is that I want to use this query but with date limits.
I want the count, grouped per day, between certain date limits.
// the query below does not work as soon as date matching is added
// this query always return 0 documents
[
{'$match': {
'some_id': ObjectId('foobar'),
'some_boolean_value': true,
'some_date':
{
'$gte': '2015-08-01T00:00:00.000Z',
'$lte': '2015-08-31T23:59:59.999Z'
}
}
},
{'$project':
{'day':
{'$substr': ['$some_date', 0, 10]}}
},
{'$group': {_id: '$day', count: { '$sum': 1 }}},
{'$sort': {_id: -1}}
]
You want to filter documents and match only those in a specified datetime window. But you use string comparison instead of date comparison.
Therefore replace this:
'$gte': '2015-08-01T00:00:00.000Z',
'$lte': '2015-08-31T23:59:59.999Z'
with this:
'$gte': new Date('2015-08-01T00:00:00.000Z'),
'$lte': new Date('2015-08-31T23:59:59.999Z')
Related
I'd like to sort a collection, then add a virtual property to the result which is their numerical order in which the results where displayed as.
So for example, we have a collection called calls, and we'd like to ascertain the current call queue priority as a number so it can be synced to our CRM via reverse ETL.
We have to do this inside of the query itself because we don't have an intermediary step where we can introduce any logic to determine this logic.
So my current query is
db.getCollection('callqueues').aggregate([
{
$match: {
'invalidated': false,
'assigned_agent': null
}
},
{ $sort: {
score: -1, _id: -1
} },
{
$addFields: {
order: "<NEW ORDER PROPERTY HERE>",
}
},
])
So I was wondering how would I insert as a virtual property their order, where the first element after the sort should be 1, second 2, etc
One option (since mongoDB version 5.0) is to use $setWindowFields for this:
db.collection.aggregate([
{$match: {invalidated: false, assigned_agent: null}},
{$setWindowFields: {
sortBy: {score: -1, _id: -1},
output: {
order: {
$sum: 1,
window: {documents: ["unbounded", "current"]}
}
}
}}
])
See how it works on the playground example
EDIT: If your mongoDB version is earlier than 5.0, you can use a less efficient query, involving $group and $unwind:
db.collection.aggregate([
{$match: {invalidated: false, assigned_agent: null}},
{$sort: {score: -1, _id: -1}},
{$group: {_id: 0, data: {$push: "$$ROOT"}}},
{$unwind: {path: "$data", includeArrayIndex: "order"}},
{$replaceRoot: {newRoot: {$mergeObjects: ["$data", {order: {$add: ["$order", 1]}}]}}}
])
See how it works on the playground example < 5.0
db.system.js.insertOne(
{
_id : "getPreviousDayDates" ,
value : function (){ return db.Smx_20213_ShiftOEEDaily.aggregate(
[
{
$group:
{
_id: "$machineName",
maxlogdate: { $max: "$logDate" }
}
}
]
); }
}
);
db.eval("getPreviousDayDates()")
//Getting Current Date from this collection, but I wants to get previous day, So firstly I am getting current date from maxlogdate and now I wants to take previous date so, that's why I wrote this query but facing the issue as: unknown group operator '$gte'
If you want to insert a new document, that contains a field that is retrieved from existing documents, you can use an aggregation pipeline with a $merge step:
db.collection.aggregate([
{$match: {machineName: "a"}},
{$group: {_id: 0, timestamp: {$max: "$timestamp"}}},
{$addFields: {data: newObjectToInsert}},
{$set: {"data.previousDate": "$timestamp"}},
{$replaceRoot: {newRoot: "$data"}},
{$merge: {into: "collection"}}
])
As you can see on this playground example.
First we $match all the documents with the same machineName. Then we $sort them according to the timestamp, and $limit to one, in order to have just the last value. Now we need to add our newObjectToInsert, $set the new field and $merge to insert to the collection.
If you want to subtract one day from the date, you can use:
{
$set: {"data.previousDate": {
$dateAdd: {
startDate: "$timestamp",
unit: "day",
amount: -1
}
}
}
},
instead of:
{$set: {"data.previousDate": "$timestamp"}},
This question already has answers here:
How to filter array in subdocument with MongoDB [duplicate]
(3 answers)
Closed 6 years ago.
In Mongo DB Getting Whole collection of date even if one is matched inside it.
Creating a new Collection with the below data:
db.details.insert({
"_id": 1,
"name": "johnson",
"dates": [
{"date": ISODate("2016-05-01")},
{"date": ISODate("2016-08-01")}
]
})
Fetching Back:
db.details.find().pretty()
Output:
{
"_id": 1,
"name": "Johnson",
"dates": [
{"date": ISODate("2016-05-01T00:00:00Z")},
{"date": ISODate("2016-08-01T00:00:00Z")}
]
}
So here there is a collection called dates inside another collection details.
Now I want to filter the date inside dates using Greater than and want the result showing "2016-08-01".
But when I search like the following:
db.details.find(
{"dates.date": {$gt: ISODate("2016-07-01")}},
{"dates.date": 1, "_id": 0}
).pretty()
Getting the Result as below, Its giving me the entire collection even if one date is matched in it:
{
"dates": [
{"date": ISODate("2016-05-01T00:00:00Z")},
{"date": ISODate("2016-08-01T00:00:00Z")}
]
}
Please help in getting the Expected data, i.e.:
{
"date": ISODate("2016-08-01T00:00:00Z")
}
You can use aggregate framework for this:
db.details.aggregate([
{$unwind: '$dates'},
{$match: {'dates.date': {$gt: ISODate("2016-07-01")}}},
{$project: {_id: 0, 'dates.date': 1}}
]);
Another way (Works only for mongo 3.2):
db.details.aggregate([
{$project: {
_id: 0,
dates: {
$filter: {
input: '$dates',
as: 'item',
cond: {
$gte: ['$$item.date', ISODate('2016-08-01T00:00:00Z')]
}
}
}
}
}]);
To only return the date field:
db.details.aggregate([
{$unwind: '$dates'},
{$match: {'dates.date': {$gt: ISODate('2016-07-01')}}},
{$group: {_id: '$dates.date'}},
{$project: {_id: 0, date: '$_id'}}
]);
Returns:
{
"date" : ISODate("2016-08-01T00:00:00Z")
}
I have the following kind of docs in a collection in mongo db
{ _id:xx,
iddoc:yy,
type1:"sometype1",
type2:"sometype2",
date:
{
year:2015,
month:4,
day:29,
type:"day"
},
count:23
}
I would like to do a sum over the field count grouping by iddoc for all docs where:
type1 in ["type1A","type1B",...]
where type2 in ["type2A","type2B",...]
date.year: 2015,
date.month: 4,
date.type: "day"
date.day between 4 and 7
I would like then to sort these sums.
I think this is probably easy to do within mongo db aggregation framework but I am new to it and would appreciate a tip to get started.
This is straightforward to do with an aggregation pipeline:
db.test.aggregate([
// Filter the docs based on your criteria
{$match: {
type1: {$in: ['type1A', 'type1B']},
type2: {$in: ['type2A', 'type2B']},
'date.year': 2015,
'date.month': 4,
'date.type': 'day',
'date.day': {$gte: 4, $lte: 7}
}},
// Group by iddoc and count them
{$group: {
_id: '$iddoc',
sum: {$sum: 1}
}},
// Sort by sum, descending
{$sort: {sum: -1}}
])
If I understood you correctly:
db.col.aggregate
(
[{
$match:
{
type1: {$in: ["type1A", type1B",...]},
type2: {$in: ["type2A", type2B",...]},
"date.year": 2015,
"date.month": 4,,
"date.day": {$gte: 4, $lte: 7},
"date.type": "day"
}
},
{
$group:
{
_id: "$iddoc",
total_count: {$sum: "$count"}
}
},
{ $sort: {total_count: 1}}]
)
This is filtering the field date.day between 4 and 7 inclusive (if not, use $gt and $lt to exclude them). And it sorts results from lower to higher (ascending), if you want to do a descending sort, then:
{ $sort: {total_count: -1}}
In an aggregation process I've got this data:
{
"_id" : "billing/DefaultController/actionIndex",
"min_time" : 0.033,
"max_time" : 5.25,
"exec_time" : 555.490999999997,
"qt" : 9059,
"count" : 2,
"date" : [
ISODate("2014-02-10T00:00:00.000Z"),
ISODate("2014-02-11T00:00:00.000Z")
]
},
How to change my query:
db.page_speed_reduced.aggregate([
{$group: {
_id: "$value.route",
min_time: {$min: "$value.min_time"},
max_time: {$max: "$value.max_time"},
exec_time: {$sum: "$value.exec_time"},
qt: {$sum: "$value.qt"},
count: {$sum: NumberInt(1)},
date: {$push: "$_id.date"},
}}
]);
for getting "$date" as concatenated string:
2014-02-10, 2014-02-11
UPDATE:
I tried this variant, but mongodb generated the error:
db.page_speed_reduced.aggregate([
{$group: {
_id: "$value.route",
min_time: {$min: "$value.min_time"},
max_time: {$max: "$value.max_time"},
exec_time: {$sum: "$value.exec_time"},
qt: {$sum: "$value.qt"},
count: {$sum: NumberInt(1)},
date: {$push: "test sting"},
}},
{$project: {
'date': {$concat: ['$date']}
//'date': {$concat: '$date'} //some error
}}
]);
uncaught exception: aggregate failed: {
"errmsg" : "exception: $concat only supports strings, not Array",
"code" : 16702,
"ok" : 0
}
'date': {$concat: '$date'}
As per comments so far it is unclear what you are grouping or what you want as the end result, other than to say that you want to get your dates concatenated into something like "just the day" with no hours or minutes together. Presumably you want those distinct days for some purpose.
There are various Date Operators in the pipeline you can use on dates, and the is the $concat operator as well. Unfortunately all of the Date Operators produce an integer as their result, and for the sort of Date string you want, $concat will only work with strings. The other problem being that you cannot cast the integer into a string type within aggregation.
But you can use sub-documents, here we'll just work with the date:
db.record.aggregate([
// Unwind the array to work with it
{$unwind: "$date"},
// project into our new 'day' document
{$project:{
day: {
year: {$year: "$date"},
month: {$month: "$date"},
day: {$dayOfMonth: "$date"}
}
} },
// optionalally sort if date order is important [ oldest -> newest ]
{$sort: { "day.year": -1, "day.month": -1, "day.day": -1}},
// Wind back unique values into the array
{$group: {_id:"$_id", days: {$addToSet: "$day"} }}
])
So, it's not a string, but it can easily be post-processed into one, but most importantly it's grouped and sortable.
The principles remain the same if you want the unique dates this way as an array at the end or whether you want to group totals by those dates. So primarily keep in mind the $unwind and $project parts using the date operators.
--EDIT--
With thanks to the community as shown in this post there is this undocumented behavior of $substr, in which integers can be cast as strings.
db.record.aggregate([
// Unwind the array to work with it
{$unwind: "$date"},
// project into our new 'day' document
{$project:{
day: {
year: {$year: "$date"},
month: {$month: "$date"},
day: {$dayOfMonth: "$date"}
}
} },
// optionalally sort if date order is important [ oldest -> newest ]
{$sort: { "day.year": -1, "day.month": -1, "day.day": -1}},
// now we are going to project to a string ** magic #heinob **
{$project: {
day: {$concat: [
{$substr: [ "$day.year", 0, 4 ]},
"-",
{$substr: [ "$day.month", 0, 2 ]},
"-",
{$substr: [ "$day.day", 0, 2 ]}
]}
}},
// Wind back unique values into the array
{$group: {_id:"$_id", days: {$addToSet: "$day"} }}
])
And now the days are strings. As I noted before, if the ordering is important to you then the best approach is to project into a document type as has been done and sort on the numeric keys. Naturally the $project that transforms the date can be wound into the $group stage for brevity, which is probably what you want to do when working with the whole document.
This link might give you a hint:
http://docs.mongodb.org/manual/reference/operator/aggregation/concat/
year: {$concat: [ $year ]}