I'm working on a query which is grouping records per day and counting them on MongoDB
here is my query
db.getCollection('CustomerApplications').aggregate(
[
{
$group:
{
_id: { day: { $dayOfYear: { $toDate: "$data.submittedAt" }}, year: { $year: { $toDate: "$data.submittedAt" } } },
count: { $sum: 1 }
}
}
]
)
$data.submittedAt is a double so I need to convert it to date then pull $dayOfYear from it
but I get
Unrecognized expression '$toDate'
my data structure is like
{
"_id" : ObjectId("5c942f50dae240feb1942b00"),
"data" : {
"id" : "624c0d17-b683-4c89-9d7c-011577d4e3b8",
"email" : "i8888#eee.com",
"name" : "ianh",
"phoneNumber" : "+1222222",
"score" : 12,
"status" : "PENDING",
"submittedAt" : 1553215312006.0,
"surveyVersion" : "1"
},
"updatedAt" : ISODate("2019-03-21T00:41:52.192Z")
}
any Idea is this doable in MongoDB if yes how to correctly do it?
$toDate New in version 4.0. Please check your version
Can you try with this.
db.getCollection('CustomerApplications').aggregate(
[
{
$group:
{
_id : { $substr: ["$data.submittedAt", 0, 10] },
count: { $sum: 1 }
}
}
])
May this will help you
$toDate - Converts a value to a date (New in version 4.0)
$dayOfMonth - Returns the day of the month for a date as a number between 1 and 31
$dayOfYear - Returns the day of the year for a date as a number between 1 and 366
Related
I have a MongoDB collection named Bookings
{
"_id" : ObjectId("5fca982d219fee6f00e631a0"),
"price" : 45.9,
"createdAt" : ISODate("2020-12-04T20:12:29.117Z")
}
{
"_id" : ObjectId("5fca990b219fee6f00e631a1"),
"price" : 45.9,
"createdAt" : ISODate("2020-12-04T20:16:11.925Z")
}
{
"_id" : ObjectId("5fcab925a912a2064fe7b916"),
"price" : 45.9,
"createdAt" : ISODate("2020-12-04T22:33:09.958Z")
}
{
"_id" : ObjectId("5fcab938a912a2064fe7b917"),
"price" : 45.9,
"createdAt" : ISODate("2020-12-04T22:33:28.641Z")
}
{
"_id" : ObjectId("5fcab94aa912a2064fe7b918"),
"createdAt" : ISODate("2020-12-04T22:33:46.118Z")
}
{
"_id" : ObjectId("5fcb73e0e396cf18e6141dc6"),
"price" : 45.9,
"createdAt" : ISODate("2020-12-05T11:49:52.544Z")
}
{
"_id" : ObjectId("5fcb73eee396cf18e6141dc7"),
"price" : 45.9,
"createdAt" : ISODate("2020-12-05T11:50:06.914Z")
}
{
"_id" : ObjectId("5fcbee785ef206248fa9513e"),
"price" : 35.7,
"createdAt" : ISODate("2020-12-05T20:32:56.508Z")
}
{
"_id" : ObjectId("5fcbf0045ef206248fa9513f"),
"price" : 2047.66,
"createdAt" : ISODate("2020-12-05T20:39:32.369Z")
}
I need to display the data according to a week and collective price for that week. If I use aggregation pipeline, It would only give me the range of dates on which booking is made.
{$group: {
_id: {
$week: "$createdAt"
},
start_date: {$min: "$createdAt"},
end_date: {$max: "$createdAt"}
}}
Suppose the previous week started from 14-06-21 and ended on 20-06-21
Instead, I want a result which would actually include start_date as 14-06-21 and end_date as 20-06-21, and if no earning is made it would be 0 otherwise the total price in a given week and so on for other group of weeks for whole year or month accordingly.
You can categorize total price by week number using $week operator, but it is hard to get the week's start date and end date in MongoDB, I would suggest you to get start date and end date from the week number in your client-side language.
$group by createdAt's week using $week operator and get total price by $sum
db.collection.aggregate([
{
$group: {
_id: { $week: "$createdAt" },
totalPrice: { $sum: "$price" }
}
}
])
Playground
For fetching the start day of a week, you can make use of the $dateToString operator and pass the %g and %v in format which represents ISO Year and Week of the year respectively.
Similarly, add 518400000 (6 days in milliseconds) to the start date of the week to get the end date.
Also, these two operations will only work inside the _id field of the $group stage, so parse the sub-object of the _id key to get the required data values. The weekNo key is useless for the $group stage, but keep it if it's required.
db.collection.aggregate([
{
$group: {
_id: {
"weekNo": {
$week: "$createdAt"
},
"start_date": {
"$dateFromString": {
"dateString": {
"$dateToString": {
"date": "$createdAt",
"format": "%G/%V",
},
},
"format": "%G/%V"
}
},
"end_date": {
"$add": [
{
"$dateFromString": {
"dateString": {
"$dateToString": {
"date": "$createdAt",
"format": "%G/%V",
},
},
"format": "%G/%V"
}
},
518400000,
],
},
},
totalPrice: {
$sum: "$price"
}
}
}
])
Mongo Playground Sample Execution
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : "2020-05-14T14:12:46.058-0400",
"ServerTimeStamp" : "2020-05-14T14:12:46.058-0400"
}
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : "2020-05-14T15:12:46.058-0400",
"ServerTimeStamp" : "2020-05-14T14:12:46.058-0400"
}
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : "2020-05-14T18:12:46.058-0400",
"ServerTimeStamp" :"2020-05-14T14:12:46.058-0400"
}
I would like to calculate the time difference of above columns : time and ServerTimeStamp ( in seconds) as below . here the time field is in string format instead of datetime
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : "2020-05-14T14:12:46.058-0400",
v"ServerTimeStamp" : "2020-05-14T14:12:46.058-0400",
"time_difference" :0
}
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : "2020-05-14T15:12:46.058-0400",
"ServerTimeStamp" : "2020-05-14T14:12:46.058-0400"
"time_difference" : 3600
}
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : "2020-05-14T18:12:46.058-0400",
"ServerTimeStamp" : "2020-05-14T14:12:46.058-0400"
"time_difference" : 10800
}
I tried the below query but getting error and i have already converted the time to date from string
db.hello.aggregate([
{
$project:{duration : {
$divide: [{$subtract:
[ $StringToDate:("$time") ,$StringToDate:("$ServerTimeStamp") ]
},3600000]
}
}}
])
You will have to use the built in $toDate operator, available from MongoDB v4.0
db.hello.aggregate([
{
$addFields: { // use $addFields to add new fields while keeping previous fields
duration : {
$divide: [
{
$subtract: [
{ $toDate: "$time" },
{ $toDate: "$ServerTimeStamp" }
]
},
1000 // the result was in millisecond, if you want in seconds you have to divide by 1000, if you want in hours divide by 3600000
]
}
}
}
])
For earlier MongoDB versions, there is $dateFromString operator, available from MongoDB v3.6
db.hello.aggregate([
{
$addFields: { // use $addFields to add new fields while keeping previous fields
duration : {
$divide: [
{
$subtract: [
{ $dateFromString: { dateString: "$time" } },
{ $dateFromString: { dateString: "$ServerTimeStamp" } }
]
},
1000 // the result was in millisecond, if you want in seconds you have to divide by 1000, if you want in hours divide by 3600000
]
}
}
}
])
I am trying to build a dashboard chart in Mongo-Atlas.
The Table should should show the date on x-axis, the _id on y-axis.
The Values should be the count difference to the date before.
I have a collection with data points such as:
_id: "someName"
timestamp: 2019-09-05T06:24:24.689+00:00
count: 50
_id: "someName"
timestamp: 2019-09-04T06:24:24.689+00:00
count: 40
...
The goal is to get the difference of the count to the data point before. Having the same name.
_id: "someName"
timestamp: 2019-09-05T06:24:24.689+00:00
count: 50
difference: 10
_id: "someName"
timestamp: 2019-09-04T06:24:24.689+00:00
count: 40
difference: 17
...
That way I could make a table listing the differences
so far I created a aggregation pipeline
[
{$sort: {
"timestamp": -1
}},
{$group: {
_id: "$_id",
count: {
$push: { count: "$count", timestamp: "$timestamp" }
}
}},
{$project: {
_id: "$_id",
count: "$count",
countBefore: { $slice: [ "$count", 1, { $size: "$count" } ] }
}}
]
I was hoping to substract count and countBefore such that i get an array with the datapoints an the difference...
So I tried to follow with:
{$project: {
countDifference: {
$map: {
input: "$countBefore",
as: "before",
in: {
$subtract: ["$$before.count", "$count.count"]
/*"$count.count" seems to be the problem, since an integer works*/
}
}
}
}
}
Mongo Atlas only shows "An unknown error occurred"
I would be glad for some advice :)
The following query can get us the expected output:
db.collection.aggregate([
{
$sort:{
"timestamp":1
}
},
{
$group:{
"_id":"$id",
"counts":{
$push:"$count"
}
}
},
{
$project:{
"differences":{
$reduce:{
"input":"$counts",
"initialValue":{
"values":[],
"lastValue":0
},
"in":{
"values":{
$concatArrays:[
"$$value.values",
[
{
$subtract:["$$this","$$value.lastValue"]
}
]
]
},
"lastValue":"$$this"
}
}
}
}
},
{
$project:{
"_id":0,
"id":"$_id",
"plots":"$differences.values"
}
}
]).pretty()
Data Set:
{
"_id" : ObjectId("5d724550ef5e6630fde5b71e"),
"id" : "someName",
"timestamp" : "2019-09-05T06:24:24.689+00:00",
"count" : 50
}
{
"_id" : ObjectId("5d724550ef5e6630fde5b71f"),
"id" : "someName",
"timestamp" : "2019-09-04T06:24:24.689+00:00",
"count" : 40
}
{
"_id" : ObjectId("5d724796ef5e6630fde5b720"),
"id" : "someName",
"timestamp" : "2019-09-06T06:24:24.689+00:00",
"count" : 61
}
{
"_id" : ObjectId("5d724796ef5e6630fde5b721"),
"id" : "someName",
"timestamp" : "2019-09-07T06:24:24.689+00:00",
"count" : 72
}
{
"_id" : ObjectId("5d724796ef5e6630fde5b722"),
"id" : "someName",
"timestamp" : "2019-09-08T06:24:24.689+00:00",
"count" : 93
}
{
"_id" : ObjectId("5d724796ef5e6630fde5b723"),
"id" : "someName",
"timestamp" : "2019-09-09T06:24:24.689+00:00",
"count" : 100
}
Output:
{ "id" : "someName", "plots" : [ 40, 10, 11, 11, 21, 7 ] }
Explanation: We are pushing count for the same id into counts array and then applying $reduce operation on it to prepare a set of new values in which current value would hold difference between the current and previous value of counts array. For the very first value, the previous value is taken as zero.
I have a user base stored in mongo. Users may record their date of birth.
I need to run a report aggregating users by age.
I now have a pipeline that groups users by year of birth. However, that is not precise enough because most people are not born on January 1st; so even if they are born in, say, 1970, they may well not be 43 yet.
db.Users.aggregate([
{ $match : { "DateOfBirth" : { $exists : true} } },
{ $project : {"YearOfBirth" : {$year : "$DateOfBirth"} } },
{ $group : { _id : "$YearOfBirth", Total : { $sum : 1} } },
{ $sort : { "Total" : -1 } }
])
Do you know if it's possible to perform some kind of arithmetic within the aggregation framework to exactly calculate the age of a user? Or is this possible with MapReduce only?
It seems like the whole thing is possible with the new Mongo 2.4 version just released, supporting additional Date operations (namely the "$subtract").
Here's how I did it:
db.Users.aggregate([
{ $match : { "DateOfBirth" : { $exists : true} } },
{ $project : {"ageInMillis" : {$subtract : [new Date(), "$DateOfBirth"] } } },
{ $project : {"age" : {$divide : ["$ageInMillis", 31558464000] }}},
// take the floor of the previous number:
{ $project : {"age" : {$subtract : ["$age", {$mod : ["$age",1]}]}}},
{ $group : { _id : "$age", Total : { $sum : 1} } },
{ $sort : { "Total" : -1 } }
])
There are not enough dateTime operators and math operators to project out the date. But you might be able to create age ranges by composing a dynamic query:
Define your date ranges as cut-off dates as
dt18 = today - 18
dt25 = today - 25
...
dt65 = today - 65
Then do nested conditionals, where you progressively use the cut off dates as age group markers, like so:
db.folks.save({ "_id" : 1, "bd" : ISODate("2000-02-03T00:00:00Z") });
db.folks.save({ "_id" : 2, "bd" : ISODate("2010-06-07T00:00:00Z") });
db.folks.save({ "_id" : 3, "bd" : ISODate("1990-10-20T00:00:00Z") });
db.folks.save({ "_id" : 4, "bd" : ISODate("1964-09-23T00:00:00Z") });
db.folks.aggregate(
{
$project: {
ageGroup: {
$cond: [{
$gt: ["$bd",
ISODate("1995-03-19")]
},
"age0_18",
{
$cond: [{
$gt: ["$bd",
ISODate("1988-03-19")]
},
"age18_25",
"age25_plus"]
}]
}
}
},
{
$group: {
_id: "$ageGroup",
count: {
$sum: 1
}
}
})
I'm attempting to use use the new MongoDB aggregation features to tally some statistics by date. Below is a sample of the documents that I am working with, my attempted code and desired result. The aggregation function retuns "UNDEFINED". Can someone tell me why that is? And secondly, I want my aggregation function to group results by date in mm-dd-yyyy format. However as it is currently written I think the code is going to execute the aggregation by the full ISO date. Can someone please tell me how to fix this?
DOCUMENT EXAMPLE
{
user: "2A8761E4-C13A-470E-A759-91432D61B6AF-25982-0000352D853511AF",
language: "English",
imageFileName: "F7A5ED9-D43C-4671-A5C6-F06C7E41F902-7758-000008371FB5B834",
audioFileName: "F6D5727D-9377-4092-A28A-AA900F02653D-7758-0000083749066CF2",
date: ISODate("2012-10-22T02:43:52Z"),
correct: "1",
_id: ObjectId("5084b2e8179c41cc15000001")
}
AGGREGATION FUNCTION
var getUserStats = function(user, language, callback) {
var guessCollection = db.collection('Guesses');
guessCollection.aggregate(
{ $match: {
user: user,
language: language,
}},
{ $sort: {
date: 1
}},
{ $project : {
user : 1,
language : 1,
date : 1,
correct : 1,
incorrect : 1,
} },
{ $unwind : "$language" },
{ $group : {
_id : "$date",
correct : { $sum : "$correct" },
incorrect : { $sum : "$incorrect" }
} }
, function(err, result){
console.log(result);
callback(result);
});
DESIRED RESULT
{
"result" : [
//...snip...
{
"_id" : "2A8761E4-C13A-470E-A759-91432D61B6AF-25982-0000352D853511AF",
"correct" : 32,
"incorrect" : 17,
"date" : 2012-10-22
},
{
"_id" : "2A8761E4-C13A-470E-A759-91432D61B6AF-25982-0000352D853511AF",
"correct" : 16,
"incorrect" : 7,
"date" : 2012-10-23
}
],
"Ok" : 1
}
Regarding your first question about it returning undefined, there are two problems:
You are using the $unwind operator on a field ($language) that isn't an array.
You are using the $sum operator on a string field ($correct); that's only supported for number fields.
For your second question about grouping on just the date, you need to project the date components you want to group on and then use those components in your $group operator's _id value:
For example:
test.aggregate(
{ $match: {
user: user,
language: language
}},
{ $sort: {
date: 1
}},
{ $project : {
user : 1,
language : 1,
year : { $year: '$date' },
month : { $month: '$date' },
day : { $dayOfMonth: '$date'},
correct : 1,
incorrect : 1
}},
{ $group : {
_id : { year: "$year", month: "$month", day: "$day" },
correct : { $sum : "$correct" },
incorrect : { $sum : "$incorrect" }
}},
function(err, result){
console.log(result);
}
);
Produces output of:
[ { _id: { year: 2012, month: 10, day: 22 },
correct: 0,
incorrect: 0 } ]
You can assemble that into '2012-10-22' in code from there.