How can I change null values in aggregation using MongoDB - mongodb

I'm having a hard time to get this code working . I have an aggregation query in mongo db in which I do some operations inside . The query without the $ifNull operator is working fine , however I'm getting some null values that I do not want to be printing. I've read the documentation of $ifNull operators , but I am not able to get this code working. I have this code so far , witch is giving me this error :
SyntaxError: Unexpected token :
I think that's the ":" from the ifnull but I don't know why. I'm following the sintax that is on the documentation in the official website. I've also tried to use another project field in order to filter the Null values , but it didn't work.
This is de query so far :
db.cleandrivers.aggregate(
[
{
$project :
{ _id:"$_id",
week : {$week : "$recruiter.date"},
dif: {
$ifNull: [$divide : [{$subtract : ["$personal.stat.first_inspected","$recruiter.date"]} , 1000 * 60 * 60 ], "0"]
}
}
}, { $match : { week: 11 } }
])
Edit : Everything is working fine , this is a query that is actually working counting the recruited ppl giving an specific week.
db.cleandrivers.aggregate(
... [
... {
... $project :
... { _id:"$_id",
... week : {$week : "$personal.stat.recruited"}
... }
... },
... { $match : { $and: [ {week: 10}] } },
... {$group: { _id: "Sum",sum: {$sum : 1}}} ])
Output :
{ "_id" : "Sum", "sum" : 199 }
This is my document sintax :
"personal" : {
"stat" : {
"recruited" : ISODate("2015-02-02T14:30:32.089Z"),
"first_inspected" : ISODate("2015-02-02T14:33:15.956Z"),
"targo" : ISODate("2015-02-06T17:51:00.672Z")
}
The problem is that when I use the same query that i used when i was trying to count the recruited ppl , its giving me an error ,because there are some documents that do not have that field. I am really confused on how do I have to filter all the documents that have the field first_inspected in their documents , and count them giving a week .
The error happens on the $week line witch tries to convert a null date into a week and it gives me an error.

I reformatted and patched the syntax error:
db.cleandrivers.aggregate([
{ "$project" : {
"_id" : "$_id",
"week" : { "$week" : "$recruiter.date" },
"dif" : {
"$ifNull" : [{ "$divide" : [{ "$subtract" : ["$personal.stat.first_inspected", "$recruiter.date"] }, 1000 * 60 * 60 ] }, "0"]
}
} },
{ "$match" : { "week" : 11 } }
])
The problem was that you didn't have the $divide expression inside an object, so it was like
[ "$divide" : stuff, more, stuff]
instead of
[{ "$divide" : stuff }, more, stuff]
I can't help you with getting the aggregation to return what you want as I don't know exactly what you're looking for. If you need more help with that, could you edit the question to include a sample document and a description of what result you want to get from it?

Related

MongoDB $divide on aggregate output

Is there a possibility to calculate mathematical operation on already aggregated computed fields?
I have something like this:
([
{
"$unwind" : {
"path" : "$users"
}
},
{
"$match" : {
"users.r" : {
"$exists" : true
}
}
},
{
"$group" : {
"_id" : "$users.r",
"count" : {
"$sum" : 1
}
}
},
])
Which gives an output as:
{ "_id" : "A", "count" : 7 }
{ "_id" : "B", "count" : 49 }
Now I want to divide 7 by 49 or vice versa.
Is there a possibility to do that? I tried $project and $divide but had no luck.
Any help would be really appreciated.
Thank you,
From your question, it looks like you are assuming result count to be 2 only. In that case I can assume users.r can have only 2 values(apart from null).
The simplest thing I suggest is to do this arithmetic via javascript(if you're using it in mongo console) or in case of using it in progam, use the language you're using to access mongo) e.g.
var results = db.collection.aggregate([theAggregatePipelineQuery]).toArray();
print(results[0].count/results[1].count);
EDIT: I am sharing an alternative to above approach because OP commented about the constraint of not using javascript code and the need to be done only via query. Here it is
([
{ /**your existing aggregation stages that results in two rows as described in the question with a count field **/ },
{ $group: {"_id": 1, firstCount: {$first: "$count"}, lastCount: {$last: "$count"}
},
{ $project: { finalResult: { $divide: ['$firstCount','$lastCount']} } }
])
//The returned document has your answer under `finalResult` field

Querying date range in aggregate query returns nothing or ignores dates

In my aggregate query I'm trying to add conditions in the $match statement to return only records within given date range. Without converting to ISOString, I get a set of records that ignores the date range completely. When I convert to ISOString, I get nothing (returns empty set). I've tried using the $and operator, still nothing.
I've tried all the solutions on stack to no avail. Here's my code:
$match: {
$and: [
{'author.id': { $ne: req.user._id }},
{'blurtDate': { $gte: test1.toISOString() }},
{'blurtDate': { $lte: test2.toISOString() }}
]
}
test1 and test2 are correct, I checked them on console log they reflect as follows:
2019-06-02T12:44:39.000Z -- 2019-07-02T12:44:39.928Z
I also tried without the $and operator like so:
$match: {
'author.id': { $ne: req.user._id },
'blurtDate': { $gte: test1.toISOString() },
'blurtDate': { $lte: test2.toISOString() }
}
Which again returns nothing. Any help much appreciated!
EDIT: Wanted to emphasize that test1 and test2 are new date objects:
test1 = new Date(qryDateFrom); //Tried .toISOString() as well
test2 = new Date(qryDateTo);
Without .toISOString(), I get a return of values that ignores the dates. With .toISOString I get an empty return.
Here's an example document that should be returned:
{
"_id" : ObjectId("5d0a807345c85d00ac4b7217"),
"text" : "<p>Seriously RV style.</p>",
"blurtDate" : ISODate("2019-06-19T18:35:31.156Z"),
"blurtImg" : "04643410-92c1-11e9-80b6-a3262311afff.png",
"vote" : 0,
"author" : {
"id" : ObjectId("5cb5df0ef7a3570bb4ac6e05"),
"name" : "Benjamin Paine"
},
"__v" : 0
}
When I remove .toISOString(), I get documents outside of the expected date range, such as this one in May (query should only return between june 2 and july 2).
{
"_id" : ObjectId("5d07ebaf9a035117e4546349"),
"text" : "<p>A start to something more...</p>",
"blurtDate" : ISODate("2019-05-15T19:36:15.737Z"),
"blurtImg" : "2be7a160-9137-11e9-933f-6966b2e503c7.png",
"vote" : 0,
"author" : {
"id" : ObjectId("5cb5df0ef7a3570bb4ac6e05"),
"name" : "Benjamin Paine"
},
"__v" : 0
}
Your docs contain actual Date objects, so remove the .toISOString()s from your query. But you'll also need to combine your $gte and $lte terms into a single object:
$match: {
'author.id': { $ne: req.user._id },
'blurtDate': { $gte: test1, $lte: test2 }
}

querying Date in mongodb with borders included

I'm evaluating the following query on my collection with fake data:
db.test_result.find({"date": {$gte: ISODate("2021-07-27"), $lte: ISODate("2021-08-31")}}).count()
And despite the fact that I use $lte it does not include the second date value. Is it a bug? If so then how do I make the query so that the left and right borders are included?
Here is what a fake json obj looks like:
{
"nfl": "Some",
"rStatus": false,
"mac": "02:00:00:00:00:00",
"date": "2021-07-27T12:17:57",
"MDCode": "123132132123",
}
With this given input:
{
"_id" : ObjectId("596c6fea53cc7100104628fa"),
"timestamp" : ISODate("2017-07-17T08:06:02.041Z"),
}
{
"_id" : ObjectId("596c7162973f33000fc8bb81"),
"timestamp" : ISODate("2017-07-17T08:12:18.170Z")
}
{
"_id" : ObjectId("596c736c15371f00106b9e3a"),
"timestamp" : ISODate("2017-07-17T08:21:00.291Z")
}
This query:
...find({"timestamp": {$gte: ISODate("2017-07-17T08:06:02.041Z"), $lte: ISODate("2017-07-17T08:12:18.170Z")}})
Would return:
{
"_id" : ObjectId("596c6fea53cc7100104628fa")
"timestamp" : ISODate("2017-07-17T08:06:02.041Z")
}
{
"_id" : ObjectId("596c7162973f33000fc8bb81"),
"timestamp" : ISODate("2017-07-17T08:12:18.170Z")
}
Which is what you would expect, the results match to the dot the 2 borders. So in a nutshell it would be included if it is an exact match, otherwise you would get only the once in between.

MongoDb aggregation Group by Date

I'm trying to group by timestamp for the collection named "foo" { _id, TimeStamp }
db.foos.aggregate(
[
{$group : { _id : new Date (Date.UTC({ $year : '$TimeStamp' },{ $month : '$TimeStamp' },{$dayOfMonth : '$TimeStamp'})) }}
])
Expecting many dates but the result is just one date. The data i'm using is correct (has many foo and different dates except 1970). There's some problem in the date parsing but i can not solve yet.
{
"result" : [
{
"_id" : ISODate("1970-01-01T00:00:00.000Z")
}
],
"ok" : 1
}
Tried this One:
db.foos.aggregate(
[
{$group : { _id : { year : { $year : '$TimeStamp' }, month : { $month : '$TimeStamp' }, day : {$dayOfMonth : '$TimeStamp'} }, count : { $sum : 1 } }},
{$project : { parsedDate : new Date('$_id.year', '$_id.month', '$_id.day') , count : 1, _id : 0} }
])
Result :
uncaught exception: aggregate failed: {
"errmsg" : "exception: disallowed field type Date in object expression (at 'parsedDate')",
"code" : 15992,
"ok" : 0
}
And that one:
db.foos.aggregate(
[
{$group : { _id : { year : { $year : '$TimeStamp' }, month : { $month : '$TimeStamp' }, day : {$dayOfMonth : '$TimeStamp'} }, count : { $sum : 1 } }},
{$project : { parsedDate : Date.UTC('$_id.year', '$_id.month', '$_id.day') , count : 1, _id : 0} }
])
Can not see dates in the result
{
"result" : [
{
"count" : 412
},
{
"count" : 1702
},
{
"count" : 422
}
],
"ok" : 1
}
db.foos.aggregate(
[
{ $project : { day : {$substr: ["$TimeStamp", 0, 10] }}},
{ $group : { _id : "$day", number : { $sum : 1 }}},
{ $sort : { _id : 1 }}
]
)
Group by date can be done in two steps in the aggregation framework, an additional third step is needed for sorting the result, if sorting is desired:
$project in combination with $substr takes the first 10 characters (YYYY:MM:DD) of the ISODate object from each document (the result is a collection of documents with the fields "_id" and "day");
$group groups by day, adding (summing) the number 1 for each matching document;
$sort ascending by "_id", which is the day from the previous aggregation step - this is optional if sorted result is desired.
This solution can not take advantage of indexes like db.twitter.ensureIndex( { TimeStamp: 1 } ), because it transforms the ISODate object to a string object on the fly. For large collections (millions of documents) this could be a performance bottleneck and more sophisticated approaches should be used.
It depends on whether you want to have the date as ISODate type in the final output. If so, then you can do one of two things:
Extract $year, $month, $dayOfMonth from your timestamp and then reconstruct a new date out of them (you are already trying to do that, but you're using syntax that doesn't work in aggregation framework).
If the original Timestamp is of type ISODate() then you can do date arithmetic to subtract the hours, minutes, seconds and milliseconds from your timestamp to get a new date that's "rounded" to the day.
There is an example of 2 here.
Here is how you would do 1. I'm making an assumption that all your dates are this year, but you can easily adjust the math to accommodate your oldest date.
project1={$project:{_id:0,
y:{$subtract:[{$year:"$TimeStamp"}, 2013]},
d:{$subtract:[{$dayOfYear:"$TimeStamp"},1]},
TimeStamp:1,
jan1:{$literal:new ISODate("2013-01-01T00:00:00")}
} };
project2={$project:{tsDate:{$add:[
"$jan1",
{$multiply:["$y", 365*24*60*60*1000]},
{$multiply:["$d", 24*60*60*1000]}
] } } };
Sample data:
db.foos.find({},{_id:0,TimeStamp:1})
{ "TimeStamp" : ISODate("2013-11-13T19:15:05.600Z") }
{ "TimeStamp" : ISODate("2014-02-01T10:00:00Z") }
Aggregation result:
> db.foos.aggregate(project1, project2)
{ "tsDate" : ISODate("2013-11-13T00:00:00Z") }
{ "tsDate" : ISODate("2014-02-01T00:00:00Z") }
This is what I use in one of my projects :
collection.aggregate(
// group results by date
{$group : {
_id : { date : "$date" }
// do whatever you want here, like $push, $sum...
}},
// _id is the date
{$sort : { _id : -1}},
{$orderby: { _id : -1 }})
.toArray()
Where $date is a Date object in mongo. I get results indexed by date.

sort by date with aggregate request in mongodb

I would like to retrieve a list of values ​​that comes from the oldest document currently signed.But i failed to select a document absed on the date.Thanks
here is json :
"ad" : "noc3",
"createdDate" : ISODate(),
"list" : [
{
"id" : "p45",
"value" : 21,
},
{
"id" : "p6",
"value" : 20,
},
{
"id" : "4578",
"value" : 319
}
]
and here my aggregate request :
db.friends.aggregate({$match:{advertiser:"noc3", {$sort:{timestamps:-1},{$limit:1} }},{$unwind:"$list"},{$project:{_id: "$list.id", value:{$add:[0]}}});
Your aggregate query is incorrect. You add the sort and limit to the match, but that's now how you do that. You use different pipeline operators:
db.friends.aggregate( [
{ $match: { advertiser: "noc3" } },
{ $sort: { createdDate: -1 } },
{ $limit: 1 },
Your other pipeline operators are bit strange too, and your code vs query mismatches on timestamps vs createdDate. If you add the expected output, I can update the answer to include the last bits of the query too.