How do I do calculation in sort in aggreation mongoose - mongodb

I am trying to do this calculation but I am having some trouble. Anyone have any ideas how I can do calculation inside my aggregation? I wanna do something like this:
const test = await Test.aggregate([
{
$sort: {
$divide: [
'value',
Math.pow(1.1, new Date() - 'date'),
],
},
},
]);
For example here, I wanna do 1.1^number of days has passed. The Test schema has a "value" of type Float and a date of type date.

You have two issues here:
You're trying to use javascript functions within the pipeline, while this is possible by using $function it is not recommended, especially if you can execute the same logic using Mongodb operators.
$sort stage has this following structure:
{ $sort: { <field1>: <sort order>, <field2>: <sort order> ... } }
As you can tell it's not being followed in your example as you're trying to use an expression.
So how can we solve these?
Well you can use $pow instead of Math.pow, $$NOW instead of new Date() and $subtract instead of the - javascript operator.
You will also need to add a "sortField" to sort by to match the $sort stage structure, all of this would look like this:
db.collection.aggregate([
{
"$addFields": {
"sortField": {
$divide: [
"$value",
{
$pow: [
1.1,
{
$subtract: [
"$$NOW",
"$date"
]
}
]
}
]
}
}
},
{
$sort: {
sortField: 1
}
}
])
Mongo Playground
Mind you subtracting dates will give you result in miliseconds, you will have to divide it by the required number ( 60 * 1000 * 60 * 24 for a day ) to get the right mesaurement.

Related

How to write a query to find the mongoDB documents whose time difference between two Date fields is larger than a certain value?

I have a mongoDB that contains documents like this:
The data types of start_local_datetime and last_update_local_datetime are both Date.
How can I find the documents whose difference between last_update_local_datetime and start_local_datetime is larger than 10 days?
I mean I want to query data like this:
start_local_datetime: 2019-08-23T10:17:42.000+00:00
terminate_local_datetime: 2019-09-19T10:17:42.000+00:00
Documents like this aren't something that I want.
start_local_datetime: 2019-08-23T10:17:42.000+00:00
terminate_local_datetime: 2019-08-25T10:17:42.000+00:00
Because terminate_local_datetime - start_local_datetime is smaller than 10 days.
You can write an aggregation pipeline, using $dateDiff operator, like this:
db.collection.aggregate([
{
"$match": {
$expr: {
"$gt": [
{
"$dateDiff": {
"startDate": "$start_local_datetime",
"unit": "day",
"endDate": "$terminate_local_datetime"
}
},
10
]
}
}
}
])
See it working here. However, this will only work in Mongo 5.0 or above, as the operator was added in that version. For other versions, this will work
db.collection.aggregate([
{
"$addFields": {
"timeDifference": {
"$divide": [
{
"$subtract": [ <-- Returns difference between two dates in milliseconds
"$terminate_local_datetime",
"$start_local_datetime"
]
},
86400000
]
}
}
},
{
"$match": {
$expr: {
"$gt": [
"$timeDifference",
10
]
}
}
},
{
"$project": {
timeDifference: 0
}
}
])
Here, we calculate the time difference manually and then compare it, with 10.
This is the playground link.

Is it possible to list in mongodb the list of elements whose value is less than 10% of another field?

I basically have a database where I record motorcycles and their mileage.
{
"motorcycle":"A",
"current_km":4600,
"review_km":5000
},
{
"motorcycle":"B",
"current_km":4000,
"review_km":5000
},
{
"motorcycle":"C",
"current_km":4900,
"review_km":5000
},
{
"motorcycle":"D",
"current_km":3000,
"review_km":5000
}
I have a field called current_km that determines your current mileage and I have another field called review_km, which consists of specifying the mileage in which your review should be done, as long as your current mileage (current_km) is greater than 10% of Mileage review (review_km).
So I would like to list the elements where:
current_km is greater than:
(review_km - ( review_km * 0.10))
for example:
current_km = 4600;
review_km = 5000;
result = 5000 - (5000 * 0.10);
4600 (current_km)> = 4500 (result) // in this case it is showed
In my database it would show the results of motorcycles A and C
how can I do it? I don't know if it is possible to do it in mongodb directly.
Need to use aggregation with $subtract and $multiply,
$addFields add new fields, we are generating result field, equation (review_km - ( review_km * 0.10)) using $subtract and $multiply
$match equation in $expr if current_km >= result if its correct then returns document
db.collection.aggregate([
{
$addFields: {
result: {
$subtract: [
"$review_km",
{
$multiply: [
"$review_km",
0.10
]
}
]
}
}
},
{
$match: {
$expr: {
$gte: [
"$current_km",
"$result"
]
}
}
}
])
Working Playground: https://mongoplayground.net/p/s2qenvuzLKF
Shorter version
If you don't want result field in response then combined condition in $match and $addFields is no longer needed
db.collection.aggregate([
{
$match: {
$expr: {
$gte: [
"$current_km",
{
$subtract: [
"$review_km",
{
$multiply: [
"$review_km",
0.10
]
}
]
}
]
}
}
}
])
Working Playground: https://mongoplayground.net/p/fii__3tTika

MongoDB date math with aggregation variable

I'm trying to build an aggregation of things that haven't reported in by some interval (heartbeat) - I need to calculate a value based on a stored heartbeat:
db.things.aggregate([
{$project: {"lastmsg":1, "props.settings":1}},
{$unwind: "$props.settings"},
{$project: {
_id:0,
"lastmsg": "$lastmsg",
"heartbeat": {$multiply: [{$toInt: "$props.settings.heartbeat"},2000]},
"now": new Date(), "subtracted": new Date(new Date().getTime()- "$heartbeat")
}
}
])
Result returned is like this:
{ "lastmsg" : ISODate("2020-04-23T12:41:37.667Z"), "heartbeat" : 240000, "now" : ISODate("2020-05-14T16:26:11.824Z"), "subtracted" : ISODate("1970-01-01T00:00:00Z") }
{ "lastmsg" : ISODate("2020-05-14T16:24:24.228Z"), "heartbeat" : 240000, "now" : ISODate("2020-05-14T16:26:11.824Z"), "subtracted" : ISODate("1970-01-01T00:00:00Z") }
The "subtracted" projection is not doing the date math as expected. I can plug in a specific number and it works but this defeats the purpose...
As a last step I will match to see what of these things hasn't checked in within the interval of heartbeat:
{ $match: { "lastmsg":{$gte: "$subtracted")}
Any help would be greatly appreciated...
I don't know how your data is like (you should post your data to help), but I think this can solve the problem.
You can use the $$NOW variable, that returns the current date in ISODate format.
Test data:
[
{
"lastmsg": ISODate("2020-04-23T12:41:37.667Z"),
"heartbeat": 240000
},
{
"lastmsg": ISODate("2020-05-14T16:24:24.228Z"),
"heartbeat": 240000
}
]
Query:
db.collection.aggregate([
{
$addFields: {
"now": "$$NOW",
"subtracted": {
$subtract: [
"$$NOW",
"$heartbeat"
]
}
}
},
{
$match: {
"lastmg": {
$gte: "$subtracted"
}
}
}
])

Mongo query using aggregation for dates

In the following query I'm trying to find entries in my articles collection made in the last week, sorted by the number of votes on that article. The $match doesn't seem to work(maybe I dont know how to use it). The following query works perfectly, so its not a date format issue,
db.articles.find(timestamp:{
'$lte':new Date(),
'$gte':new Date(ISODate().getTime()-7*1000*86400)}
})
But this one doesn't fetch any results. Without the $match it also fetches the required results(articles sorted by votecount).
db.articles.aggregate([
{
$project:{
_id:1,
numVotes:{$subtract:[{$size:"$votes.up"},{$size:"$votes.down"}]}}
},
{
$sort:{numVotes:-1}
},
{
$match:{
timestamp:{
'$lte':new Date(),
'$gte':new Date(ISODate().getTime()-7*1000*86400)}
}
}
])
You are trying to match at the end of your pipeline, which supposes you have projected timestamp field, and you haven't done that.
I believe what you want is to filter data before aggregation, so you should place match at the top of your aggregation array.
Try this:
db.articles.aggregate([{
$match: {
timestamp: {
'$lte': new Date(),
'$gte': new Date(ISODate().getTime() - 7 * 1000 * 86400)
}
}
}, {
$project: {
_id: 1,
numVotes: {
$subtract: [{
$size: "$votes.up"
}, {
$size: "$votes.down"
}]
}
}
}, {
$sort: {
numVotes: -1
}
}])

Mongo $subtract date doesn't work in aggregation $match block

I am creating a mongo aggregation query which use a $subtract operator in my $match block. As explained in these codes below.
This query doesn't work:
db.coll.aggregate(
[
{
$match: {
timestamp: {
$gte: {
$subtract: [new Date(), 24 * 60 * 60 * 1000]
}
}
}
},
{
$group: {
_id: {
timestamp: "$timestamp"
},
total: {
$sum: 1
}
}
},
{
$project: {
_id: 0,
timestamp: "$_id.timestamp",
total: "$total",
}
},
{
$sort: {
timestamp: -1
}
}
]
)
However, this second query work:
db.coll.aggregate(
[
{
$match: {
timestamp: {
$gte: new Date(new Date() - 24 * 60 * 60 * 1000)
}
}
},
{
$group: {
_id: {
timestamp: "$timestamp"
},
total: {
$sum: 1
}
}
},
{
$project: {
_id: 0,
timestamp: "$_id.timestamp",
total: "$total",
}
},
{
$sort: {
timestamp: -1
}
}
]
)
I need to use $subtract on my $match block so I can't use the last query.
As of mongodb 3.6 you can use $subtract in the $match stage via the $expr. Here's the docs: https://docs.mongodb.com/manual/reference/operator/query/expr/
I was able to get a query like what you're describing via this $expr and a new system variable in mongodb 4.2 called $$NOW. Here is my query, which gives me orders that have been created within the last 4 hours:
[
{ $match:
{ $expr:
{ $gt: [
"$_created_at",
{ $subtract: [ "$$NOW", 4 * 60 * 60 * 1000] } ]
}
}
}
]
Well you cannot do that and you are not meant to do so either. Another valid thing is that you say to "need" to do this but in reality you really do not.
Pretty much all of the general aggregation operators outside of the pipeline operators are really only valid within a $project or a $group pipeline stage. Mostly within $project but certainly not in others.
A $match pipeline is really the same as a general "query" operation, so the only things valid in there are the query operators.
As for the case for your "need", any "value" that is submitted within an aggregation pipeline and particularly within a $match needs to be evaluated outside of the actual pipeline before the BSON representation is sent to the server.
The only exception is the notation that defines variables in the document, particularly "fieldnames" such a "$fieldname" and then only really in $project or $group. So that means something that "refers" to an existing value of a document, and that is something that cannot be done within any type of "query" document expression.
If you need to work with the value of another field in the document then you work it out with $project first, as in:
db.collection.aggregate([
{ "$project": {
"fieldMath": { "$subtract": [ "$fieldOne", "$fieldTwo" ] }
}},
{ "$match": { "fieldMath": { "$gt": 2 } }}
])
For any other purpose you really want to evaluate the value "outside" the pipeline.
The above answers the question you asked, but this answers the question you didn't ask.
Your pipeline doesn't make any sense since grouping on the "timestamp" alone would be unlikely to group anything since the values are of millisecond accuracy and there is likely not to be more than just a few at best for very active systems.
It appears like you are looking for the math to group by "day", which you can do like this:
db.collection.aggregate([
{ "$group": {
"_id": {
"$subtract": [
{ "$subtract": [ "$timestamp", new Date(0) ] },
{ "$mod": [
{ "$subtract": [ "$timestamp", new Date(0) ] },
1000 * 60 * 60 * 24
]}
]
},
"total": { "$sum": "$total" }
}}
])
That "rounds" your timestamp value to a single day and has a much better chance of "aggregating" something than you would otherwise have.
Or you can use the "date aggregation operators" to do much the same thing with a composite key.
So if you want to "query" then it evaluates externally. If you want to work on a value "within the document" then you must do so in either a $project or $group pipeline stage.
The $subtract operator is a projection-operator. It is only available during a $project step. So your options are:
(not recommended) Add a $project-step before your $match-step to convert the timestamp field of all documents for the following match-step. I would not recommend you to do this because this operation needs to be performed on every single document on your database and prevents the database from using an index on the timestamp field, so it could cost you a lot of performance.
(recommended) Generate the Date you want to match against in the shell / in your application. Generate a new Date() object, store it in a variable, subtract 24 hours from it and perform your 2nd query using that variable.