MongoDB: subtract months from date, with value from database - mongodb

Is it possible in MongoDB to somehow subtract only months from a date, with a value from the same document. So for example i have to subtract date from field "valid_to" with months from field "offset", only months need to be changed.
So for example:
2019-11-25 - 3 months = 2019-08-25
2019-12-20 - 5 months = 2019-07-20
Documents in collection:
{
"_id" : ObjectId("5d96f027ad768a1532aa2698"),
"name" : "Q1",
"valid_to" : ISODate("2019-11-25T14:36:32.221+01:00"),
"offset" : 3,
},
{
"_id" : ObjectId("5d96f027ad768a1532aa2699"),
"name" : "Q2",
"valid_to" : ISODate("2019-12-20T14:36:32.221+01:00"),
"offset" : 5,
}
I tried just as an example using JavaScript date function and taking current date and subtracting months but it's not working like this
{
$project: {
"subtractedDate": new Date(new Date().setMonth({ $subtract: [new Date().getMonth(), "$offset"] }))
}
},
])
As a result I am getting ISODate("1970-01-01T01:00:00.000+01:00")

Javascript functions do not work with aggregation framework. The whole aggregation is parsed to BSON format on input, so you can't call any JS inside. The reason is in converting BSON to C++ implementation.
In your example you are using mix of operators and JS, when you try to $subtract inside .setMonth():
$project: {
"subtractedDate": new Date(new Date().setMonth({ $subtract: [new Date().getMonth(), "$offset"] }))
}
So JS doesn't know in advance how to handle Mongo operators and doesn't know about $offset value
Instead you have to use JS, that will be executed before input:
$project: {
"subtractedDate": new Date(new Date().setMonth(new Date().getMonth() - 5))
}
That's the only solution to get you example work. But it won't help in your case since you don't know the exact offset.
What you can do is implement your logic using aggregation operators, there are lots of functions to work with Date like $dateToParts and $dateFromParts where you can easily manipulate with months, years and so on separately.
Or you can add a callback to your aggregation where you can handle all your dates:
db.collection.aggregate([
{
// ...
}
], function( err, results ) {
if ( !err ) {
results.forEach( result => {
result.date = new Date(result.date)
.setMonth(new Date(result.date).getMonth() - result.offset);
});
}
callback(err, results);
});
}

Related

mongodb: how to make selection

I Have a mongodb with objects like this:
{
"_id" : "7XXXXXXXXXX",
"apps" : [
{
"id" : "e0d538e0df9a345e",
"os" : "android",
"token" : "f1zp-VSi7Ec:APA91bEAbfH8nMeVidkIjPrJ28WHRFDy-BhvKCQdDSdCYsylUzET9GjbPHjofCrr1NMQurinMCI4fuiF7VWNPXne-80op_h0MloK217kc1zKptdo9FTgAH5R932uDphcyB1xQ27-AFdR",
"version" : "3.2.1",
"build" : "8115680e",
"timestamp" : NumberLong(1571740696818)
}
]
}
How i can select objects older certain date using timestampin my case, for example older 3 month?
You can use $toDate operator in aggregation to do the desired operation,
I hope you are using mongo version 4.0+
$toDate is supported in mongo version 4.0 and on
let selectedDate = new Date();
selectedDate.setDate(d.getDate()-30); //subtracting 30 days from today's date
db.collection("your_collection_name").aggregate({$unwind:{ path: "$apps"}},
{$addFields: { dateValue: {$toDate: "$apps.timestamp" }}},
{$match: { dateValue: {$lte: selectedDate }}},
(err, result) => {
//perform your desired operations
});
Explanation:
basically, I am first unwinding apps array, which will result in having a separate document of each entry in apps.
Then operate on the timestamp field in each document, and convert it into a proper date with $toDate.
Then in the next stage apply your filter with $match.
UPDATE (from comments):
as you are using mongo version 3.2 the above solution will not work.
then I think, you can opt for another approach here:
Query all your documents in this particular collection, find the proper date from the timestamp field.
Update each document with a new field which will now have the value of computed date from the above step, and save it.
I suggest you write a migration script for the above two steps.
Make sure when inserting a new document, you already add this newly computed date field.
Then you can have simple query like:
db.collection("your_collection_name").find({"app.newDateField": {$lte: {selectedDate }}},
{ "apps.$": 1},
(err, result)=>{
})

Get all records with specified properties between a certain date range

I am passing in two dates formatted as MM-DD-YYYY which is a date range. I need to query all records within that range and include specified fields. I've had no luck.
Part of a record in Mongo:
{
"_id": "some ID",
"date": {
"$date": "2015-06-26T13:02:12.121Z"
},
Query:
var Start = '09-07-2015'
var End = '09-14-2015'
If I do:
var query = Order.find({
date : {
$lt : End,
$gt : Start
}
});
I get the full document within the week ranges as expected. However, I want to specify the fields to return rather than full document. So I've tried using grouping and project to specify those fields:
var query = Order.aggregate(
{
$match :
{
date: {
$gte: start,
$lt: end
}
},
$group:
{
cust_ID: '$request.headers.customer_id',
wholesaler_ID: '$request.headers.wholesalerID'
}
}
);
Likewise: I've also tried it using project to get the results I want. I thought maybe it won't match on a date string like 09-07-2015, so I included the ISO date directly. Still no luck... the query comes back undefined or empty:
var query = Order.aggregate(
{
$project:
{
date: 'date',
cust_ID: '$request.headers.custID',
wholesaler_ID: '$request.headers.wholesalerID'
}
},
{
$match :
{
date: {
$gte: "2014-12-09T21:02:56.872Z",
$lt: "2015-12-09T21:02:56.872Z"
}
}
}
);
var query = Order.find({
date : {
$lt : End,
$gt : Start
}}, {cust_ID:1, wholeseller_ID:1}
);
This will work.
I just tested this using Robomongo
db.getCollection('offerdb').find({time_posted:{$gt: '2015-10-21T21:40:04+05:30', $lte:'2015-12-14T05:53:14+05:30'}},{_id:1, merchant_id:1})
Works like a charm for me.
Try this command in mongo shell
use dbname
db.collection_name.find({date: {$gte: ISODate('2015-09-07 00:00:00'), $lte: ISODate('2015-09-14 23:59:59.999999')}},{'cust_ID':1,'_id':0,'wholeseller_ID':1})

How can convert string to date with mongo aggregation?

In a collection, I store this kind of document
{
"_id" : 1,
"created_at" : "2016/01/01 12:10:10",
...
}.
{
"_id" : 2,
"created_at" : "2016/01/04 12:10:10",
...
}
I would like to find documents have "creared_at" > 2016/01/01 by using aggregation pipeline.
Anybody have solution to convert "created_at" to date so can conpare in aggregation?
All the above answers use cursors but however, mongodb always recommend to use aggregation pipeline. With the new $dateFromString in mongodb 3.6, its pretty much simple.
https://docs.mongodb.com/manual/reference/operator/aggregation/dateFromString/
db.collection.aggregate([
{$project:{ created_at:{$dateFromString:{dateString:'$created_at'}}}}
])
As you have mentioned, you need to first change your schema so that the created_at field holds date objects as opposed to string as is the current situation, then you can query your collection either using the find() method or the aggregation framework. The former would be the most simple approach.
To convert created_at to date field, you would need to iterate the cursor returned by the find() method using the forEach() method, within the loop convert the created_at field to a Date object and then update the field using the $set operator.
Take advantage of using the Bulk API for bulk updates which offer better performance as you will be sending the operations to the server in batches of say 1000 which gives you a better performance as you are not sending every request to the server, just once in every 1000 requests.
The following demonstrates this approach, the first example uses the Bulk API available in MongoDB versions >= 2.6 and < 3.2. It updates all
the documents in the collection by changing the created_at fields to date fields:
var bulk = db.collection.initializeUnorderedBulkOp(),
counter = 0;
db.collection.find({"created_at": {"$exists": true, "$type": 2 }}).forEach(function (doc) {
var newDate = new Date(doc.created_at);
bulk.find({ "_id": doc._id }).updateOne({
"$set": { "created_at": newDate}
});
counter++;
if (counter % 1000 == 0) {
bulk.execute(); // Execute per 1000 operations and re-initialize every 1000 update statements
bulk = db.collection.initializeUnorderedBulkOp();
}
})
// Clean up remaining operations in queue
if (counter % 1000 != 0) { bulk.execute(); }
The next example applies to the new MongoDB version 3.2 which has since deprecated the Bulk API and provided a newer set of apis using bulkWrite():
var cursor = db.collection.find({"created_at": {"$exists": true, "$type": 2 }}),
bulkOps = [];
cursor.forEach(function (doc) {
var newDate = new Date(doc.created_at);
bulkOps.push(
{
"updateOne": {
"filter": { "_id": doc._id } ,
"update": { "$set": { "created_at": newDate } }
}
}
);
if (bulkOps.length === 1000) {
db.collection.bulkWrite(bulkOps);
bulkOps = [];
}
});
if (bulkOps.length > 0) { db.collection.bulkWrite(bulkOps); }
Once the schema modification is complete, you can then query your collection for the date:
var dt = new Date("2016/01/01");
db.collection.find({ "created_at": { "$gt": dt } });
And should you wish to query using the aggregation framework, run the following pipeline to get the desired result. It uses the $match operator, which is similar to the find() method:
var dt = new Date("2016/01/01");
db.collection.aggregate([
{
"$match": { "created_at": { "$gt": dt } }
}
])
If we have documents:
db.doc.save({ "_id" : 1, "created_at" : "2016/01/01 12:10:10" })
db.doc.save({ "_id" : 2, "created_at" : "2016/01/04 12:10:10" })
Simple query:
db.doc.find({ "created_at" : {"$lte": Date()} })
Aggregate query:
db.doc.aggregate([{
"$match": { "created_at": { "$lte": Date() } }
}])
Date() method which returns the current date as a string.
new Date() constructor which returns a Date object using the ISODate() wrapper.
ISODate() constructor which returns a Date object using the ISODate()
wrapper.
More information about the type of date here and here

Mongodb Aggregation Framework and timestamp

I have a collection
{ "_id" : 1325376000, "value" : 13393}
{ "_id" : 1325462400, "value" : 13393}
ObjectIds are Unix Timestamp and are storing as Number manually.(at insert time).
now I'm searching for a solution that i could calculate sum of values for each month with Aggregation Framework.
Here is a way you can do it by generating the aggregation pipeline programmatically:
numberOfMonths=24; /* number of months you want to go back from today's */
now=new Date();
year=now.getFullYear();
mo=now.getMonth();
months=[];
for (i=0;i<numberOfMonths;i++) {
m1=mo-i+1; m2=m1-1;
d = new Date(year,m1,1);
d2=new Date(year,m2,1);
from= d2.getTime()/1000;
to= d.getTime()/1000;
dt={from:from, to:to, month:d2}; months.push(dt);
}
prev="$nothing";
cond={};
months.forEach(function(m) {
cond={$cond: [{$and :[ {$gte:["$_id",m.from]}, {$lt:["$_id",m.to]} ]}, m.month, prev]};
prev=cond;
} );
/* now you can use "cond" variable in your pipeline to generate month */
db.collection.aggregate( { $project: { month: cond , value:1 } },
{ $group: {_id:"$month", sum:{$sum:"$value"} } }
)

Why are dates in match aggregate query being ignored?

I'm trying to run an aggregation statement in my mongo db. I have a document whose structure is (at least) as follows:
{
"_id": ObjectId,
"date": ISODate,
"keywordGroupId": NumberLong,
"ranking": NumberLong,
}
I would like to run an aggregation statement that aggregates the 'ranking' field for a given 'keywordGroupId' and a given 'date' interval.
I have been trying with the following aggregate command:
{
aggregate : "KeywordHistory",
pipeline : [
{ $match: { keywordGroupId: 75 , "$date": {$gte: ISODate("2013-01-01T00:00:00.0Z"), $lt: ISODate("2013-02-01T00:00:00.0Z")}} },
{ $group: { _id: { null }, count: { $sum: "$ranking" } } }
]
}
This command executes without errors and returns a result. If I try to change the value for the 'keywordGroupId' field, the command returns a different value, so I assume that the $match statement works for that field (NumberLong). Though, if I change the 'date' range and I specify a time interval for which I don't have any data in the database, it still returns a result (I would actually expect an empty result set). So I have to assume that the $match statement is ignoring the date interval specified.
Can anyone help me with this point?
Remove the $ prefix on the $date field of your $match:
{ $match: {
keywordGroupId: 75,
date: {$gte: ISODate("2013-01-01T00:00:00.0Z"), $lt: ISODate("2013-02-01T00:00:00.0Z")}
}},
You only use the $ prefix when the field name is used in a value, not as a key.
Sometimes ISodate does not works . so in Case if you want to match date using only "one" date the best way is:---
ex:-- Let a schema be:---
var storeOrder = new Schema({
store_name:{type:String, required:true},
date :{type:Date ,default:moment(new Date()).format('YYYY-MM-DD')},
orders : [{
vegetable : String,
quantity : Number,
price:Number
}]
});
mongoose.model('storeorder',storeOrder);
now to aggregate by matching date :--
storeOrder.aggregate([$match:{date :new Date("2016-12-26T00:00:00.000Z")} ])
**It is must to use new Date("2016-12-26T00:00:00.000z") instead of Date("2016-12-26T00:00:00.000z") because Date(your_date) !== new Date(your_date).
THANK YOU
The aggregate expects a Javascript Date Object and doesn't work otherwise.
new Date();
new Date(year, month, day);
Please note the month start with 0 and not 1 (Your January is 0 and December 11)