I have a large amount of data in my MongoDB and I want to query those records which were updated in the last 2 hours. can someone help
Pretty simple actually. First, add an updatedAt attribute at your collection. But I assume that you already have this.
So, in short:
db.collection.find({ "updatedAt" : { $lt: new Date(Date.now() - 2 * 60 * 60 * 1000) } })
If you did not have an updatedAt attribute, then this one is also possible.
db.collection.find({ $where: function () {
return Date.now() - this._id.getTimestamp() < (2 * 60 * 60 * 1000)
} })
Explanation:
You will find all documents whose updatedAt attribute is less than 7200 seconds.
You will find all documents whose ObjectID is less than 7200 seconds.
Remember that ObjectID's timestamp can be retrieved.
Related
MongoDB 5
9 million records in a collection
sparse index on a createdAt field.
Aggregation match to find all records created before a certain date
Issue is, the index is hit, but the IXSCAN takes about 60 seconds.
What can be done to speed this up?
const result = await collection.aggregate([{ $match: { createdAt: { $lte: /* SOME DATE */ } } }]);
I want make a query to return all documents from a collection created in that day.
This is what i done so far:
db.entities.find({"creDate":{"$gte": new Date(Date.now() - (1000 * 60 * 60 * 24))}});
The problem is that the atributte "creDate" format is unix. Example: 1653408094.1847994
So, in the above query i need to convert "creDate" to Date Object.
Thanks!
I am creating a script that would run after every x minutes and needs to gather data from MongoDB by timestamps.
For example, how would I match the documents with aggregation that have a timestamp in the following timeframe:
start_time = current_time - 60 min
end_time = start_time + 30 min
And I would need to get all the documents that stay within that time frame.
The MongoDB objects have ISODate timestamps on them.
Thanks!
You can create date objects in mongo shell like so:
db.getCollection('order').aggregate([
{
$match : {
start_time : {$gte : new Date(ISODate().getTime() - 1000 * 60 * 60)},
end_time : {$lte : new Date(ISODate().getTime() + 1000 * 60 * 30)}
}
}
...
])
You can use this in aggregate but also in normal find queries.
Note I wrote this without testing, so it might have syntax errors..
db.collection.find({"createdAt": { $gt: new Date('2017-04-25')},"updatedAt":{$lt:new Date('2017-06-25')}})
updatedAt and createdAt are the feilds I have taken at the time of designing the schema by timestamp. You could give feilds according to you design.
the find query would be little better than aggregate in this case as no complex functions have to be performed
Data in MongoDB collection has format
{ "_id" : ObjectId("57a1bfc103c8851a98dba3b2"), "createdOn": NumberLong("1470218177561"), "name": "Raja", "description": "Employee Raja" }
Mongo DB Query and Results
> new Date(1470218177561);
ISODate("2016-08-03T09:56:17.561Z")
> new Date(1888888888888);
ISODate("2029-11-09T03:21:28.888Z")
> db.employee.find("{createdOn: { $lt: new Date(NumberLong(1888888888888)) }}");
The last query returns no result without any errors, so I can't determine what is wrong with my query.
Basically, want to find the records for the last 5 days with `$lt operator in Jongo. Tried the next query, but it also not working
Calendar cal = Calendar.getInstance();
cal.add(Calendar.DATE, -5);
Date dateBefore = cal.getTime();
collection.find("{createdOn: {$gte : #}}", dateBefore).as(type);
Thanks.
You are comparing
NumberLong("1470218177561") vs new Date(NumberLong(1888888888888))
instead of comparing
NumberLong("1470218177561") vs NumberLong(1888888888888)
In your particular case the query will be
db.employee.find( { createdOn: { $lt: NumberLong(1888888888888) } } );
To find documents that are 5 days old you can look at.
To get current date in milliseconds
> new Date(new Date()).getTime()
1470232681000
To find 5 days old documents use $gt operator on the date that is 5 days before now
> new Date(new Date()).getTime() - 5 * 24 * 60 * 60 * 1000
1469800939000
So the query will be
db.employee.find( { createdOn: { $gt: new Date(new Date()).getTime() - 5 * 24 * 60 * 60 * 1000 } } );
To get the same result with Jongo
Calendar cal = Calendar.getInstance();
cal.setTime(new Date());
employees.find("{ createdOn: { $gt: # } }", cal.getTimeInMillis() - 5 * 24 * 60 * 60 * 1000);
The problem with your query for Jongo was the same. You were trying to find by Date type, but date field of documents in your collection is NumberLong, that is millisecond long representation of Date type.
You cannot compare 2 different types with operators like $gt.
I'm trying to get a specific range of documents, based on when they were created. What I'm trying to do is something like:
/getclaims/2015-01
/getclaims/2015-02
...
that way a user can browse through all records based on the selected month.
In my database I'm not storing a created_at date, but I know mongodb stores this in the objectid.
I found that I can get records like this:
db.claims.find({
$where: function () { return Date.now() - this._id.getTimestamp() < (365 * 24 * 60 * 60 * 1000) }
})
of course that doesn't filter based on a specific month, but only within a certain time limit.
What would be a possible way of limited a query based on a specific month, using the Timestamp from the objectid's?
I'm using mongoose, but it's probably a good idea to start in mongo shell itself.
Based on the function borrowed from the answer to this question - https://stackoverflow.com/a/8753670/131809
function objectIdWithTimestamp(timestamp) {
// Convert date object to hex seconds since Unix epoch
var hexSeconds = Math.floor(timestamp/1000).toString(16);
// Create an ObjectId with that hex timestamp
return ObjectId(hexSeconds + "0000000000000000");
}
Create a start and an end date for the month you're looking for:
var start = objectIdWithTimestamp(new Date(2015, 01, 01));
var end = objectIdWithTimestamp(new Date(2015, 01, 31));
Then, run the query with $gte and $lt:
db.claims.find({_id: {$gte: start, $lt: end}});