A capture from mongoDb with the structure of a Date time
You can set TTL indexes while creating your records.TTL indexes are special single-field indexes that MongoDB can use to automatically remove documents from a collection after a certain amount of time or at a specific clock time.
To create a TTL index, use the db.collection.createIndex() method with the expireAfterSeconds option on a field whose value is either a date or an array that contains date values.
For example, to create a TTL index that removes the record after 30 days on the createdDate field of the User collection, use the following operation in the mongo shell:
db.User.createIndex( { "createdDate": 1 }, { expireAfterSeconds: 2592000 } )
src: https://docs.mongodb.com/manual/core/index-ttl/
Related
I'm trying to update the value of the expiry date in a document. Here's the Node.js code I use to set the index everytime I want to update the document:
database.collection(collectionName).createIndex({ "expires_on": 1 }, { expireAfterSeconds: 0 })
database.collection(collectionName).updateOne(query, update, { upsert: true }, (error, result) => {...})
The thing is that I set the expires_on date to the date of tomorrow and the value is set correctly in database but it expires within a few seconds. Is there a problem with the way I'm updating this field? I don't know how I can update this document in a way that it would expire on the last expires_on date it has been updated for.
I have noticed that you can't update the indexes in MongoDB but am I updating the index here? Isn't it different when I try to change the value, not the index itself?
You have understood ttl index bit strange way...
In your code example you set every document what have timestamp column "expires_on" to expire right now (expireAfterSeconds: 0).
Expiring process is running background and if you set expireAfterSeconds to 0, it will find out all documents which "expires_on" value is less or equal to now().
So, your update of field "expires_on" must happen at moment where "now() -lt expires_on".
Better say f.ex. "expireAfterSeconds: 3600" and then update expires_on to value "tomorrow minus that 3600 seconds"...
The fact I was creating an index every time a new data was inserted was not logical. I set the index in my database once and only updated the document each time:
database.collection(collectionName).updateOne(query, update, { upsert: true }, (error, result) => {...})
However, the problem turned out to be the value I set for the expires_on in my code. Due to the asynchronicity of Node.js, the value wasn't calculated correctly.
I need to fetch the first inserted record in a collection in MongoDB for which I am currently using the below query:
db.users.find({}).sort({"created_at":1}).limit(1);
But, this takes up a lot of memory. The collection has about 100K records.
What is the efficient way to do this?
MongoDB _id is unique identifier which is automatically generated upon insertion of document into MongoDB collection
_id field stores ObjectId value and is automatically indexed.
According to MongoDB documentation,
The 12-byte ObjectId value consists of:
4-byte value representing the seconds since the Unix epoch,
3-byte machine identifier,
2-byte process id, and
3-byte counter, starting with a random value.
According to description as mentioned into above question to fetch first inserted record please try executing following mongodb find operation into MongoDB shell.
db.users.find({}).sort({"_id":1}).limit(1);
In above query we have sorted result according to _id field since _id Object ID value consists of unix epoch timestamp
further to this you can add specific filters in query to get first record inserted for that criteria:
like suppose you collection contains data for storing employees from IT, ADMIN, FINANCE department and you want to look for the first document inserted for IT (i.e. first IT employee) then you can execute:
db.users.find({"Dept" : "IT"}).sort({"_id":1}).limit(1);
and similarly to find last employee:
db.users.find({"Dept" : "IT"}).sort({"_id":-1}).limit(1);
Note: for bigger collections/sharded collection it will take considerable time to get the result as it iterates entire _id field for ascending and descending criteria.
I am new to MongoDB. I have a MongoDB database which has documents of each minute(timestamp field). To optimise the query performance I created a compound index:
db.data.createIndex({dId:1,ts:-1})
But my query has:
"$match":{ dId: deviceId,year_agg:year_input,month_agg:month_input }
I wish to create index on only month of the timestamp so that the query time is reduced. Is that possible?
I have created one collection in mongoDB for which i have created index for field "expireAt" which is a date time and set expireAfterSeconds property to 0.
As i want to set TTL at document level not at collection level.
As i know in redis i can get the remaining TTL for the particular key
Just have question is that possible in mongoDB ? where i can get the remaining time to expire for the given document.
I have searched it on google but didn't find any relevant information.
Thank You
If you set the TTL expireAfterSeconds value to 0 for per-document expiry, the remaining time will be the difference between the current time and the TTL date field (eg. expireAt) in your documents. There isn't a specific server function to query this, but you can either calculate this in your app or use MongoDB's aggregation framework.
Example using the mongo shell to return the time left (in milliseconds) before documents are eligible to be removed in the next TTL pass:
db.mycoll.aggregate(
{ $project: {
expireAt: 1,
ttlMillis: {
$subtract: [ "$expireAt", new Date() ]
}
}}
)
Note: The TTL background thread runs every 60 seconds, so documents may persist for a minute or more past their nominal expiry.
try this
db.hellos.createIndex( { "expireAt": 1 }, { expireAfterSeconds: 3600 } )
read more here
In our company we have a retention of eight days of data (with one million of records aprox.) so we have a cronjob that remove documents older than eight days each day. Now we're using the Published field and this field is not indexed.
It takes like 15 minutes to finish to get rid off 100.000 records and we found that this operation is too long.
This is the query where 'docs' is a variable with an array of documents that we don't want to remove. The 'theDate' variable is the date of eight days ago.
records.remove( { "Published" : { $lte : theDate }, "_id" : { $nin : docs } }
Would be it better to use the _id field, which is indexed, in ordered to do this operation? How can we use the _id field in order to do the same operation?
Discard the Cron job entirely: this is a job for TTL indexes. http://docs.mongodb.org/manual/core/index-ttl/
Create a TTL index on the Published field with expireAfterSeconds: 691200 and watch as your documents are automatically removed 8 days after publication.
And if you don't want to indiscriminately delete all documents 8 days after their publication, keep your Cron job and just create a plain index on the Published field.