I have created one collection in mongoDB for which i have created index for field "expireAt" which is a date time and set expireAfterSeconds property to 0.
As i want to set TTL at document level not at collection level.
As i know in redis i can get the remaining TTL for the particular key
Just have question is that possible in mongoDB ? where i can get the remaining time to expire for the given document.
I have searched it on google but didn't find any relevant information.
Thank You
If you set the TTL expireAfterSeconds value to 0 for per-document expiry, the remaining time will be the difference between the current time and the TTL date field (eg. expireAt) in your documents. There isn't a specific server function to query this, but you can either calculate this in your app or use MongoDB's aggregation framework.
Example using the mongo shell to return the time left (in milliseconds) before documents are eligible to be removed in the next TTL pass:
db.mycoll.aggregate(
{ $project: {
expireAt: 1,
ttlMillis: {
$subtract: [ "$expireAt", new Date() ]
}
}}
)
Note: The TTL background thread runs every 60 seconds, so documents may persist for a minute or more past their nominal expiry.
try this
db.hellos.createIndex( { "expireAt": 1 }, { expireAfterSeconds: 3600 } )
read more here
Related
Mongodb has an option to set TTL on documents.
I would like to know if there is a similar feature that allows the update of a specific field after a specific amount of time.
Basically what I want to achieve is update the field STATUS of a document from PENDING to EXPIRED after a specific amount of time (only if it was PENDING).
I know I could use cronjobs, but first I want to check whether it's possible natively with Mongodb.
Furthermore, is it possible to set the TTL with condition? Like deleting a document after X days only if STATUS is EXPIRED?
what I want to achieve is update the field STATUS of a document from PENDING to EXPIRED after a specific amount of time
Not achievable today, you have to create your own script
Furthermore, is it possible to set the TTL with condition? Like
deleting a document after X days only if STATUS is EXPIRED?
I recommend you that the script that set the status also set readyToBeDeletedSince: Date and that you put your TTL on that field.
Regarding the TTL with condition, I found that there is an option called "partialFilterExpression" that you can add to the index.
For example the following will delete a document after 5 minutes only if status is "PENDING"
db.mymodel.createIndex({ createdAt: 1 }, { expireAfterSeconds: 300, partialFilterExpression: { status: 'PENDING' } })
I'm trying to update the value of the expiry date in a document. Here's the Node.js code I use to set the index everytime I want to update the document:
database.collection(collectionName).createIndex({ "expires_on": 1 }, { expireAfterSeconds: 0 })
database.collection(collectionName).updateOne(query, update, { upsert: true }, (error, result) => {...})
The thing is that I set the expires_on date to the date of tomorrow and the value is set correctly in database but it expires within a few seconds. Is there a problem with the way I'm updating this field? I don't know how I can update this document in a way that it would expire on the last expires_on date it has been updated for.
I have noticed that you can't update the indexes in MongoDB but am I updating the index here? Isn't it different when I try to change the value, not the index itself?
You have understood ttl index bit strange way...
In your code example you set every document what have timestamp column "expires_on" to expire right now (expireAfterSeconds: 0).
Expiring process is running background and if you set expireAfterSeconds to 0, it will find out all documents which "expires_on" value is less or equal to now().
So, your update of field "expires_on" must happen at moment where "now() -lt expires_on".
Better say f.ex. "expireAfterSeconds: 3600" and then update expires_on to value "tomorrow minus that 3600 seconds"...
The fact I was creating an index every time a new data was inserted was not logical. I set the index in my database once and only updated the document each time:
database.collection(collectionName).updateOne(query, update, { upsert: true }, (error, result) => {...})
However, the problem turned out to be the value I set for the expires_on in my code. Due to the asynchronicity of Node.js, the value wasn't calculated correctly.
A capture from mongoDb with the structure of a Date time
You can set TTL indexes while creating your records.TTL indexes are special single-field indexes that MongoDB can use to automatically remove documents from a collection after a certain amount of time or at a specific clock time.
To create a TTL index, use the db.collection.createIndex() method with the expireAfterSeconds option on a field whose value is either a date or an array that contains date values.
For example, to create a TTL index that removes the record after 30 days on the createdDate field of the User collection, use the following operation in the mongo shell:
db.User.createIndex( { "createdDate": 1 }, { expireAfterSeconds: 2592000 } )
src: https://docs.mongodb.com/manual/core/index-ttl/
I use Robo Mongo to run my Mongo DB queries. I have a collection which has 'requestReceivedTimestamp' column that holds date time (Example: 12/13/2016 23:18:56 EST). I have used the below commands to setup the TTL Expiry index on this column. For some reason, i don't see the records getting removed. Am i doing something wrong here?
db.logging.createIndex( { "requestReceivedTimestamp": 1 }, { expireAfterSeconds: 2592000 } ) --> Ran this command to create a TTL index on 'requestReceivedTimestamp'
To Enable the MongoDB TTL Monitoring: db.adminCommand({setParameter:1, ttlMonitorEnabled:true}); --> This command to make sure the ttlMonitor is ON. I don't know how to see, if it is ON or OFF, so i had run this command to turn it ON.
This is how data looks like using mongo DB.
How the collection looks like in RoboMongo
The value of requestReceivedTimestamp should be an ISO Date (maybe a timestamp): https://docs.mongodb.com/manual/tutorial/expire-data/
To create an ISO Date:
new Date()
new Date('July 22, 2013 14:00:00')
Thanks Paul Rey. After inserting the column data using new Date(), i was able to the delete the rows after indexing the new column with ttl expiry date. Used the same command - db.logging.createIndex( { "recordCreateDate": 1 }, { expireAfterSeconds: 2592000 } )
Image of the inserted record
In our company we have a retention of eight days of data (with one million of records aprox.) so we have a cronjob that remove documents older than eight days each day. Now we're using the Published field and this field is not indexed.
It takes like 15 minutes to finish to get rid off 100.000 records and we found that this operation is too long.
This is the query where 'docs' is a variable with an array of documents that we don't want to remove. The 'theDate' variable is the date of eight days ago.
records.remove( { "Published" : { $lte : theDate }, "_id" : { $nin : docs } }
Would be it better to use the _id field, which is indexed, in ordered to do this operation? How can we use the _id field in order to do the same operation?
Discard the Cron job entirely: this is a job for TTL indexes. http://docs.mongodb.org/manual/core/index-ttl/
Create a TTL index on the Published field with expireAfterSeconds: 691200 and watch as your documents are automatically removed 8 days after publication.
And if you don't want to indiscriminately delete all documents 8 days after their publication, keep your Cron job and just create a plain index on the Published field.