I use Robo Mongo to run my Mongo DB queries. I have a collection which has 'requestReceivedTimestamp' column that holds date time (Example: 12/13/2016 23:18:56 EST). I have used the below commands to setup the TTL Expiry index on this column. For some reason, i don't see the records getting removed. Am i doing something wrong here?
db.logging.createIndex( { "requestReceivedTimestamp": 1 }, { expireAfterSeconds: 2592000 } ) --> Ran this command to create a TTL index on 'requestReceivedTimestamp'
To Enable the MongoDB TTL Monitoring: db.adminCommand({setParameter:1, ttlMonitorEnabled:true}); --> This command to make sure the ttlMonitor is ON. I don't know how to see, if it is ON or OFF, so i had run this command to turn it ON.
This is how data looks like using mongo DB.
How the collection looks like in RoboMongo
The value of requestReceivedTimestamp should be an ISO Date (maybe a timestamp): https://docs.mongodb.com/manual/tutorial/expire-data/
To create an ISO Date:
new Date()
new Date('July 22, 2013 14:00:00')
Thanks Paul Rey. After inserting the column data using new Date(), i was able to the delete the rows after indexing the new column with ttl expiry date. Used the same command - db.logging.createIndex( { "recordCreateDate": 1 }, { expireAfterSeconds: 2592000 } )
Image of the inserted record
Related
I'm trying to update the value of the expiry date in a document. Here's the Node.js code I use to set the index everytime I want to update the document:
database.collection(collectionName).createIndex({ "expires_on": 1 }, { expireAfterSeconds: 0 })
database.collection(collectionName).updateOne(query, update, { upsert: true }, (error, result) => {...})
The thing is that I set the expires_on date to the date of tomorrow and the value is set correctly in database but it expires within a few seconds. Is there a problem with the way I'm updating this field? I don't know how I can update this document in a way that it would expire on the last expires_on date it has been updated for.
I have noticed that you can't update the indexes in MongoDB but am I updating the index here? Isn't it different when I try to change the value, not the index itself?
You have understood ttl index bit strange way...
In your code example you set every document what have timestamp column "expires_on" to expire right now (expireAfterSeconds: 0).
Expiring process is running background and if you set expireAfterSeconds to 0, it will find out all documents which "expires_on" value is less or equal to now().
So, your update of field "expires_on" must happen at moment where "now() -lt expires_on".
Better say f.ex. "expireAfterSeconds: 3600" and then update expires_on to value "tomorrow minus that 3600 seconds"...
The fact I was creating an index every time a new data was inserted was not logical. I set the index in my database once and only updated the document each time:
database.collection(collectionName).updateOne(query, update, { upsert: true }, (error, result) => {...})
However, the problem turned out to be the value I set for the expires_on in my code. Due to the asynchronicity of Node.js, the value wasn't calculated correctly.
A capture from mongoDb with the structure of a Date time
You can set TTL indexes while creating your records.TTL indexes are special single-field indexes that MongoDB can use to automatically remove documents from a collection after a certain amount of time or at a specific clock time.
To create a TTL index, use the db.collection.createIndex() method with the expireAfterSeconds option on a field whose value is either a date or an array that contains date values.
For example, to create a TTL index that removes the record after 30 days on the createdDate field of the User collection, use the following operation in the mongo shell:
db.User.createIndex( { "createdDate": 1 }, { expireAfterSeconds: 2592000 } )
src: https://docs.mongodb.com/manual/core/index-ttl/
I have created one collection in mongoDB for which i have created index for field "expireAt" which is a date time and set expireAfterSeconds property to 0.
As i want to set TTL at document level not at collection level.
As i know in redis i can get the remaining TTL for the particular key
Just have question is that possible in mongoDB ? where i can get the remaining time to expire for the given document.
I have searched it on google but didn't find any relevant information.
Thank You
If you set the TTL expireAfterSeconds value to 0 for per-document expiry, the remaining time will be the difference between the current time and the TTL date field (eg. expireAt) in your documents. There isn't a specific server function to query this, but you can either calculate this in your app or use MongoDB's aggregation framework.
Example using the mongo shell to return the time left (in milliseconds) before documents are eligible to be removed in the next TTL pass:
db.mycoll.aggregate(
{ $project: {
expireAt: 1,
ttlMillis: {
$subtract: [ "$expireAt", new Date() ]
}
}}
)
Note: The TTL background thread runs every 60 seconds, so documents may persist for a minute or more past their nominal expiry.
try this
db.hellos.createIndex( { "expireAt": 1 }, { expireAfterSeconds: 3600 } )
read more here
In a MongoDB table multiple records will be inserted at a time.
For example below records will the same timestamp
{ "_id" : ObjectId("567a68517507b377a0a20902"), value: "bar", time: "2012" }
{ "_id" : ObjectId("567a68517507b377a0a20903"), value: "baz", time: "2011" }
It's possible to get the timestamp of individual document.
ObjectId("567a68517507b377a0a20903").getTimestamp()
ISODate("2015-12-23T09:24:33Z")
Is it possible to have a field where it automatically generates the last updated or created timestamp to each and every document upon insert?
Similar to CURRENT_TIMESTAMP in MYSQL
Note: Data will be inserted from Apache Spark Streaming
You can do it using $currentDate. But it works only in update query.
Depending on your language, yes. In MongoShell, you can use new ISODate()
Anyone have a handy mongo command to remove all entries from a DB that are older than X date/ X days?
Basically have a dev and production DB, I'm looking to prune the dev DB out a bit to limit size.
Thanks for the help!
You could do something like this from the mongo shell.
var older=Date.parse("2013-03-01"),collection=db.so,all=collection.find();
all.forEach(function(doc) { var ts = doc._id.getTimestamp();
if (ts < older) { collection.remove(doc); } });
The above line (which you'd paste into the shell) will delete all documents in the specified collection (collection=db.so) created before the first of March, 2013. It relies on the fact that each ObjectId has an embedded timestamp (based on the timestamp of document creation (docs)), which can be retrieved and used.
You could of course change the query to look for a specific timestamp field in a document.
if (doc.timestampField < older) { collection.remove(doc); } })
You can use the mongo concept of deleting data after some specified amount of time, Expire Data from Collections by Setting TTL.
Please refer below link to do so
http://docs.mongodb.org/manual/tutorial/expire-data/