I'm writing a long-polling script to check for new documents in my mongo collection and the only way I know of checking to see whether or not changes have been made in the past iteration of my loop is to do a query getting the last document's ID and parsing out the timestamp in that ID and seeing if it's greater than the timestamp left since I last made a request.
Is there some kind of approach that doesn't involve making a query every time or something that makes that query the fastest?
I was thinking a query like this:
db.chat.find().sort({_id:-1}).limit(1);
But it would be using the PHP driver.
The fastest way will be creating indexes on timestamp field.
Creating index:
db.posts.ensureIndex( { timestamp : 1 } )
Optimizes this query:
db.posts.find().sort( { timestamp : -1 } )
findOne give you only one the last timestamp.
nice to help you.
#Zagorulkin Your suggestion is surely going to help in the required scenario. However i don't think so sort() works with findOne().
Related
I have smallish size database with 2 million records of phone calls.
When I execute:
db.getCollection('calls').find({
'IsIncoming':true, 'DateCreated' : { '$gte':ISODate('2010-12-02T02:26:22.478Z') }, 'CallerIdNum':"2545874578"
}).limit(100).count({})
it is supper fast and it takes 95ms. Note that IsIncoming, DateCreted and CallerIdNum have indexes. Every time I search using those fields it is supper fast
The moment I search for something containing part of a text it is very slow. For example this query now takes 25 seconds:
db.getCollection('calls').find({
'IsIncoming':true, 'DateCreated' : { '$gte': ISODate('2010-12-02T02:26:22.478Z') }, 'CallerIdNum' : /2545874/
}).limit(100).count({})
I know the reason is because I am searching within CallerIdNum. If I where to know the full caller id in advance like on my first query then it will be fast.
Question
I will like the last query to execute faster. I know it is probably impossible, and the only way of getting a great performance is by searching by the whole CallerIdNum. But maybe/hopefully I am wrong and someone can help me find a way of executing my last query faster.
The problem here is that you are searching for a substring of a caller ID number /2545874/. The is not sargable, and generally can't use an index. Assuming you really want numbers which start with that prefix, then use this sargable version:
db.getCollection('calls').find({
'IsIncoming':true, 'DateCreated' : { '$gte': ISODate('2010-12-02T02:26:22.478Z') }, 'CallerIdNum' : /^2545874/
}).limit(100).count({})
You might also want to add a compound index on all three fields, though at least the version of the query I gave above can use an index involving the CallerIdNum field.
community!
I am in a weird situation. The direct equally check returns result, but when using $in I am not getting any records.
db.getCollection("voter").find({"id":{$in:["db1eefc5-09ad-4d4f-a31a-db63d8261913"]}})
db.voter.find({"id":{$in:["db1eefc5-09ad-4d4f-a31a-db63d8261913"]}})
Doesn't return anything.
db.voter.find({id: "db1eefc5-09ad-4d4f-a31a-db63d8261913"})
Returns the desired record.
Being more of a fullstack developer, I don't know what's happening in-depth, but I am sure that both things shall work ideally which is not the case here.
Extra info:
I have defined hashed unique indexes on id.
Thanks.
The problem is pretty simple:
On the first screen you're running your query against admin database
while second query gets executed against crmadmin db
Is there a simple OR elegant method (or query that I can write) to retrieve the last updated timestamp (of the last updated document) in a collection. I can write a query like this to find the last inserted document
db.collection.find().limit(1).sort({$natural:-1})
but I need information about the last updated document (it could be an insert or an update).
I know that one way is to query the oplog collection for the last record from a collection. But it seems like an expensive operation given the fact that oplog could be of very large size (also not trustworthy as it is a capped collection). Is there a better way to do this?
Thanks!
You could get the last insert time same way you mentioned in the question:
db.collection.find().sort({'_id': -1}).limit(1)
But, There isn't any good way to see the last update/delete time. But, If you are using replica sets you could get that from the oplog.
Or, you could add new field in document as 'lastModified'.
You can also checkout collection-hooks. I hope this will help
One way to go about it is to have a field that holds the time of last update. You can name it updatedAt. Every time you make an update to the document, you'll just update the value to the current time. If you use the ISO format to store the time, you'll be able to sort without issues (that's what I use).
The other way is the _id field.
Method 1
db.collection.find().limit(1).sort({updatedAt: -1})
Method 2
db.collection.find().limit(1).sort({_id: -1})
You can try with ,
db.collection.findOne().sort({$natural:-1}).limit(1);
like:
{id:4563214321,updateTime:long("124354354")}
there are always new collections enter the db, so I would like to always get latest updated documents aka the largest update time. how to design the shell script? thanks in advance.
You can use a combination of limit and sort to achieve this goal.
db.collectionName.find({}).sort({"updateTime" : -1}).limit(1)
This will sort all of your fields based on update time and then only return the one largest value.
I would recommend adding an index to this field to improve performance.
This is a repeated question, you can find an answer in this link
Using findOne in mongodb to get element with max id
use like this,
db.collection.find().sort({updateTime:-1}).limit(1).pretty()
as findOne you can do it with this syntax:
db.collection.findOne({$query:{},$orderby:{_updateTime:-1}})
I have a collection of data and I want to get it sorted by insertion time. I have not any additional fields to store the insert time. But as I found out I can get this time from Id.
I have tried this code:
return bookmarks.find({}, {sort: {_id.getTimestamp(): 1}, limit: 10});
or
return bookmarks.find({}, {sort: {ObjectId(_id).getTimestamp(): 1}, limit: 10});
but get the error message:
=> Your application has errors. Waiting for file change.
Is there any way to sort collection by insertion datetime using only id field ?
At the moment this isn't possible with Meteor, even if it is with MongoDB. The ObjectID's created with meteor don't bear a timestamp. See http://docs.meteor.com/#collection_object_id
The reason for this is client side code can insert code and it can arrive late on the server, hence there is no guarantee the timestamp portion of the ObjectID will be accurate. In addition to the latency the client side's date is used meaning if they're off it's going to get you incorrect data. I think this is the reason they use an ObjectID but it is completely random.
If you want to sort by date you have to store the time/date separately.
The part what i striked out is not accurate. Meteor use it is own id generation which is based on a random string that is while does not apply the doc what i linked before. Check sasha.sochka's comment under.
It is nearly but not 100% good if you just sort for the _id field . While as it is constructed the first 4 byte is the timestamp in secs (so sorting for the getTimestamps value is not better). Under one second resolution you cannot get the exact order, as it is mentioned in the documentation: http://docs.mongodb.org/manual/reference/object-id/#objectid
It is still true that you can try to check the exact order of the insert/update ops against your collection in the oplog, if you have an oplog, but as it is a capped collection anyway you will see the recent operations only. http://docs.mongodb.org/manual/core/replica-set-oplog/.