I have been trying to sort documents in Mongo collection by ObjectId in descending order from Pymongo and use the last added record using this code
record = collection.find({}).sort('_id', -1).limit(1)
However, I am getting 'expected a character buffer object'error. Not sure what is happening here.
Also, if anyone can think of a better way to get the last added record from mongo collection using Pymongo, it will be great help.
for item in collection.find().sort("_id", pymongo.DESCENDING).limit(1):
# do something with your item.
This error indicates to me that your "collection" variable holds a string, not a PyMongo Collection instance. I need to see the code that sets the "collection" variable to know what mistake you're making. But let's start from the top:
import pymongo
client = pymongo.MongoClient()
collection = client.db
cursor = collection.find({}).sort('_id', -1).limit(1)
record = cursor.next()
This will get you a recently-added document, but not always the most recently added. The timestamp portion of an ObjectId is only precise to one second, and since it's generated on the client machine that inserts the document, ObjectId is subject to clock skew.
If and only if the collection is capped, you can reliably get the last document like:
cursor = collection.find({}).sort('$natural', -1).limit(1)
record = cursor.next()
Related
I am using PyMongo to insert data (title, description, phone_number ...) into MongoDB. However, when I use mongo client to view the data, it displays the properties in a strange order. Specifically, phone_number property is displayed first, followed by title and then comes description. Is there some way I can force a particular order?
The above question and answer are quite old. Anyhow, if somebody visits this I feel like I should add:
This answer is completely wrong. Actually in Mongo Documents ARE ordered key-value pairs. However when using pymongo it will use python dicts for documents which indeed are not ordered (as of cpython 3.6 python dicts retain order, however this is considered an implementation detail). But this is a limitation of the pymongo driver.
Be aware, that this limitation actually impacts the usability. If you query the db for a subdocument it will only match if the order of the key-values pairs is correct.
Just try the following code yourself:
from pymongo import MongoClient
db = MongoClient().testdb
col = db.testcol
subdoc = {
'field1': 1,
'field2': 2,
'filed3': 3
}
document = {
'subdoc': subdoc
}
col.insert_one(document)
print(col.find({'subdoc': subdoc}).count())
Each time this code gets executed the 'same' document is added to the collection. Thus, each time we run this code snippet the printed value 'should' increase by one. It does not because find only maches subdocuemnts with the correct ordering but python dicts just insert the subdoc in arbitrary order.
see the following answer how to use ordered dict to overcome this: https://stackoverflow.com/a/30787769/4273834
Original answer (2013):
MongoDB documents are BSON objects, unordered dictionaries of key-value pairs. So, you can't rely on or set a specific fields order. The only thing you can operate is which fields to display and which not to, see docs on find's projection argument.
Also see related questions on SO:
MongoDB field order and document position change after update
Can MongoDB and its drivers preserve the ordering of document elements
Ordering fields from find query with projection
Hope that helps.
I'm currently trying to use pymongo's find_one query. When I run the Mongo Shell and execute a findOne query, it get a document that is returned. However when I try using pymongo's find_one query, I always seem to get just the field names instead of an actual entry.
#app.route("/borough/manhattan/")
def manhattan():
restaurantmanhattan = restaurants.find_one({'borough':'Manhattan'})
json_restaurantmanhattan = []
for restaurant in restaurantmanhattan:
json_restaurantmanhattan.append(restaurant)
json_restaurantmanhattan = json.dumps(json_restaurantmanhattan)
return json_restaurantmanhattan
Once I navigate to http://0.0.0.0:5000/borough/manhattan/ I get the following:
["cuisine","borough","name","restaurant_id","grades","address","_id"]
I believe I should be seeing a document entry that meets the query that it has Manhattan listed in the borough.
I'm at a loss as to how I should be writing the query to return that.
Can anyone explain what I'm seeing?
There are many things wrong with your view.
First as you may already know, find_one return a single document as Python dictionary. So in your for loop, you iterating the dictionary keys.
You really do not need that for loop.
import json
#app.route("/borough/manhattan/")
def manhattan():
restaurant_manhattan = restaurants.find_one({'borough':'Manhattan'})
return json.dumps(restaurant_manhattan)
I am trying to work with mongo shell and I am having challenges in storing the value inside of mongo shell
when I find a user in the document users, i store it in the variable doc
> var doc = db.users.find({"username":"anil"});
When I type doc in the mongo shell i see the json object
> doc
{ "_id" : ObjectId("57325aaa48d9231b11e5cb81"), "username" : "anil", "email" : "mongodb#gmail.com" }
However when I type doc again, i dont see anything. its gone.. what am I doing wrong here ?
> doc
>
It might be something simple but I am not getting it. Can someone point out what I am doing wrong ?
This because find returns a cursor and you can only consume all the value once. Here because you filter your document based on _id value, in the return cursor you only have one document which is consumed the first time you time. If you need to iterate your result many times, you will need to return an array that contains all the documents from the cursor using toArray, but in your case what you need is findOne
when you open a mongo shell it tries to find a mongod server and connects to test by default. when you assign some variable in mongo shell to some document in mongod, it actually fetches the document from the database, to get the same result again you need to connect to the database again (means you need to type var doc = db.users.find({"username":"anil"}); again). Unlike the scenario where you define var doc = 4 in shell and typing doc will return 4 every time.
If you want to stop transmission at the beginning and do some processing then you need to add null after it like
var doc = db.users.find({"username":"anil"});null;
// do your work on doc
an ex.
One other option could be to use the next() function of the find() cursor. This will give the value of the first record in to the variable and it will persist. Then any required operation can be done on the variable.
I have a collection of data and I want to get it sorted by insertion time. I have not any additional fields to store the insert time. But as I found out I can get this time from Id.
I have tried this code:
return bookmarks.find({}, {sort: {_id.getTimestamp(): 1}, limit: 10});
or
return bookmarks.find({}, {sort: {ObjectId(_id).getTimestamp(): 1}, limit: 10});
but get the error message:
=> Your application has errors. Waiting for file change.
Is there any way to sort collection by insertion datetime using only id field ?
At the moment this isn't possible with Meteor, even if it is with MongoDB. The ObjectID's created with meteor don't bear a timestamp. See http://docs.meteor.com/#collection_object_id
The reason for this is client side code can insert code and it can arrive late on the server, hence there is no guarantee the timestamp portion of the ObjectID will be accurate. In addition to the latency the client side's date is used meaning if they're off it's going to get you incorrect data. I think this is the reason they use an ObjectID but it is completely random.
If you want to sort by date you have to store the time/date separately.
The part what i striked out is not accurate. Meteor use it is own id generation which is based on a random string that is while does not apply the doc what i linked before. Check sasha.sochka's comment under.
It is nearly but not 100% good if you just sort for the _id field . While as it is constructed the first 4 byte is the timestamp in secs (so sorting for the getTimestamps value is not better). Under one second resolution you cannot get the exact order, as it is mentioned in the documentation: http://docs.mongodb.org/manual/reference/object-id/#objectid
It is still true that you can try to check the exact order of the insert/update ops against your collection in the oplog, if you have an oplog, but as it is a capped collection anyway you will see the recent operations only. http://docs.mongodb.org/manual/core/replica-set-oplog/.
I am trying to use mongodb's ObjectID to do a range query on the insertion time of a given collection. I can't really find any documentation that this is possible, except for this blog entry: http://mongotips.com/b/a-few-objectid-tricks/ .
I want to fetch all documents created after a given timestamp. Using the nodejs driver, this is what I have:
var timeId = ObjectId.createFromTime(timestamp);
var query = {
localUser: userId,
_id: {$gte: timeId}
};
var cursor = collection.find(query).sort({_id: 1});
I always get the same amount of records (19 in a collection of 27), independent of the timestamp. I noticed that createFromTime only fills the bytes in the objectid related to time, the other ones are left at 0 (like this: 4f6198be0000000000000000).
The reason that I try to use an ObjectID for this, is that I need the timestamp when inserting the document on the mongodb server, not when passing the document to the mongodb driver in node.
Anyone knows how to make this work, or has another idea how to generate and query insertion times that were generated on the mongodb server?
Not sure about nodejs driver in ruby, you can simply apply range queries like this.
jan_id = BSON::ObjectId.from_time(Time.utc(2012, 1, 1))
feb_id = BSON::ObjectId.from_time(Time.utc(2012, 2, 1))
#users.find({'_id' => {'$gte' => jan_id, '$lt' => feb_id}})
make sure
var timeId = ObjectId.createFromTime(timestamp) is creating an ObjectId.
Also try query without localuser