I am using PyMongo to insert data (title, description, phone_number ...) into MongoDB. However, when I use mongo client to view the data, it displays the properties in a strange order. Specifically, phone_number property is displayed first, followed by title and then comes description. Is there some way I can force a particular order?
The above question and answer are quite old. Anyhow, if somebody visits this I feel like I should add:
This answer is completely wrong. Actually in Mongo Documents ARE ordered key-value pairs. However when using pymongo it will use python dicts for documents which indeed are not ordered (as of cpython 3.6 python dicts retain order, however this is considered an implementation detail). But this is a limitation of the pymongo driver.
Be aware, that this limitation actually impacts the usability. If you query the db for a subdocument it will only match if the order of the key-values pairs is correct.
Just try the following code yourself:
from pymongo import MongoClient
db = MongoClient().testdb
col = db.testcol
subdoc = {
'field1': 1,
'field2': 2,
'filed3': 3
}
document = {
'subdoc': subdoc
}
col.insert_one(document)
print(col.find({'subdoc': subdoc}).count())
Each time this code gets executed the 'same' document is added to the collection. Thus, each time we run this code snippet the printed value 'should' increase by one. It does not because find only maches subdocuemnts with the correct ordering but python dicts just insert the subdoc in arbitrary order.
see the following answer how to use ordered dict to overcome this: https://stackoverflow.com/a/30787769/4273834
Original answer (2013):
MongoDB documents are BSON objects, unordered dictionaries of key-value pairs. So, you can't rely on or set a specific fields order. The only thing you can operate is which fields to display and which not to, see docs on find's projection argument.
Also see related questions on SO:
MongoDB field order and document position change after update
Can MongoDB and its drivers preserve the ordering of document elements
Ordering fields from find query with projection
Hope that helps.
Related
I have a dictionary. I need to insert column 2 into mongodb corresponding to column 1(key).
Say this is the dictionary:
values = {'a':['1','2','3'],
'b':['1','2'],
'c':['3','4'] }
Right now I am doing this:
for k,v in values.items():
col4.update({"name":k},{"$set":{"fieldName":v}})
But this takes 3 accesses to the db. Is it possible to do it one go like the way $in works.
In your code you are finding each document by name field and set its fieldName to v. There is no update operation in Mongo that can do such thing in one shot for multiple documents.
However there is a bulk insert statement which can be more efficient than multiple inserts or updates. http://docs.mongodb.org/manual/core/bulk-inserts/.
I thinks I previously didn't quite understand what you were asking and wrote the answer below, but I'm still not sure what you mean by $in. Perhaps you can provide example of data before and after update in DB, that way it will be absolutely clear what you are trying to achieve.
OLD answer ... (I'll edit it soon)
You need to restructure your loop. Build up a query (not running) by adding {field: newValue} to $set clause. After the loop is done you will have an analog of {$set:{"a": 1, "b": 1, "c": 3}}. Then you will update all fields in one shot.
Here is official documentation:
http://docs.mongodb.org/manual/reference/operator/update/set/
What's the easiest way to get all the documents from a collection that are unique based on a single field.
I know I can use db.collections.distrinct to get an array of all the distinct values of a field, but I want to get the first (or really any one) document for every distinct value of one field.
e.g. if the database contained:
{number:1, data:'Test 1'}
{number:1, data:'This is something else'}
{number:2, data:'I'm bad at examples'}
{number:3, data:'I guess there\'s room for one more'}
it would return (based on number being unique:
{number:1, data:'Test 1'}
{number:2, data:'I'm bad at examples'}
{number:3, data:'I guess there\'s room for one more'}
Edit: I should add that the server is running Mongo 2.0.8 so no aggregation and there's more results than group will support.
Update to 2.4 and use aggregation :)
When you really need to stick to the old version of MongoDB due to too much red tape involved, you could use MapReduce.
In MapReduce, the map function transforms each document of the collection into a new document and a distinctive key. The reduce function is used to merge documents with the same distincitve key into one.
Your map function would emit your documents as-is and with the number-field as unique key. It would look like this:
var mapFunction = function(document) {
emit(document.number, document);
}
Your reduce-function receives arrays of documents with the same key, and is supposed to somehow turn them into one document. In this case it would just discard all but the first document with the same key:
var reduceFunction = function(key, documents) {
return documents[0];
}
Unfortunately, MapReduce has some problems. It can't use indexes, so at least two javascript functions are executed for every single document in the collections (it can be limited by pre-excluding some documents with the query-argument to the mapReduce command). When you have a large collection, this can take a while. You also can't fully control how the docments created by MapReduce are formed. They always have two fields, _id with the key and value with the document you returned for the key.
MapReduce is also hard to debug an troubleshoot.
tl;dr: Update to 2.4
What I'm trying to do:
Filter a field of a collection that matches a given condition. Instead of returning every item in the field (which is an array of items), I only want to see matched items.
Similar to
select items from test where items.histPrices=[10,12]
It is also similar to what's found on the mongodb website here: http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields
Here's what I have been trying:
db.test.save({"name":"record", "items":[{"histPrices":[10,12],"name":"stuff"}]})
db.test.save({"name":"record", "items":[{"histPrices":[10,12],"name":"stuff"},
{"histPrices":[12,13],"name":"stuff"},{"histPrices":[11,14],"name":"stuff"}]})
db.test.find({},{"name":1,"items.histPrices":[10, 12]})
It will return all the objects that have a match for items.histPrices:[10,12], including ALL of the items in items[]. But I don't want the ones that don't match the condition.
From the comments left on Mongodb two years ago, the solution to get only the items with that histPrices[10,12] is to do it with javascript code, namely, loop through the result set and filter out the other items.
I wonder if there's a way to do that with just the query.
Your find query is wrong
db.test.find({},{"name":1,"items.histPrices":[10, 12]})
Your condition statement should be in the first part of the find statement.In your query {} means fetch all documents similar to this sql
select items from test (no where clause)
you have to change your mongodb find to
db.test.find({"items.histPrices":[10, 12]},{"name":1})
make it work
since your items is an array and if you wanted to return only the matching sub item, you have to use positional operator
db.test.find({"items.histPrices":[10, 12]},{"name":1,'items.$':1})
When working with arrays Embedded to the Document, the best approach is the one suggested by Chien-Wei Huang.
I would just add another aggregation, with the $group (in cases the document is very long, you may not want to retrieve all its content, only the array elements) Operator.
Now the command would look like:
db.test.aggregate({$match:{name:"record"}},
{$unwind:"$items"},
{$match {"items.histPrices":[10, 12]}},
{$group: {_id: "$_id",items: {$push: "$items"}}});)
If you are interested to return only one element from the array in each collection, then you should use projection instead
The same kind of issue solved here:
MongoDB Retrieve a subset of an array in a collection by specifying two fields which should match
db.test.aggregate({$unwind:"$items"}, {$match:{"items.histPrices":[10, 12]}})
But I don't know whether the performance would be OK. You have to verify it with your data.
The usage of $unwind
If you want add some filter condition like name="record", just add another $march at first, ex:
db.test.aggregate({$match:{name:"record"}}, {$unwind:"$items"}, {$match:{"items.histPrices":[10, 12]}})
https://jira.mongodb.org/browse/SERVER-828
Get particular element from mongoDB array
MongoDB query to retrieve one array value by a value in the array
I have 2 collections on 2 separate DBs. Both store an array field. I plan to query both at once so that:
All collection 1 documents that have elements [A,B] in their array
field and their _ids are present in collection 2's array field with a
specific document _id.
As an example:
docs (collection 1, DB 1):
[{"_id":ObjectId("doc1"), "array1":["A","B"]}, {"_id":ObjectId("doc2"), "array1":["A","C"]}]
user_docs (collection 2, DB 2):
[{"_id":ObjectId("usr1"), "array2": [ObjectId("doc1"),ObjectId("foo")]}, {"_id":ObjectId("usr2"), "array2": [ObjectId("bar"),ObjectId("baz")]}]
I need a query that given A,B and usr1, returns the 'doc1' object (because it has A,B in it's array1 field and usr1 has it in it's array2 field).
I obviously can fetch all docs having A,B in one query and all usr1's docs in another query and find the common elements at application level, but is there any better way of doing it using MongoDB?
Thanks for your help.
Ok im not sure i understand exactly what your trying to do from your description. But i dont understand why you would query data across db's this just seems very heavy handed to me why cant you store both the data sets in the same db. You can always separate later if required? Im not sure this will solve your vague problem but it would be a good place to start.
best of Luck.
You will have to query MongoDB twice, since you have no possibility of a join. You will have to do it on application level. If you can denormalize, do it. Cash the needed data in a embedded doc, so that you can do one query only.
I think #Eamonn is right, that you shouldn't have to do a query across DBs.
I have a number of different documents in a mongo collection.
The attrs are all numeric values. I don't know apriori what the fieldnames are (I do but they can vary from doc to doc).
I want to write a program that
a) gets all the unique fieldnames in a collection
b) finds the max and min value of each field in the collection
and then reports it in a tabular form with rows "fieldname, maxvalue, minvalue" or in JSON that is equivalent. I am using pymongo but I don't have to, ruby or js or even java driver is fine.
How do I get programmatic access to the list of unique fieldnames in a collection? That's
the major question. I can manage the rest.
Either you main the list of used key inside your application as part of your application logic in some document inside the same collection or a meta-collection yourself or you have to iterate over all documents to figure out the list of keys...there is nothing in MongoDB helping you here since MongoDB is schemaless.