I want to design a schema in MongoDB. In this, there are many fields ~6 fields and fields in subdocuments ~3 which I can query. Is there any way to make it queryable quicker then seq scan? I could have used index but the fields query can come in any order and also I don't want to make compound index for every combination.
Is there any way I can design this kind of schema?
Related
I would like to understand which of the below queries would be faster, while doing updates, in mongo db? I want to update few thousands of records at one stretch.
Accumulating the object ids of those records and firing them using $in or using bulk update?
Using one or two fields in the collection which are common for those few thousand records - akin to "where" in sql and firing an update using those fields. These fields might or might not be indexed.
I know that query will be much smaller in the 2nd case as every single "_id" (oid) is not accumulated. Does accumulating _ids and using those to update documents offer any practical performance advantages?
Does accumulating _ids and using those to update documents offer any practical performance advantages?
Yes because MongoDB will certainly use the _id index (idhack).
In the second method - as you observed - you can't tell whether or not an index will be used for a certain field.
So the answer will be: it depends.
If your collection has million of documents or more, and / or the number of search fields is quite large you should prefer the first search method. Especially if the id list size is not small and / or the id values are adjacent.
If your collection is pretty small and you can tolerate a full scan you may prefer the second approach.
In any case, you should testify both methods using explain().
Since MongoDB 3.x introduces lock per record and not on collection or database, does it make sense to write all of your data to single collection with one extra identifier field "documentType".
It will help simulate "join" through map-reduce operation.
Couchbase does the same thing with "buckets" instead of collection.
Does anybody see any disadvatanges with this approach ?
There's one big general-case disadvantage: indexes.
With Mongo, you generally want to set up indexes so that most, if not all, queries you make, use them. So in addition to the one on _id, you'll set up indexes on the primary fields you search by (often compounded with those you sort by).
If you're storing everything in one single collection, that means you need to have all those indexes on that collection. Which means two things:
The indexes are be bigger, since there's more documents to index. Granted, this can be somewhat mitigated by using sparse indexes.
Inserting or modifying documents in the collection requires Mongo to update all these indexes (where it'd just update the relevant indexes in the standard use-many-collections approach). This kills your write performance.
Furthermore, if you have in your application a query that somehow doesn't use one of those many indexes, it needs to scan through the entire collection, which is O(n) where n is the number of documents in the collection -- in your case, that means the number of documents in the entire database.
Collections are cheap. Use them ;)
I am new to MongoDB. I have read that Indexes limit the documents to be scanned when we query for some data.
Reference:-http://docs.mongodb.org/manual/core/indexes-introduction
I am confusing it with the Query Criteria as it limits the data.
For example:- db.users.find({score:{"$lt":30}}).
In the manual this example is given and explained in the sense of indexes. What are indexes and how are they different than Query Criteria?
Thank you
Indexes in MongoDB are similar, but not identical to indexes in relational databases. So, to get a basic feel, you can think of the two. Query criteria defines what subset of documents your queries are interested in. An index may be able to USE the query criteria to answer the query faster.
Suppose you have a collection with no indexes, and you do db.users.find({score:{$lt:30}}). With no index, you will need to scan the entire collection to answer the query, processing all documents regardless of their value. With an index on 'score', the query will be able to use the index to drill down on ONLY the documents that match your query, thereby executing faster.
Query criteria limits the data that is send to the client from server but it has to scan each and every document for the matching. On the other hand Index limits the scanning of the documents by having special data structure(B-tree in mongodb).
Ref:-http://docs.mongodb.org/manual/core/indexes-introduction
Is there a way to specify the natural ordering of data in mongodb, similar to how a primary index would order data in a RDBMS table?
My use case is that all my queries return data sorted by a date field, say birthday. According to MongoDB: Sorting and Natural Order, the natural order for a standard (non-capped) collection is roughly the insertion order, but not guaranteed. This would imply sorting is needed after the data is retrieved.
I believe what you are referring to is a clustered index, not a primary index.
MongoDB 2.0 does not have a clustered index feature, so a regular index on date would be your most efficient option to retrieve.
It's probably premature optimization to think about the physical order on disk with MongoDB. MongoDB uses memory-mapped files, so depending on your working set + queries + RAM you may not need to load data from disk as often as expected.
If you are looking for something to act like the primary index in a RDBMS then sort by _id. It will be roughly insert order since the _id is prefixed with timestamp. If you try to use $natural order it will cause it to miss indexes.
Also, I would add that you should look into using the built-in timestamps in the document IDs instead of relying on a separate date field, as it allows you to store less data and removes an index.
Jason
MongoHQ
I guess it would be difficult to achieve what you want without the help of indexes. To support sharding, the _id field in MongoDB takes values based on the timestamp at the moment the document is created. As a consequence, you can't have them monotonically increasing unlike the identity column in RDBMS table..I think you must create an index on Birthdate column if all your queries return documents sorted in the order of Birthdate. Once the index is created, the queries become efficient enough..
Refer this:
MongoDB capped collection and monotically increasing index
I'm logging different actions users make on our website. Each action can be of different type : a comment, a search query, a page view, a vote etc... Each of these types has its own schema and common infos. For instance :
comment : {"_id":(mongoId), "type":"comment", "date":4/7/2012,
"user":"Franck", "text":"This is a sample comment"}
search : {"_id":(mongoId), "type":"search", "date":4/6/2012,
"user":"Franck", "query":"mongodb"} etc...
Basically, in OOP or RDBMS, I would design an Action class / table and a set of inherited classes / tables (Comment, Search, Vote).
As MongoDb is schema less, I'm inclined to set up a unique collection ("Actions") where I would store these objects instead of multiple collections (collection Actions + collection Comments with a link key to its parent Action etc...).
My question is : what about performance / response time if I try to search by specific columns ?
As I understand indexing best practices, if I want "every users searching for mongodb", I would index columns "type" + "query". But it will not concern the whole set of data, only those of type "search".
Will MongoDb engine scan the whole table or merely focus on data having this specific schema ?
If you create sparse indexes mongo will ignore any rows that don't have the key. Though there is the specific limitation of sparse indexes that they can only index one field.
However, if you are only going to query using common fields there's absolutely no reason not to use a single collection.
I.e. if an index on user+type (or date+user+type) will satisfy all your querying needs - there's no reason to create multiple collections
Tip: use date objects for dates, use object ids not names where appropriate.
Here is some useful information from MongoDB's Best Practices
Store all data for a record in a single document.
MongoDB provides atomic operations at the document level. When data
for a record is stored in a single document the entire record can be
retrieved in a single seek operation, which is very efficient. In some
cases it may not be practical to store all data in a single document,
or it may negatively impact other operations. Make the trade-offs that
are best for your application.
Avoid Large Documents.
The maximum size for documents in MongoDB is 16MB. In practice most
documents are a few kilobytes or less. Consider documents more like
rows in a table than the tables themselves. Rather than maintaining
lists of records in a single document, instead make each record a
document. For large media documents, such as video, consider using
GridFS, a convention implemented by all the drivers that stores the
binary data across many smaller documents.