For example this is our document schema, which has an descending index on created_at field.
var titles = {
title: String,
created_at:{
type: Date,
index: -1
}
}
Here are 2 questions:
1st:
If we call db.titles.find() or db.titles.findOne() how result order will be?
I mean it will returns objects in desc or asc order?
2nd:
How about here?
db.titles.find().sort({created_at: -1})
How does MongoDB behave in the code above? Sort the result on the fly or it just use index order which we defined in the schema?
1st: If we call db.titles.find() or db.titles.findOne() how result
order will be? I mean it will returns objects in desc or asc order?
It will return documents in natural order - in order in which documents are stored on disk. If you want to sort documents by created_at you should explicitly specify it:
db.titles.find().sort({created_at: -1})
How does MongoDB behave in the code above? Sort the result on the fly
or it just use index order which we defined in the schema?
Since you already have an index on created_at field, Mongo will use it. But if you'll add some query
db.titles.find({title: /Foo/}).sort({created_at: -1})
then Mongo will no longer be able to use your index to sort query results and will be forced to sort it on the fly.
You can find more aboun MongoDB indexes in this blog post.
Related
I am using mongodb 3.2.4
When I execute db.mytable.find().explain() The winning plan is 'Collscan'
But when I execute db.mytable.find().hint(_id:1).explain() The winning plan is 'IXscan'
So here comes a question: since _id is the default index of a table, why mongodb does not use this index to query?
An index can be used when there is a filter criteria or a sort operation - when the fields in the index are used in the filter predicate and/or the sort. In your case, the find method doesn't have a filter criteria or a sort - so no index is used, and you can see that in the query plan as a collection scan. It is as expected. But, when you provide a hint to the find method the query optimizer tries to use the index, and in your case it did (and you see it in the query plan as an IXSCAN). In either case, with or the without the hint, the find has to scan all the documents or keys in the index.
The _id has a default unique index, yes, but unless you are using the _id field in the query filter predicate or in a sort, the query cannot use it (or, specify explicitly to use index with a hint). You can verify with the following queries, db.mytable.find( { _id: 123 } ) or db.mytable.find( { } ).sort( { _id: -1 } ) the query planner will show index scan even though you do not specify the hint.
The main purpose of the indexes is to make your queries run fast; it is about query performance. It has to be a query with filter predicate and/or a sort operation to use an index (and the fields used in the filter or sort must be indexed for performance). With the find method, in your case, without any of the two you are just accessing all the documents as they are in the collection and the index is of no use (and the query optimizer shows that in the plan).
I have a collection product in my DB. Below is one sample document:
{
"sku_id":"12345678",
"priduct_name":"milk",
"product_rank":3,
"product_price": 2.4
}
There are 100k such unique documents in our collection.
I want to query this collection using $in query, as shown below.
db.product.find({"sku_id" :{$in :["12345678","23213"]}}).sort( { product_rank: 1 } )
Our requirement is to search documents based on $in query and sort any field in document(asc or desc).
I have created both forward and reverse index on all fields for this collection.
Note: This sku_id array inside $in query can have 1000+ sku_ids.
My doubt is if I use the filter like $in with an array of sku_id and get the sorted result on any field, will it use the index for sorting or will it sort at query time?
Mongo allows you to find out if a query will use an index. As the find operation returns a cursor you can extend the method chain to include an explain() command which does exactly what you need. (suggest you use db.product.find(...).sort(...).explain('executionStats'))
Will it use the index for sorting or will it sort at query time?
The index created on the product_rank will be used in the query but, not an index on the sku_id alone. Instead create a compound index with both product_rank and sku_id(asc and desc).
https://scalegrid.io/blog/fast-paging-with-mongodb/
Example : {
_id,
name,
company,
state
}
I've gone through the 2 scenarios explained in the above link and it says sorting by object id makes good performance while retrieve and sort the results. Instead of default sorting using object id , I want to index for my own custom field "name" and "company" want to sort and pagination on this two fields (Both fields holds the string value).
I am not sure how we can use gt or lt for a name, currently blocked on how to resolve this to provide pagination when a user sort by name.
How to index and do pagination for two fields?
Answer to your question is
db.Example.createIndex( { name: 1, company: 1 } )
And for pagination explanation the link you have shared on your question is good enough. Ex
db.Example.find({name = "John", country = "Ireland"}). limit(10);
For Sorting
db.Example.find().sort({"name" = 1, "country" = 1}).limit(userPassedLowerLimit).skip(userPassedUpperLimit);
If the user request to fetch 21-30 first documents after sorting on Name then country both in ascending order
db.Example.find().sort({"name" = 1, "country" = 1}).limit(30).skip(20);
For basic understand of Indexing in MonogDB
Indexes support the efficient execution of queries in MongoDB. Without indexes, MongoDB must perform a collection scan, i.e. scan every document in a collection, to select those documents that match the query statement. If an appropriate index exists for a query, MongoDB can use the index to limit the number of documents it must inspect.
Indexes are special data structures, that store a small portion of the collection’s data set in an easy to traverse form. The index stores the value of a specific field or set of fields, ordered by the value of the field.
Default _id Index
MongoDB creates a unique index on the _id field during the creation of a collection. The _id index prevents clients from inserting two documents with the same value for the _id field. You cannot drop this index on the _id field.
Create an Index
Syntax to execute on Mongo Shell
db.collection.createIndex( <key and index type specification>, <options> )
Ex:
db.collection.createIndex( { name: -1 } )
for ascending use 1,for descending use -1
The above rich query only creates an index if an index of the same specification does not already exist.
Index Types
MongoDB provides different index types to support specific types of data and queries. But i would like to mention 2 important types
1. Single Field
In addition to the MongoDB-defined _id index, MongoDB supports the creation of user-defined ascending/descending indexes on a single field of a document.
2. Compound Index
MongoDB also supports user-defined indexes on multiple fields, i.e. compound indexes.
The order of fields listed in a compound index has significance. For instance, if a compound index consists of { name: 1, company: 1 }, the index sorts first by name and then, within each name value, sorts by company.
Source for my understanding and answer and to know more about MongoDB indexing MongoDB Indexing
I am trying to add and retrieve documents from collection.I went through mongodb manual and didn't find ways to add documents in descending order by date. Is it possible to add documents to collection in descending order by date while inserting the document as i don't want to query and return the top 50 documents.
TIA.
MongoDB does not guarantee the retrieval order of documents. Even if you insert them in descending order, there is no guarantee the records will be returned in this order. As suggested by BatScream you could add an descending index on your date field and the order your result when querying the data.
I have a query in mongo such that I want to give preference to the first field and then the second field.
Say I have to query such that
db.col.find({category: A}).sort({updated: -1, rating: -1}).limit(10).explain()
So I created the following index
db.col.ensureIndex({category: 1, rating: -1, updated: -1})
It worked just fined scanning as many objects as needed, i.e. 10.
But now I need to query
db.col.find({category: { $ne: A}}).sort({updated: -1, rating: -1}).limit(10)
So I created the following index:
db.col.ensureIndex({rating: -1, updated: -1})
but this leads to scanning of the whole document and when I create
db.col.ensureIndex({ updated: -1 ,rating: -1})
It scans less number of documents:
I just want to ask to be clear about sorting on multiple fields and what is the order to be preserved when doing so. By reading the MongoDB documents, it's clear that the field on which we need to perform sorting should be the last field. So that is the case I assumed in my $ne query above. Am I doing anything wrong?
The MongoDB query optimizer works by trying different plans to determine which approach works best for a given query. The winning plan for that query pattern is then cached for the next ~1,000 queries or until you do an explain().
To understand which query plans were considered, you should use explain(1), eg:
db.col.find({category:'A'}).sort({updated: -1}).explain(1)
The allPlans detail will show all plans that were compared.
If you run a query which is not very selective (for example, if many records match your criteria of {category: { $ne:'A'}}), it may be faster for MongoDB to find results using a BasicCursor (table scan) rather than matching against an index.
The order of fields in the query generally does not make a difference for the index selection (there are a few exceptions with range queries). The order of fields in a sort does affect the index selection. If your sort() criteria does not match the index order, the result data has to be re-sorted after the index is used (you should see scanAndOrder:true in the explain output if this happens).
It's also worth noting that MongoDB will only use one index per query (with the exception of $ors).
So if you are trying to optimize the query:
db.col.find({category:'A'}).sort({updated: -1, rating: -1})
You will want to include all three fields in the index:
db.col.ensureIndex({category: 1, updated: -1, rating: -1})
FYI, if you want to force a particular query to use an index (generally not needed or recommended), there is a hint() option you can try.
That is true but there are two layers of ordering you have here since you are sorting on a compound index.
As you noticed when the first field of the index matches the first field of sort it worked and the index was seen. However when working the other way around it does not.
As such by your own obersvations the order needed to be preserved is query order of fields from first to last. The mongo analyser can sometimes move around fields to match an index but normally it will just try and match the first field, if it cannot it will skip it.
try this code it will sort data first based on name then keeping the 'name' in key holder it will sort 'filter'
var cursor = db.collection('vc').find({ "name" : { $in: [ /cpu/, /memo/ ] } }, { _id: 0, }).sort( { "name":1 , "filter": 1 } );
Sort and Index Use
MongoDB can obtain the results of a sort operation from an index which
includes the sort fields. MongoDB may use multiple indexes to support
a sort operation if the sort uses the same indexes as the query
predicate. ... Sort operations that use an index often have better
performance than blocking sorts.
db.restaurants.find().sort( { "borough": 1, "_id": 1 } )
more information :
https://docs.mongodb.com/manual/reference/method/cursor.sort/