I have applied index on embedded document field which is of type date. Due to unpredictable behavior of MongoDB query with $lt & $gt operators I am trying to use min() max() functions of Cursor which force both lower & upper bounds on the index. But when I used it, it gave me error:
planner returned error: unable to find relevant index for max/min query"
Query looks like:
db.user.find().min({'record.date':ISODate("2014-12-01")}).max({'record.date':ISODate("2014-12-01")}).explain()
I have indexed the field 'record.date'. I even tried to force it with hint() but of no use. It showed me this error:
planner returned error: hint provided does not work with min query"
Somewhere on the Internet I saw on one of the forums that we can query with embedded fields in min(), max() functions. Field 'record' is an array of sub-documents. I have another field of type date in main document for which min(), max() worked fine (this field is indexed too). Can anyone guess why is this happening?
sample document:
user:
{name:'ad',dob: ISODate('yyyy-mm-dd'),createdAt: ISODate(),...,record:[
{date:ISODate:(),...
},...]}
Index was created as:
db.user.ensureIndex({'record.date':1})
Thanks.
Related
I have posts collection and I want to take all the posts that its "start" field is greater that current time and user_id not equal to "1".
Query query = firebaseFirestore
.collection("posts")
.whereNotEqualTo("user_id", "1")
.whereGreaterThan("start", Timestamp.now());
I added index like on a screenshot, but still I get an error that I can’t use double condition.
What is wrong?index screenshot
An index won't help you here. The problem is that Firestore doesn't support the query you're trying to perform. Please review the documentation on query limitations to better understand:
In a compound query, range (<, <=, >, >=) and not equals (!=, not-in) comparisons must all filter on the same field.
Your query is using a range filter and an inequality filter on different fields, which is not supported.
I have multiple environments whith MongoDB on them that stores the same types of data (same types of collections, different documents according to the environment)
I run in MongoDB the following query:
db.incident.count({ $and: [{"tags.display_name": "Policy Violation"},{start_time: {$gte: ISODate("2020-07-11T09:30:04.887Z")}}]})
and I get a number as expected (for example: 279)
But on some of my environment, when I run this query:
db.incident.count({"start_time": {$gte: ISODate("2020-07-11T09:30:04.887Z")}})
I get a lower number (for example from the same environment from the example above : 274) which is an impossible result (as you can see the first query is a subquery of the second)
I read some documents and found:
Avoid using the db.collection.count() method without a query predicate since without the query predicate, the method returns results based on the collection’s metadata, which may result in an approximate count. In particular, on a sharded cluster, the resulting count will not correctly filter out orphaned documents.
After an unclean shutdown, the count may be incorrect
but I couldn't find anywhere the definition (or any example, etc') for 'query predicate'
Can someone please help? how can I get an exact result?
So, I read the following definition of indexes from [MongoDB Docs][1].
Indexes support the efficient execution of queries in MongoDB. Without indexes, MongoDB must perform a collection scan, i.e. scan every document in a collection, to select those documents that match the query statement. If an appropriate index exists for a query, MongoDB can use the index to limit the number of documents it must inspect.
Indexes are special data structures that store a small portion of the
collection’s data set in an easy to traverse form. The index stores
the value of a specific field or set of fields, ordered by the value
of the field. The ordering of the index entries supports efficient
equality matches and range-based query operations. In addition,
MongoDB can return sorted results by using the ordering in the index.
I have a sample database with a collection called pets. Pets have the following structure.
{
"_id": ObjectId(123abc123abc)
"name": "My pet's name"
}
I created an index on the name field using the following code.
db.pets.createIndex({"name":1})
What I expect is that the documents in the collection, pets, will be indexed in ascending order based on the name field during queries. The result of this index can potentially reduce the overall query time, especially if a query is strategically structured with available indices in mind. Under that assumption, the following query should return all pets sorted by name in ascending order, but it doesn't.
db.pets.find({},{"_id":0})
Instead, it returns the pets in the order that they were inserted. My conclusion is that I lack a fundamental understanding of how indices work. Can someone please help me to understand?
Yes, it is misunderstanding about how indexes work.
Indexes don't change the output of a query but the way query is processed by the database engine. So db.pets.find({},{"_id":0}) will always return the documents in natural order irrespective of whether there is an index or not.
Indexes will be used only when you make use of them in your query. Thus,
db.pets.find({name : "My pet's name"},{"_id":0}) and db.pets.find({}, {_id : 0}).sort({name : 1}) will use the {name : 1} index.
You should run explain on your queries to check if indexes are being used or not.
You may want to refer the documentation on how indexes work.
https://docs.mongodb.com/manual/indexes/
https://docs.mongodb.com/manual/tutorial/sort-results-with-indexes/
After having read the official documentations on indexes, sort, intersection, i'm a little bit confuse on how everything work together.
I've trouble making my query use the indexes i've created. I work on a mongodb 3.0.3, on a collection having ~4millions of document.
To simplify, let's say my document is composed of 6 fields:
{
a:<text>,
b:<boolean>,
c:<text>,
d:<boolean>,
e:<date>,
f:<date>
}
The query I want to achieve is the following :
db.mycoll.find({ a:"OK", b:true, c:"ProviderA", d:true, e:{ $gte:ISODate("2016-10-28T12:00:01Z"),$lt:ISODate("2016-10-28T12:00:02") } }).sort({f:1});
So intuitively I've created two indexes
db.mycoll.createIndex({a: 1, b: 1, c: 1, d:1, e:1 }, {background: true,name: "test1"})
db.mycoll.createIndex({f:1}, {background: true,name: "test2"})
But the explain() give me that the first index is not used at all.
I known there is some kind of limitation when there is ranges in play in the filter (in the e field), but I can't find my way around it.
Also instead of having a single index on f, I try a compound index on {e:1,f:1} but it didn't change anything.
So What I have misunderstood?
Thanks for your support.
Update: also I find some time the following predicate for mongodb 2.6 :
A good rule of thumb for queries with sort is to order the indexed fields in this order:
First, the field(s) on which you will query for exact values.
Second, the field(s) on which you will sort.
Finally, field(s) on which you will query for a range of values (e.g., $gt, $lt, $in)
An example of using this rule of thumb is in the section on “Sorting the results of a complex query on a range of values” below, including a link to further reading.
Does this also apply for 3.X version?
Update 2: following above predicate, I created the following index
db.mycoll.createIndex({a: 1, b: 1, c: 1, d:1 , f:1, e:1}, {background: true,name: "test1"})
And for the same query :
db.mycoll.find({ a:"OK", b:true, c:"ProviderA", d:true, e:{ $gte:ISODate("2016-10-28T12:00:01Z"),$lt:ISODate("2016-10-28T12:00:02") } }).sort({f:1});
the index is indeed used. However too much keys seems to be scan, I may need to find a better order the fields in the query/index.
Mongo acts sometimes a bit strange when it comes to the index selection.
Mongo automagically decides what index to use. The smaller an index is the more likely it is used (especially indexes with only one field) - this is my experience. May be this happens because it is more often already loaded in RAM? To find out what index to use when Mongo performs test queries when it is idle. However the result is sometimes unexpected.
Therefore if you know what index to use you can force a query to use a specific index using the $hint option. You should try that.
Your two indexes used in the query and the sort does not overlap so MongoDB can not use them for index intersection:
Index intersection does not apply when the sort() operation requires an index completely separate from the query predicate.
What is the difference among MongoDB's aggregation $min versus query modifier $min versus find().sort() that returns the minimum of the_field with: db.the_collection.find().sort({the_field:1}).limit(1)?
Which one should I use if there are about a few hundred calls per minute to retrieve the minimum element in a collection and work with it independently each time?
Side question: Can someone show me the correct syntax using either $min to give me the minimum of a field in a collection? db.the_collection.find().min({the_field:10}) doesn't work.
To get the document for lowest value of 'the_field' you should use
db.the_collection.find().sort({the_field:1}).limit(1)
So over here we are Sorting the document first and then taking the first one out of it as you can see from the query.
Aggregation $min :
It is used when we are grouping the document into a single document and we want to keep the value of the this single document as minimum of all the documents from where it was grouped.
$min operator :
It is used for inclusive lower bound for a specific index in order to constrain the results of find(). In simple words, if the_field is indexed and we want to keep some constraint on the find() then we can use it. It is generally used for improving the performance.
The syntax which you were entering was correct but it requires an Indexed field and the result will be different from what you actually want.
Because min() requires an index on a field, and forces the query to use this index even though a better index is available to be picked up,
So you must prefer the $gte or the sort operations for the query if possible.
db.the_collection.find().sort({the_field:1}).limit(1)
So the above query is better is to use as compare the $min operation.
Refer the below mentioned link.
http://docs.mongodb.org/manual/reference/operator/meta/min/#interaction-with-index-selection
While running the min operator on a no index key, It will give the following error
Error: error: {
"$err" : "Unable to execute query: error processing query: ns=db.dbName limit=0 skip=0\nTree: status == \"approved\"\nSort: {}\nProj: {}\n planner returned error: unable to find relevant index for max/min query",
"code" : 17007
}