MongoDB 3.2 is now providing a filter expression to partially index the collection.
Based on that feature, I wonder how MongoDB could help me for the following case.
I do have many pre defined queries very near the filter expression. The principle will be to create many filter expression index.
The index will in fact keep the ids of the maching document and will be updated on each document changes.
For performance reason, I prefer to use the index engine from MongoDB that trying to use an external tools with Trigger solution.
How could I accomplish such feature by extending MongoDB. Any others noSQL could help ?
Thanks
Related
I'm querying a collection with aggregate function in MongoDB and I have to look up some other collections in its aggregation. But I have a question about it:
Does MongoDB use indexes for foreignField? I wasn't able to figure this out and I searched
everywhere for this but I didn't get my answer. It must certainly use indexes for it but I just want to be sure.
The best way to determine how the database is executing a query is to generate and examine the explain output for the operation. With aggregations that include the $lookup stage specifically you will want to use the more verbose .explain("executionStats") mode. You may also utilize the $indexStats operator to confirm that the usage count of the intended index is increasing.
The best answer we can give based on the limited information in the question is: MongoDB will probably use the index. Query execution behavior, including index usage, depends on the situation and the version. If you provide more information in your question, then we can provide more specific information. There is also some details about index usage on the $lookup documentation page.
I Want to query using part of id to get all the matched documents. So I tried “starts with” and "contains" which works find but is there any performance issue for large collection?
The best way to make this search optimum :
Add $text index on the fields you want to do search in. This is really important because internally it tokenize your string to that you could search for a part of it.
Use regex which is also quicker to do.
If you are using aggregate, read this mongodb official doc about aggregation optimization which might help you to implement this in efficient manner : https://docs.mongodb.com/manual/core/aggregation-pipeline-optimization/
Last but not the least, if you are not yet fully inclined towards mongodb and project is fresh, look out for elasticsearch service which is based on Lucene. Its extremely powerful doing these kinds of searches.
I want to know as we have index creation feature in mognodb to speed up the query process https://docs.mongodb.org/v3.0/indexes/ what do we have for elasticsearch for this purpose? I googled it but I was unable to find any suitable information, I used indexing in mongodb on most frequently used fields to speed up the query process and now I want to do same in elasticsearch i want to know is there anything that elasticsearch provides .Thanks
Elasticsearch also has indices: https://www.elastic.co/blog/what-is-an-elasticsearch-index
They are also used as part of the database's key features to provide swift search capabilities.
It is annoying that "index" is used in a different context with ES and many other databases. I'm not as familiar with MongoDB so I'll resort to their documentation at v3.0/core/index-types.
Basically Elasticsearch was designed to serve efficient "filtering" (yes/no queries) and "scoring" (relevance ranking via tf-idf etc.), and it uses Lucene as the underlying inverted index.
MongoDB concepts and their ES counter-parts:
Single Field Index: trivially supported, perhaps as not_analyzed fields for exact matching
Compound Index: Lucene applies AND filter condition via efficient bitmaps, can ad-hoc merge any "single field" indexes
Multikey Index: Transparent support, no difference values and an array of values
Geospatial Index: directly supported via geo-shapes
Text Index: In some way ES was optimized for this use-case as analyzed field type
In my view at search applications relevance is more important that plain filtering the results, as some words occur at almost every document and thus are less relevant when searching.
Elasticsearch has other very useful concepts as well such as aggregations, nested documents and child/parent relationships.
I 'm familiar with mongodb.
you know, there are many index types in mongodb, such as:
multikey index : http://docs.mongodb.org/manual/core/index-multikey/
, which is very useful for keyword search, I ever used it to build a simple search engine.
compound index is also very useful in mongodb : http://docs.mongodb.org/manual/tutorial/create-a-compound-index/ which is used for multi fields' query.
but I need to migrate my database from mongodb to hbase, do you know some similar index in hbase which can realize the same function with multikey and compound index in mongodb?
HBase doesn't support secondary indexes, that's one of the trade-offs in order to be able to scale to massive data sets. These are the options you have:
http://hbase.apache.org/book/secondary.indexes.html
It all depends on the amount of data you're going to handle and your access patterns. For me, both dual writing to "index" tables & summary tables are the best approaches, just keep in mind that this has to be done manually.
There is no concept of indexing in HBase as of now. I know there is some demand within the community for Indexing. But there are other projects which provide indexing on top of Hbase, One particular one i looked at was Huawei Hindex
Does MongoDB supports comparing two fields in same collection by using native operators (not $where and JavaScript)?
I already looked at similar questions and all answers used $where / JavaScript.
MongoDB documentation clearly states that:
JavaScript executes more slowly than the native operators listed on this page, but is very flexible.
My primary concern is speed and I would like to use indexes if possible. So is comparing two fields in MongoDB possible without using JavaScript?
This is not currently possible, but it will be possible through the new aggregation framework currently under development (2.1+). This aggregation framework is native and does not rely on relatively slow JavaScript execution paths.
For more details check http://www.mongodb.org/display/DOCS/Aggregation+Framework
and the progress at https://jira.mongodb.org/browse/SERVER-447
From reading the documentation you link it doesn't look like MongoDB has the ability to compare two document properties using only native operators.
Perhaps you can modify the documents themselves (and/or the code which saves the documents) to include a boolean property with value resulting from the comparison (ahead-of-time) and then simply query on that new property as needed. You could even index it for even better performance.