I am using Elasticsearch along with MongoDB.
Mongo is my primary DB and Elasticsearch is used for search feature.
Mongo changes are synched to Elssticsearch using Mongo Oplog.
I have a case where I need to get a document by passing document id(i.e., Mongo '_id'). Which DB is efficient for this query, Mongo DB or Elasticsearch?
Thanks
If this Id is also your shard key in MongoDb then Mongo would be more efficient as it would know in which shard to look. Elasticsearch will search all of the shards and so be less efficient.
If MongoDb is your single source of truth and you have the Id then use Mongo. If you need full text search then use Elasticsearch.
Related
We have recently moved to mongo java driver core/sync 4.4.0 from 3.12.1 with spring data mongo db from 2.2.5.RELEASE to 3.3.0 also spring boot version 2.6.2 and mongo server version 4.2.5
We are getting exceptions while hitting upsert queries on sharded collection with above mentioned error
There is way to insert shard key in query filter but that is not feasible for us, hence we tried adding #Sharded annotations to our DTO as we have different shard keys for different collections
Still we are getting the above mentioned error also we are unable to get exact meaning of what is full copy of entity meant in below statement
update/upsert operations replacing/upserting a single existing document as long as the given UpdateDefinition holds a full copy of the entity.
Other queries are working fine also upsert queries are working fine on addition of shard key in query filter but that change is not feasible for us we need quick solution
Please help as not able to find any solution on any platform. Thanks in advance!
So here's the deal:
You cannot upsert on Sharded Collection in MongoDB UNLESS you include the shard key in the filter provided for update operation.
To quote MongoDB Docs:
For a db.collection.update() operation that includes upsert: true and is on a sharded collection, you must include the full shard key in the filter:
For an update operation.
For a replace document operation (starting
in MongoDB 4.2).
If you are using MongoDB 4.4+ then you have a workaround as mentioned below:
However, starting in version 4.4, documents in a sharded collection can be missing the shard key fields. To target a document that is missing the shard key, you can use the null equality match in conjunction with another filter condition (such as on the _id field). For example:
{ _id: <value>, <shardkeyfield>: null } // _id of the document missing shard key
Ref: - https://docs.mongodb.com/manual/reference/method/db.collection.update/#behavior
Elasticsearch is a search engine, and MongoDB is a NOSQL-based data store.
I saw a lot of cases using MongoDB and Elasticsearch together.
Usually stored in MongoDB and synchronized it to elasticsearch and analyzed it.
But I do not understand.
Data stored in MongoDB stores Data on elasticsearch anyway.
Why save duplicate Data?
Why need MongoDB?
I am working on processing the mongodb oplog and I create a collection in mongodb to add the processed data and I don't want this collection to again generate oplog.
I want all other collection to generate oplog but need to exclude one of the collection. How can I achieve this. Is there any settings to let mongodb know not to generate oplog for a collection.
Any help is appreciated.
Collection in local database are not part of replication. So, If you create a collection in the local database and insert records to that, oplog entries are not created.
I'm running a MongoDB service and some of the collections are data store only and I don't want let MongoDB loads these collections' data into memory. Is there any configuration for that?
MongoDB does not load the collection into memory, only the collection's indexes.
By default index is _id field only. You can't remove the _id index.
We have an api biuld with mongodb as database. As the data is huge, we indexed all data in elastic search, so how can we convert mongodb query to elasticsearch query? Are there any plugins vailable? Please sugest