I have a big project and I want to use firebase storage and full Text search like algolia or elastic search search but want to use mongoDb due to price and some more issue so how can my firebase database send data to mongoDb and How can I pass query to mongoDb
Please use the firebase cloud function to save data on MongoDB on data insert and update in firebase.then you can easily query on mongodb for full text search.
you can not perform a search on Firestore or Realtime Database, it's totally fine to have searchable data duplicated in a different DB. Algolia and Mongo Atlas search are good solutions to use. You need to keep in sync a firestore collection with whatever solution you want to use for search, just create a function the triggers onWrite and insert/update the record.
"functions.store" has ready to use function the keep collection sync with Algolia
Related
I'm new to mongodb, and therefore it is still hard for me to create queries on my own. I found mongodb compass helpful to modify my documents, but would like to have queries in order to apply changes quicker to the production database.
Is there a way to modify a document visually using mongodb compass and then to get the query that mongodb compass uses behind the curtain?
If not, is there some different GUI application that could help me to create queries?
I and my colleagues will migrate from firestore to MongoDB and we need to find a suitable way or a tool to analyze the firebase schema and know exactly the model for each collection and each sub-collection inside a collection in a nested level.
I tried to write a script but it was tedious due to the level of sub-collection we have.
I was able to get collection and sub-collections but I found it is hard to loop on each document and get its schema.
So any suggestions?
I've a list of items in my ElasticSearch. User enters a query and I fetch the results from elastic search. Now, I've some user preferences stored in mongodb based on which I want to filter the results of elastic search.
Suppose, I get a list of items(item_ids) from Elasticsearch.
Mongo DB has following schema.
id, user_id, item_id
I choose this MongoDB schema because a user could have a very big list of items(in order of Millions), which he doesn't want to see in results.
How do I achieve this with scale? Do I need to change my schema?
You should use elasticsearch filtering for this, you can include the filter criteria in your ES query which would reduce the number of results to return without which
You have to return huge data set from ES and then do the filtering in MongoDB which is two step process and costly at both ES and mongo side.
With filters at ES, it would return less data which would avoid extra post-processing at mongoDB and filters are executed first and by default cached at elasticsearch side so you don't need further caching solution like redis etc.
Refer filter and query context and from same official doc, info about filter cache.
Frequently used filters will be cached automatically by Elasticsearch,
to speed up performance.
In my project I have to implement text search and and have to choose a feasible
approach among the two that are :-
Synchronising MongoDB database with ElasticSearch.
MongoDB's own text indexes that has Elastic Search like text searching
capabilities.
I have gone through many articles that provide the pros of each of the cases but haven't found any relevant document that provides comparison between the two approaches and which approach is better than the other or what are the limitation for a specific approach.
Note:- I am using Node.js with express.js.
MongoDB is a general purpose database, Elasticsearch is a distributed text search engine backed by Lucene. You can store data in MongoDB and use Elasticsearch exclusively for its' full-text search capabilities. According to your use case, you can only send a subset of the mongo data fields to elastic.
Synchronization of mongodb with Elasticsearch can be done with mongoosastic. You can solve data safety concern by persisting in mongo and speed up your search by using elasticsearch. Also can use python script using elasticsearch.py package to sync mongo and elasticsearch.
Mongodb search is very slow as compared to elasticsearch. Also indexing in mongodb takes up more time and more resources.
It is not recommended to use ElasticSearch as the only storage from some obvious reasons like security, transactions etc. So how it is usually used together with other database?
Say, I want to store some documents in MongoDB and be able to effectively search by some of their properties. What I'd do would be to store full document in Mongo as usual and then trigger insertion to ElasticSearch but I'd insert only searchable properties plus MongoDB ObjectID there. Then I can search using ElasticSearch and having ObjectID found, go to Mongo and fetch whole documents.
Is this correct usage of ElasticSearch? I don't want to duplicate whole data as I have them already in Mongo.
The best practice is for now to duplicate documents in ES.
The cool thing here is that when you search, you don't have to return to your database to fetch content as ES provide it in only one single call.
You have everything with ES Search Response to display results to your user.
My 2 cents.
You may like to use mongodb river take a look at this post
There are more issue then the size of the data you store or index, you might like to have MongoDB as a backup with "near real time" query for inserted data. and as a queue for the data to indexed (you may like to use mongodb as cluster with the relevant write concern suited for you application