I'm using Couchbase to power the back end of my mobile app and am experiencing a strange error when using views.
I have a view set up to fetch a specific document type and am querying that view via the Sync-Gateway admin API. Normally it works well but I've found that if a document has been recently added to the database then the view query will return 0 results on the first request. The second identical request will then return the expected response.
I suspect that the new document hasn't been indexed by couchbase yet and the query triggers a re-indexing of documents. What I'm wondering is if there's a way of notifying couchbase that I'm about to query the view and to prepare the documents in advance. I don't want to have to perform 2 requests for each query.
Has anyone else come across this issue?
Any solutions?
By default, Sync Gateway allows using a "stale" index, meaning a query won't necessarily rebuild the index before processing a query.
To override this, add stale=false to your query.
(Allowed options are false, ok, and update_after. The default is update_after.)
Related
I am new to MongoDB. And I have the following issue on currently developing web application.
We have an application where we use mongoDB to store data.
And we have an API where we search for the document via text search.
As an example: if the user type “New York” then the request should send the all the available data in the collection to the keyword “New York". (Here we call the API for each letter typed.) We have nearly 200000 data in the DB. Once the user searches for a document then it returns nearly 4000 data for some keywords. We tried with limiting the data to 5 – so it returns the top 5 data, and not the other available data. And we tried without limiting data now it returns hundreds and thousands of data as I mentioned. And it causes the request to slow down.
At Frontend we Bind search results to a dropdown. (NextJs)
My question:
Is there an optimizing way to search a document?
Are there any suggestions of a suitable way that I can implement this requirement using mongoDB and net5.0?
Or any other Implementation methods regarding this requirement?
Following code segment shows the query to retrieve the data to the incomming keyword.
var hotels = await _hotelsCollection
.Find(Builders<HotelDocument>.Filter.Text(keyword))
.Project<HotelDocument>(hotelFields)
.ToListAsync();
var terminals = await _terminalsCollection
.Find(Builders<TerminalDocument>.Filter.Text(keyword))
.Project<TerminalDocument>(terminalFeilds)
.ToListAsync();
var destinations = await _destinationsCollection
.Find(Builders<DestinationDocument>.Filter.Text(keyword))
.Project<DestinationDocument>(destinationFields)
.ToListAsync();
So this is a classic "autocomplete" feature, there are some known best practices you should follow:
On the client side you should use a debounce feature, this is a most. there is no reason to execute a request for each letter. This is most critical for an autocomplete feature.
On the backend things can get a bit more complicated, naturally you want to be using a db that is suited for this task, specifically MongoDB have a service called Atlas search that is a lucene based text search engine.
This will get you autocomplete support out of the box, however if you don't want to make big changes to your infra here are some suggestions:
Make sure the field your searching on is indexed.
I see your executing 3 separate requests, consider using something like Task.WhenAll to execute all of them at once instead of 1 by 1, I am not sure how the client side is built but if all 3 entities are shown in the same list then ideally you merge the labels into 1 collection so you could paginate the search properly.
As mentioned in #2 you must add server side pagination, no search engine can exist without one. I can't give specifics on how you should implement it as you have 3 separate entities and this could potentially make pagination implementation harder, i'd consider wether or not you need all 3 of these in the same API route.
Our Java app saves its configurations in a MongoDB collections. When the app starts it reads all the configurations from MongoDB and caches them in Maps. We would like to use the change stream API to be able also to watch for updates of the configurations collections.
So, upon app startup, first we would like to get all configurations, and from now on - watch for any further change.
Is there an easy way to execute the following atomically:
A find() that retrieves all configurations (documents)
Start a watch() that will send all further updates
By atomically I mean - without potentially missing any update (between 1 and 2 someone could update the collection with new configuration).
To make sure I lose no update notifications, I found that I can use watch().startAtOperationTime(serverTime) (for MongoDB of 4.0 or later), as follows.
Query the MongoDB server for its current time, using command such as Document hostInfoDoc = mongoTemplate.executeCommand(new Document("hostInfo", 1))
Query for all interesting documents: List<C> configList = mongoTemplate.findAll(clazz);
Extract the server time from hostInfoDoc: BsonTimestamp serverTime = (BsonTimestamp) hostInfoDoc.get("operationTime");
Start the change stream configured with the saved server time ChangeStreamIterable<Document> changes = eventCollection.watch().startAtOperationTime(serverTime);
Since 1 ends before 2 starts, we know that the documents that were returned by 2 were at least same or fresher than the ones on that server time. And any updates that happened on or after this server time will be sent to us by the change stream (I don't care to run again redundant updates, because I use map as cache, so extra add/remove won't make a difference, as long as the last action arrives).
I think I could also use watch().resumeAfter(_idOfLastAddedDoc) (didn't try). I did not use this approach because of the following scenario: the collection is empty, and the first document is added after getting all (none) documents, and before starting the watch(). In that scenario I don't have previous document _id to use as resume token.
Update
Instead of using "hostInfo" for getting the server time, which couldn't be used in our production, I ended using "dbStats" like that:
Document dbStats= mongoOperations.executeCommand(new Document("dbStats", 1));
BsonTimestamp serverTime = (BsonTimestamp) dbStats.get("operationTime");
I'm using the algolia client directly in my node js backend so i don't use instantsearch.js.
I can easily querying/indexing/updating etc. my algolia index but i can't find a way to clear the cache because my app always need to display an updated hits list in real time.
i've tried
client.initIndex('my index');
client.clearCache()
But without success. Always need to force unmount/remount manually my app to see the updated hits list.
Any solution ?
Its an old question but since there are no responses ....
Docs
Given the following instantiation:
const client = algoliasearch('H58KBL9VKQ', '••••••••••••••••••••');
const index = client.initIndex('your_index_name');
There are two methods.
client.clearCache() - when using multiple indices
index.clearCache() - when querying one index specifically
Note that BOTH return a Promise and both work only in the BROWSER, since the Node API doesn't cache results, there is no cache to clear.
I am looking into best -practices for returning search results. I have a search page that subscribes to a publication that returns a find based on the searched regex query in multiple fields. This gets put into the minimongo collection, on the client.
At this time, the way it is being handled is that facets are being set up from the subscription. My question is if the filtering for the pre-loaded results from the backend should be done client side, or if the query should be sent back.
Example :
Given a collection of fruits, i want to find all that have the color red. The server returns this, but I have facets based on the fruits. So, i have a checkbox for strawberries, apples, cherries, etc. If I click on the checkbox for cherries, should I just be filtering the current minimongo collection, or should I re-query?
Logically, I already have all the needed items in my collection that I could be filtering on, so I am not sure why I would need to hit the back-end. The only time I should hit the backend is if in the search, I type in a new query (such as blue), and the facets get re-done appropriately
If your original search is returning all matching documents then adding criteria on the client can just be done in your minimongo query if the fields on which the additional criteria were returned with the original search.
OTOH if the original search is returning a paginated list or just the top N results or if the required keys weren't included then you want to continue the search on the server.
In a traditional request-response system, you might also want to query the server each time if the underlying data was rapidly changing (ex: a reservations system). With Meteor the reactive nature of pub-sub means that the data on the client is being constantly refreshed with adds/changes/deletions via DDP over WebSocket.
I am using mongo inbound adapter for retrieving data from mongo. Currently I am using below configuration.
<int-mongo:inbound-channel-adapter
id="mongoInboundAdapter" collection-name="updates_IPMS_PRICING"
mongo-template="mongoTemplatePublisher" channel="ipmsPricingUpdateChannelSplitter"
query="{'flagged' : false}" entity-class="com.snapdeal.coms.publisher.bean.PublisherVendorProductUpdate">
<poller max-messages-per-poll="2" fixed-rate="10000"></poller>
</int-mongo:inbound-channel-adapter>
I have around 20 records in my data base which qualifies the mentioned query but as I am giving max-messages-per-poll value 2 I was expecting that i will get maximum 2 records per poll.
but I am getting all the records which qualifies the mentioned query. Not sure what I am doing wrong.
Actually I'd suggest to raise a New Feature JIRA ticket for that query-expression to allow to specify org.springframework.data.mongodb.core.query.Query builder, which has skip() and limit() options and from there your issue can be fixed like:
<int-mongo:inbound-channel-adapter
query-expression="new BasicQuery('{\'flagged\' : false}').limit(2)"/>
The mongo adapter is designed to return a single message containing a collection of query results per poll. So max-messages-per-poll makes no difference here.
max-messages-per-poll is used to short-circuit the poller and, in your case, the second poll is done immediately rather than waiting 10 seconds again. After 2 polls, we wait again.
In order to implement paging, you will need to use a query-expression instead of query and maintain some state somewhere that can be included in the query on each poll.
For example, if the documents have some value that increments you can store off that value in a bean and use the value in the next poll to get the next one.