I've built a RESTful API that fetches employees from the database, Now I want to implement search functionality that filters employees by name and then fetches it. In the frontend, I've fetched employees and then based on user input I'm converting both user input and result from db to lowercase and then matching for similarity? Do you think this problem can be approached in a different way? any other strategies? thanks
Yes, let the database do the comparison for you in SQL. Less resources will be used throughout the entire stack. One of the main features for databases is searching for specific data. Also SQL is case insensitive so no need to do a toLowerCase.
SELECT *
FROM Employees
WHERE name LIKE '%SearchValue%'
Only select the columns you need.
Related
I am new to MongoDB. And I have the following issue on currently developing web application.
We have an application where we use mongoDB to store data.
And we have an API where we search for the document via text search.
As an example: if the user type “New York” then the request should send the all the available data in the collection to the keyword “New York". (Here we call the API for each letter typed.) We have nearly 200000 data in the DB. Once the user searches for a document then it returns nearly 4000 data for some keywords. We tried with limiting the data to 5 – so it returns the top 5 data, and not the other available data. And we tried without limiting data now it returns hundreds and thousands of data as I mentioned. And it causes the request to slow down.
At Frontend we Bind search results to a dropdown. (NextJs)
My question:
Is there an optimizing way to search a document?
Are there any suggestions of a suitable way that I can implement this requirement using mongoDB and net5.0?
Or any other Implementation methods regarding this requirement?
Following code segment shows the query to retrieve the data to the incomming keyword.
var hotels = await _hotelsCollection
.Find(Builders<HotelDocument>.Filter.Text(keyword))
.Project<HotelDocument>(hotelFields)
.ToListAsync();
var terminals = await _terminalsCollection
.Find(Builders<TerminalDocument>.Filter.Text(keyword))
.Project<TerminalDocument>(terminalFeilds)
.ToListAsync();
var destinations = await _destinationsCollection
.Find(Builders<DestinationDocument>.Filter.Text(keyword))
.Project<DestinationDocument>(destinationFields)
.ToListAsync();
So this is a classic "autocomplete" feature, there are some known best practices you should follow:
On the client side you should use a debounce feature, this is a most. there is no reason to execute a request for each letter. This is most critical for an autocomplete feature.
On the backend things can get a bit more complicated, naturally you want to be using a db that is suited for this task, specifically MongoDB have a service called Atlas search that is a lucene based text search engine.
This will get you autocomplete support out of the box, however if you don't want to make big changes to your infra here are some suggestions:
Make sure the field your searching on is indexed.
I see your executing 3 separate requests, consider using something like Task.WhenAll to execute all of them at once instead of 1 by 1, I am not sure how the client side is built but if all 3 entities are shown in the same list then ideally you merge the labels into 1 collection so you could paginate the search properly.
As mentioned in #2 you must add server side pagination, no search engine can exist without one. I can't give specifics on how you should implement it as you have 3 separate entities and this could potentially make pagination implementation harder, i'd consider wether or not you need all 3 of these in the same API route.
I am using prisma and yoga graphql servers with a postgres DB.
I want to implement authorization for my graphql queries. I saw solutions like graphql-shield that solve column level security nicely - meaning I can define a permission and according to it block or allow a specific table or column of data (on in graphql terms, block a whole entity or a specific field).
The part I am stuck on is row level security - filtering rows by the data they contain - say I want to allow a logged in user to view only the data that is related to him, so depending on the value in a user_id column I would allow or block access to that row (the logged in user is one example, but there are other usecases in this genre).
This type of security requires running a query to check which rows the current user has access to and I can't find a way (that is not horrible) to implement this with prisma.
If I was working without prisma, I would implement this in the level of each resolver but since I am forwarding my queries to prisma I do not control the internal resolvers on a nested query.
But I do want to work with prisma, so one idea we had was handling this in the DB level using postgres policy. This could work as follows:
Every query we run will be surrounded with “begin transaction” and “commit transaction”
Before the query I want to run “set local context.user_id to 5"
Then I want to run the query (and the policy will filter results according to the current_setting(‘context.user_id’))
For this to work I would need prisma to allow me to either add pre/post queries to each query that runs or let me set a context for the db.
But these options are not available in prisma.
Any ideas?
You can use prisma-client instead of prisma-binding.
With prisma-binding, you define the top level resolver, then delegates to prisma for all the nesting.
On the other hand, prisma-client only returns scalar values of a type, and you need to define the resolvers for the relations. Which means you have complete control on what you return, even for nested queries. (See the documentation for an example)
I would suggest you use prisma-client to apply your security filters on the fields.
With the approach you're looking to take, I'd definitely recommend a look at Graphile. It approaches row-level security essentially the same way that you're thinking of. Unfortunately, it seems like Prisma doesn't help you move away from writing traditional REST-style controller methods in this regard.
TLDR: need standard/framework to accomplish such a query /courses?teacher.age=30 over multiple resources (course, teacher).
We are using Spring Data Rest to expose our relational database tables in a hateoas format. The entities are published as REST resources.
Now, the users would like to search over multiple resources.
For example "give me the names of all courses given by a teacher of 30 years old".
Because we need courses, this would be a search operation on /courses.
However, the age of a teacher is on /teachers.
So in 2 steps:
/teachers?age=30 which have _links to courses. Then, every course can be opened to inspect it's name (eg: /course/942).
A more intuitive operation would be to directly call /courses?teacher.age=30.
This is what we would like to accomplish.
Of course, we can implement this by hand. But the real question is: are there already existing standards (and popular libraries) to accomplish this?
We would like to find a scalable solution, where not every attribute (to search on) has to be added manually.
This dot-notation is just a way of representing the query. It is not a requirement of a standard/library.
Characteristics of our application:
relational database (postgres)
json api
hal/hateoas
I am trying to use Neo4jOperations in Spring Data Neo4j 4.
Filters can be used in Neo4jOperations for retrieving data from NodeEntity.
Eg. I have a case where there are actors with "roles" in a movie. Now,Actor and Movie is a NodeEntity and roles is a RelationshipEntity. If i query a movie for a title using Filters, it does show me the relevant details.
I want to query details in the form - "Find me actors who have relationship (any relationship) in Movie 'Cloud Atlas'." I am not able to find a way to work for this using Filters.
Filters let you specify something like key-value pairs. Can I specify key value as "roles.movies.title" and query for the results?
Tried it this way, it does not work out.
Do not want to use Custom Query as we want to keep the query generic so that it can accomodate a lot of use cases. Attaching a custom query will mean that it can address only a specific case.
It is possible to implement your own FilterFunction, however, much simpler is to use a custom query for this. Example:
#Query("MATCH (a:Actor)-[]-(n:Movie {name: "Cloud Atlas"}) return n")
public List<Actor> findActorsRelatedToCloudAtlas()
so I'm working with a database that has multiple collections and some of the data overlaps in the collection . In particular I have a collection called app-launches which contains a field called userId and one called users where the _id of a particular object is actually the same as the userId in app-launches. Is it possible to group the two collections together so I can analyze the data? Or maybe match the the userId in app-launches with the _id in users?
There is no definit answer for your question Jeffrey and none of the experts here can tell you to choose which technique over other just by having this information.
After going through various web pages over internet and mongo documentation and understanding the design patterns used in Mongo over a period of time, How I would design it depends on few things which I can try explaining it here in short.
if you have a One-To-One relation then always prefer to choose Embedding over Linking. e.g. User and its address (assuming user has only one address) thus you can utilize the atomicity (without worrying about transactions) as well easily fetch the records without too and fro to bring other information as in the case of Linking (like in DBRef)
If you have One-To-Many relation then you need to consider whether you can do the stuff by using Embedding (prefer this as explained the benefits in point 1). However, embedding would help you if you always want the information altogether e.g. Post/Comments where your requirement is to get the post and all of its comments by postId let say. But think of a situation where you need to get all the comments (and it related posts) which contains some specific tags in comments. in this case you should prefer Linking Because if you go via Embedding route then you would end up getting all the collection of comments for a post and you have to filter the desired comments.
for a Many-To-Many relations I would prefer two separate entities as well another collection for linking them e.g. Product-Category.
-$