I need to perform sorting on Elasticsearch documents...
I have one index created for MongoDB collections 'products', which have price and product ratings in it.
I have another collection 'product_hits' in which I am save one record (product_id, IP etc.) on every click of particular product by user. Now I want to sort product documents on by considering Product hit count (maybe which I can get through aggregation), price and product rating.
In short I want to rank all the products based on price and popularity as other sites does.
How can I achieve this in elasticsearch?
I gone though scripting of elasticsearch and I am able to sort on price and product rating..... but I didn't find anything useful in which we can perform sort based on multiple indices.
is it possible?? or do I have to sort all records on my own through coding?
I am using play framework.
I hope this question can be understood... Its complex..!!!
Related
Lets say I have simple document structure like:
{
"item": {
"name": "Skittles",
"category": "Candies & Snacks"
}
}
On my search page, whenever user searches for product name, I want to have a filter options by category.
Since categories can be many (like 50 types) I cannot display all of the checkboxes on the sidebar beside the search results. I want to only show those which have products associated with it in the results. So if none of the products in search result have a category, then do not show that category option.
Now, the item search by name itself is paginated. I only show 30 items in a page. And we have tens of thousands of items in our database.
I can search and retrieve all items from all pages, then parse the categories. But if i retrieve tens of thousands of items in 1 page, it would be really slow.
Is there a way to optimize this query?
You can use different approaches based on your workflow and see what works the best in your situation. Some good candidate for the solution are
Use distinct prior to running the query on large dataset
Use Aggregation Pipeline as #Lucia suggested
[{$group: { _id: "$item.category" }}]
Use another datastore(either redis or mongo itselff) to store intelligence on categories
Finally based on the approach you choose and the inflow of requests for filters, you may want to consider indexing some fields
P.S. You're right about how aggregation works, unless you have a match filter as first stage, it will fetch all the documents and then applies the next stage.
I have a headache for a idea how to properly sort data from a MongoDB. It is using 2dsphere index and has timestamp createdAt. The goal is to show latest pictures (that what this collection is about, just a field mediaUrl...) but it has to be close to the user. I'm not very familiar with complex MongoDB aggregation queries so I thought here's a good place to ask. Sorting with $near shows only items sorted by distance. But there's a upload time, e.g. if item is 5 min fresh but is like 500 meters far than older item it still should be sorted higher.
Ugly way would be to iterate every few hundreds meters and collect data but maybe there's a smarter way?
So if I am correct you want to be able to sort on 2 fields ?
distance
timestamp
You should check out this method:
https://docs.mongodb.com/manual/reference/operator/aggregation/sort/
It allows you to sort multiple columns.
QUERYING MONGODB: RETREIVE SHOPS BY NAME AND BY LOCATION WITH ONE SINGLE QUERY
Hi folks!
I'm building a "search shops" application using MEAN Stack.
I store shops documents in MongoDB "location" collection like this:
{
_id: .....
name: ...//shop name
location : //...GEOJson
}
UI provides to the users one single input for shops searching. Basically, I would perform one single query to retrieve in the same results array:
All shops near the user (eventually limit to x)
All shops named "like" the input value
On logical side, I think this is a "$or like" query
Based on this answer
Using full text search with geospatial index on Mongodb
probably assign two special indexes (2dsphere and full text) to the collection is not the right manner to achieve this, anyway I think this is a different case just because I really don't want to apply sequential filter to results, "simply" want to retreive data with 2 distinct criteria.
If I should set indexes on my collection, of course the approach is to perform two distinct queries with two distinct mehtods ($near for locations and $text for name), and then merge the results with some server side logic to remove duplicate documents and sort them in some useful way for user experience, but I'm still wondering if exists a method to achieve this result with one single query.
So, the question is: is it possible or this kind of approach is out of MongoDB purpose?
Hope this is clear and hope that someone can teach something today!
Thanks
I would like to know if mongo lets you do complex multi sorting and how to do it.
Imagine that you have Products with two bollean (0/1) fields "Sale" & "Focus" and a price (double).
I would like to return all products but order then by:
1- Products on Focus and on Sale
2- Products on Focus
3- Products on Sale
and inside each one of this three groups they should be ordered by price desc
Is this even doable in a single query sort?
So I have an interesting use case that I'm stuck trying to find a efficient mongo query for.
To begin, I have 12,000 categories with 100,000 posts. I need to randomly select a 100 pairs of posts, from random categories. The pairs are randomly selected from categories, but each pair must have both posts belonging to the same category.
Users look at each pair to rate and once they finish looking at the 100, they fetch another 100 random posts (preferably not any of the same pairs they've already seen).
So the requirements are:
Fetch 100 pairs of posts randomly from a random set of categories
Optional requirements:
Not to return the same pairs they've already rated
Mongo Collections
Users
Categories
Posts
CategoryId
Ratings (embedded collection in posts)
How would I do this in Mongo... should I move some of this data off of mongo to another db if it's easier?
Yes. Very interesting question. My suggestion is to put a randomVal field on your post documents. Then you can sort on {CategoryId: 1, randomVal: 1}. The result will be a cursor that groups all the posts by CategoryId but randomly within that grouping. If you conceptually think of this as an array, you can pick all the even indexed posts, and pair them with an odd neighbor to get unique random pairs within categories.
I think that how to select the random pairs from this list will take some experimentation, but my gut instinct is that the best approach would be to have a separate process that periodically caches a collection of pairs which are sorted by a separate randomVal2. The user facing queries would just increment through this pairs collection 100 at a time.
I think you can achieve this in two query. First you need to use aggregation framework and do a map reduce operation on Posts collection. In the map phase use category id as the key and emit post ids to reducer.
In the reduce phase choose two random id from each category. In the end of the map reduce you will have a list of Post ids. Then retrieve those posts from Posts collection.
Add a ratedBy field to Post document and when user rated a post add his or her userName to ratedBy field. Then use that field as a filter to your map reduce command in the first place so that you don't bring already rated documents to user.
Good luck