I'm using the algorithm UserUserItemScorer is possible to obtain the accuracy of recommendation, ie, the quality score of the recommended item. The only way I found was the value of "score". Is there another way besides the "score" method?
[disclaimer: LensKit lead developer]
First, a terminology thing: in recommender systems, the score and the accuracy of the recommendation are very different things. The score is how relevant the recommender thinks the item is, and is the basis for doing recommendation; the accuracy of the recommendation is how well that score models the user's actual opinion of the item.
I'll move forward assuming that you're looking for ways to get the score for an item.
There are at least three ways:
Call score on ItemScorer for individual items. This is very slow for multiple items.
Call score on ItemScorer with a batch of items. This is usually much faster. However, if you got the items from an ItemRecommender, then you are probably repeating computations.
The ItemRecommender returns a list of ‘scored IDs’, which are item IDs associated with scores. The getScore() method on the item recommender will get the score for each item.
But in general, the item scorer's score is exactly how you get relevance estimates from LensKit. The scores returned by an ItemRecommender are usually just the scores provided by the underlying item scorer.
Related
At the moment, my problem has four metrics. Each of these measures something entirely different (each has different units, a different range, etc.) and each is weighted externally. I am using Drools for scoring.
I only have only one score level (SimpleLongScore) and I have to find a way to appropriately combine the individual scores of these metrics onto one long value
The most significant problem at the moment is that the range of values for the metrics can be wildly different.
So if, for example, after a move the score of a metric with a small possible range improves by, say, 10%, that could be completely dwarfed by an alternate move which improves the metric with a larger range's score by only 1% because OptaPlanner only considers the actual score value rather than the possible range of values and how changes affect them proportionally (to my knowledge).
So, is there a way to handle this cleanly which is already part of OptaPlanner that I cannot find?
Is the only feasible solution to implement Pareto scoring? Because that seems like a hack-y nightmare.
So far I have code/math to compute the best-possible and worst-possible scores for a metric that I access from within the Drools and then I can compute where in that range a move puts us, but this also feel quite hack-y and will cause issues with incremental scoring if we want to scale non-linearly within that range.
I keep coming back to thinking I should just just bite the bullet and implement Pareto scoring.
Thanks!
Take a look at #ConstraintConfiguration and #ConstraintWeight in the docs.
Also take a look at the chapter "explaning the score", which can exactly tell you which constraint had which score impact on the best solution found.
If, however, you need pareto optimization, so you need multiple best solutions that don't dominate each other, then know that OptaPlanner doesn't support that yet, but I know of 2 cases that implemented it in OptaPlanner by hacking BestSolutionRecaller.
That being said, 99% of the cases that think of pareto optimization, are 100% happy with #ConstraintWeight instead, because users don't want multiple best solutions (except during simulations), they just want one in production.
I work for a small non-profit organization and we are looking for a way to quickly tally scores and provide a few statistics for donors at our annual golf tournament. I thought this would be fairly easy, but I'm struggling to come up with a database schema to capture the scores. I can't figure out how the player's score relates to the specific hole on the course.
This is the diagram that I have so far. Am I way off base with this?
The Schema can be found here: https://app.quickdatabasediagrams.com/#/schema/forneGJp40inm7rWlf2Sbg
Perhaps make the Scores table a m:n join table between Players and Holes to capture each player's score on each hole. This is depicted on the diagram below. To get the score for a round you'd sum all scores for all holes with a specific CourseId, for a specific event.
I also denormalised it a little, adding a total score to the Rounds table. This means that you don't need to SUM() the individual scores every time to get the tallies for each Player's Round. That's just a suggestion for performance optimisation.
Source: https://app.quickdatabasediagrams.com/#/schema/x_amshIckkeGp8KAKEAmLQ
If it is possible to play the same course twice in the same event (the first and last matches could be on the same course, for example) then you should provide for that.
I have two other suggestions:
I think the relationship between Events and Venues is in the wrong direction.
I suggest splitting Players into two tables. One representing the human being, and the other representing the human's participation in the round. Perhaps "Person" and "Contestant" would be good names.
Here is a problem I'm trying to solve using graph algorithms. Answer to this question is easy if one is familiar with different graph traversal algorithms. What I want to learn is how can we reduce the complexity of this problem?
Let say we have to traverse in someone's network - Friends, Friends of
Friends (FoF) and FoFoF (1st, 2nd, 3rd Degree.. up to 6th degree) to
search for a particular thing, say 'people living in California'. The
complexity of the problem greatly increases when you have 1000 friends
and your 1000 friends have 1000 friends each and so on.
Let's say we want to do an optimized search, where you know the
destination node (here, a person living in California). How will you
reduce the complexity of the problem?
The program you submit should return the degree by which that person
is connected to you. [where the 'destination node' is your Degree 1st
(Friend), or 2nd (friend of friend) or 3rd Degree (FoFoF) or a Degree
greater than 3rd degree].
Assuming your graph is unweighted, doing Breadth First Search will give you shortest paths (which effectively are the degrees that you need). If the destination is known you can also use Dijkstra's Algorithm to find a shortest path to that specific node, although if the graph is unweighted just doing the BFS will be more efficient as it's complexity is lower than Dijkstra's. Also if I understand correctly your output has to cover only 4 cases: Degrees 1,2,3 or higher than that. If so, you can just BFS the first three levels and store the results. Then you can answer the question in constant time by checking for the existence of such person in the data obtained via BFS.
I've read some papers of Matrix Factorization(Latent Factor Model) in Recommendation System,and I can implement the algorithm.I can get the similar RMSE result like the paper said on the MovieLens dataset.
However I find out that,if I try to generate a top-K(e.g K=10) recommended movies list for every user by rank the predicted rating,it seems that the movies that are thought to be rated high point of all users are the same.
Is that just what it works or I've got something wrong?
This is a known problem in recommendation.
It is sometimes called "Harry Potter" effect - (almost) everybody likes Harry Potter.
So most automated procedures will find out which items are generally popular, and recommend those to the users.
You can either filter out very popular items, or multiply the predicted rating by a factor that is lower the more globally popular an item is.
The problem I'm trying to solve is finding the right similarity metric, rescorer heuristic and filtration level for my data. (I'm using 'filtration level' to mean the amount of ratings that a user or item must have associated with it to make it into the production database).
Setup
I'm using mahout's taste collaborative filtering framework. My data comes in the form of triplets where an item's rating are contained in the set {1,2,3,4,5}. I'm using an itemBased recommender atop a logLikelihood similarity metric. I filter out users who rate fewer than 20 items from the production dataset. RMSE looks good (1.17ish) and there is no data capping going on, but there is an odd behavior that is undesireable and borders on error-like.
Question
First Call -- Generate a 'top items' list with no info from the user. To do this I use, what I call, a Centered Sum:
for i in items
for r in i's ratings
sum += r - center
where center = (5+1)/2 , if you allow ratings in the scale of 1 to 5 for example
I use a centered sum instead of average ratings to generate a top items list mainly because I want the number of ratings that an item has received to factor into the ranking.
Second Call -- I ask for 9 similar items to each of the top items returned in the first call. For each top item I asked for similar items for, 7 out of 9 of the similar items returned are the same (as the similar items set returned for the other top items)!
Is it about time to try some rescoring? Maybe multiplying the similarity of two games by (number of co-rated items)/x, where x is tuned (around 50 or something to begin with).
Thanks in advance fellas
You are asking for 50 items similar to some item X. Then you look for 9 similar items for each of those 50. And most of them are the same. Why is that surprising? Similar items ought to be similar to the same other items.
What's a "centered" sum? ranking by sum rather than average still gives you a relatively similar output if the number of items in the sum for each calculation is roughly similar.
What problem are you trying to solve? Because none of this seems to have a bearing on the recommender system you describe that you're using and works. Log-likelihood similarity is not even based on ratings.