Context: We are implementing a news app. For now, you can assume the news to be the same across all users, and maintains an order based on the parameters we set (according to trends, and date).
Problem: We are not sure what the best implementation for keeping track of what users read is. We want to be able to configure a way in which we can track what users read and what they didn’t.
Assumption: You can assume that the posts in the database are in a descending order, based on time.
So, the ideal scenario is that: when there are posts: A,B,C,D,E fetched from the server in the app, and the user read A,B. Now the user only gets to see C,D,E when they check for next posts. If they do previous, they see posts in the following order B-> A.
Furthermore, when P,Q is added to the database, now, the user must see next posts in the order of P->Q->C->D->E and so on.
Example: Let us assume there are 20 news in our app right now, and Gavin picks up his phone and starts reading from our app. In midst of his usage, he finds himself occupied with some other work, so quits the app after reading 5 news posts.
The challenge for us now is to figure the best way to make sure Gavin doesn’t have to re-read the 5 posts he already did.
One way we thought we could solve this problem is through use of index. We can assume uniform ordering for our posts as mentioned in the context, so we could use an index to track where Gavin was last in the order of news and show him news based on that index.
However, one problem with that approach is, we could easily have 5 new posts when Gavin picks up his phone and uses our app again. So, if we have the news based on date, technically that indexing approach means that we omit 5 unread new posts instead of the 5 read old ones.
We've also thought of maintaining three lists: Read, Unread and New so that we fetch only posts that are not in our lists. For example, in my initial example: A-B-C-D-E is in unread initially. Then, after user reads A-B, read becomes A-B. Meanwhile, when P-Q is added in the database, P-Q is added to the list of unread posts as P-Q-C-D-E.
How do you solve this problem? Any suggestions are welcome as we kind of think we're not thinking out of box when it comes to a solution for the problem. Thank you! :)
As i first read problem the solution ends up in my mind is also having 2 different list read unread and new ones are added to end of unread ones and unread list is shown in reverse order so most recent ones are on the top. However is it the most efficient way? Discussible. For example if number of new number increases a lot, then will be memory inefficient. But i assume small numbers in general.
Related
This question probably already exist but its too specific and hard to seach for it.
So, imagine that we have a ecommerce application.
On page 1 we have a list of products. And when its tapped, we go to a page 2, where it holds more information about the product that you just tapped for. Pretty much like any other ecommerce out there.
Which one of these two situations are better:
When one product is tapped, we pass via arguments all the informations about this product to the page 2. Then, no requests to the database is necessary.
When one product is tapped, we pass its ID only, then we need to do a request to get this product information from database.
You might think its obvious that the option 1 is better, but with option 2 we pretty much guarantee that all product informations have the last update from database, because the owner might change the product price milliseconds after you just clicked.
image describing user interaction
I would go for the second option most of the time.
As you already said, the information will always be up to date. Also if you request all the products with all of their information it creates quite some overhead, depending on the size of the product page and the information about the different products. Another thing is updating the information live. Maybe you'll decide to add a Stream later on, updating the information while the user is on the product page. Querying each product will make that easier as well.
If you can afford the resources of requesting the product every time and your process isn't too expensive it's in my opinion the better option.
I am trying to implement infinite scrolling for documents stored in a MongoDB collection. Every document is a restaurant that has a numeric field rating, so I am using the rating field for sorting and showing restaurants with the highest rating first.
The problem is that the collection of the restaurants is not static. The ratings of the restaurants change in real time, therefore the order of the restaurants in the collection changes constantly. As a result, although I formally have the sorting key, it does not make much sense.
I am thinking of 2 solutions of the problem:
Accept that the order of the restaurants may change slightly while someone is doing the infinite scrolling. Make the front end responsible for getting rid of possible duplicates. Accept that some of the restaurants may not appear during a scrolling at all. But that looks more like working around the problem instead of solving it.
Only perform infinite scrolling against a static copy of the collection of restaurants. Update the static copy periodically (e.g., once a day) with the rating updates. But this approach seems overengineered. Also, what happens with the infinite scrolling at the moment when the static copy of the restaurants gets updated with the new ratings? Such scrolling will be broken as well because the problem with the changing order is still here, the order just does not change that frequently.
I am sure I am far not the first one who have faced this problem. After all, there are a lot of examples of infinite scrolling implementations out there, like Facebook or Instagram feeds. At the same time, all the articles I have read so far seem too superficial and covering only cases with infinite scrolling through static collections.
What is the right approach to deal with infinite scrolling for a collection that may change its order any time?
Thank you.
Infinite scrolling, as commonly implemented, isn't a precision navigation method to begin with. Who are your users?
Power users are likely to hate it (I do on github, facebook, etc.) hence won't be using it too much.
Non-power users won't be able to tell that data is missing. If they happen to be looking for a particular restaurant and it vanishes, telling them to reload the page will be a sufficient explanation for most.
Users who scrape your data will do it without delays between requests to get all of your data.
When you show the same restaurant twice people will notice so check for those cases in the frontend.
You may also consider having a high-precision rating field for sorting. For example, if normally your UI shows integer rating, keep the floating-point rating used during the calculation and sort by that. This will produce a more stable sort.
I recently developed an app using the Instagram API, But I faced some problems with it. (I got struck)
This app is not calling the api and getting images in the real-time. Instead, it calls the api once a week and store images in the database based on a specific hashtag. I see that their api is designed in a way to call it in real time and pull images, because it has this "recent" in its url, and it only provides 20 results. So every time a user reaches the end, it will load the next 20, until u reach the end.
However, in my case, When I pull the images every week, at one point, I will reach to the oldest image, and from that point I will be stopped, because Im going from Newest to the oldest. So I want to get data as from oldest to newest.( Actually I want to get the opposite of what's offered by instagram.) They have not provided any option to sort them in their docs.
Im thinking of a solution for this, but I can't wrap my head around it. So I will be much thankful if you guys can direct me in the right path.
I searched Google, viewed old questions in SO, but I didn't find the answer.
Edit : Im planning to fetch data starting from the Oldest one like this. (Since I know the min_tag_id, because I ran this once completely).
https://api.instagram.com/v1/tags/my_tag/media/recent?&client_id=xxxxxxxxxxxxxxx&min_tag_id=1408983759354183
where min_tag_id is of the oldest image. So everytime I will be calling to the min_tag_id. Is it a good solution?
Yes, using the min_tag_id and max_tag_id is the solution.
Have a look at this answer: Instagram Search for a tag within particular date range
So basically I want to implement the same functionality as StackOverflow's:
viewed 59344 times
So here is some background information:
I want to count only unique visits. The assumption that registered users will read the article many times (it is evolving)
I use MongoDB as a store
I would like it to be close to real-time
My system will have a registration, but I want to count the views of anonymous users as well
I understand that the best way to count unique visits is through registration, but the thing is that a big chunk of users will be just passive readers who do not need to create an account to read the information from the application. As far as I understand, the most convenient way is to save the IP address of every user, who reads the post. I also understand that IP addresses will not provide uniqueness (some different users will have the same IP, because they are behind the same ISP and one user can have different IPs, by using proxies, tor, etc)
The use of Mongo is not absolutely essential, just the thing is that everything is written in Mongo right now, so I will switch only if it will be much faster/convenient.
Background
Are you certain you need to track "unique" views?
I actually wouldn't expect popular sites to try to keep the view counts unique - bigger is better and re-visits for new comments are still additional "views" in the the sense of showing new content/comments/ads. There are other possible subtleties to "correctness" that may or may not be important for your use case, such as excluding crawlers or your own company's users/IPs.
Instead of spending time tracking unique views (which isn't overly meaningful), I would look at counting unique user interactions such as voting/liking/commenting on the page. You can then determine "popularity" of a page with some formula based on those metrics. There is an interesting example of this approach in the Radioactivity module for Drupal, where a "hotness" metric is calculated based on activity based on recency of user interactions.
Approaches to consider
1) For a simple view counter in MongoDB, I would just use $inc to bump up the view count when the page is loaded. You can exclude logging users by role as needed (for example admin users).
2) For a more accurate view counter I would pass off the problem to a web analytics platform (which you should be using with your site for more detailed analysis anyway). For example, you can use Google Analytics API or an open source application like Piwik. Web analytics systems already have solutions in place for determining unique users/views, and the API calls for these can be asynchronous via JavaScript.
3) If implementing your own unique view tracking a definite requirement, I would use a separate collection for tracking views and upsert based on your uniqueness criteria (unique view per user,article pair for registered users or session_id,article pair for anon users). I would combine this with approach #1 (incrementing a view counter for the article views) by incrementing a counter of article views if the upsert results in an insert.
One of the way that you can solve the problem is using the cookies , once a user has visited the page , you can have one cookie added saying that he is already visited the page and you do not need to count him again. You can keep on appending some key to know what all pages he had visited. I know cookies can be deleted but in any solution there will be tradeoff.
From the mongoDB prospective , if you want very fast insert and read , i would suggest couple of things you can do.
1) As you create a article , create a document like this in your may be log collection
{"_id" : "Article URL" , {"Hit" : 0}}
Why i am not suggesting to add IP address or any other information because , as you will add IP addresses , the size of the document going to change mongoDB need to find new allocated space. Which is bad from performance angle. As you are only incrementing the counter it will not increase the size of the document and it will no need to change it place. + You have limitation on the maximum size of the document you can have.
2) Creating document in advance will give direct update statement and no worry to check for the existence of the document for the article Id or not.
I am currently drafting a concept for a (mostly) HTML-based collaboration suite which I plan to implement using CQRS. This software will contain messages that can be sent to the user (which can either be read or unread, obviously) and other elements which shall be marked "new" if they were created after the last user login.
Hardly something new, but I am not quite sure how that would be correctly implemented using CQRS. As I understand it, Change of any kind should, without exception, only be possible via Commands. But creating commands for every single (new) element that is being accessed seems a bit too much, not to mention the overhead.
I don't know if I need it, but what would be the best way to implement a Last-Accessed Timestamp on elements. Basically the same problem like the above, with the difference that the change happens EVERY time the element is accessed, not only the first time for each user.
CQRS seems to be an awesome concept but it really needs more learning material. Can't wait till a book is released :)
Regards
[Edit] No one? Wouldn't have thought that this is such a complicated issue..
I assume you're using event-sourcing in which case once you allow your query-service/event-handlers to raise appropriate events then this becomes fairly easy to solve.
For your messages/elements; when handling the specific creation events of your elements either add to existing or create additional event-handlers, to store to a messages read-model with a status of new and appropriate information about the element.
As part of you're user login I don't see why you can't raise a user-logged-in event (from the security/query service depending on how your implementing authentication) to say the user has logged in. An event-handler could capture this and write the last-login timestamp to a specific user-last-login read-model.
In addition the user-logged-in event-handler would need to update all the new messages (for that user) to an unread status. Seeing as we're changing the status of the messages as the user logs in do you still need to store the last-login timestamp?
For your last-accessed timestamp, perhaps you could just work this into your query service as queries for your different elements complete. Raise a query-completed event with element id/type information.