I need some guidance in implementing mail notifications for new chat messages. The mail notification would inform the user of all the chats that had new messages in the previous hour.
To get it done, I'll need to query all chats in a table within a time interval. First thing that came to mind was adding new global index where a hash would be a boolean for whether the chat has unread messages, and range would be timestamp for the latest message within that chat.
But I have learned that boolean hash keys are quite the anti-pattern, as they would squeeze the documents in a single partition.
Is there a different model that would allow us to query all items in a table within a numeric range?
I’m assuming that you want to query unread messages for a given user, since (again) I’m assuming that the read/unread status of a given notification should not change for one user if another user reads a notification for the same thing.
Going on that assumption, you should use a sparse index with the userId (or equivalent) as the hashkey and unreadNotificationTime as the sort key. When you insert a new notification into your table, set the value of unreadNotificationTime to the time stamp for the notification. When the user has read the notification, delete the unreadNotificationTime attribute from the item.
Why does this work?
DynamoDB only requires that an item has the key attributes of the base table, and any other attributes are optional. The way indexes work in DynamoDB is that an item from the base table will only appear in an index of the item has all of the key attributes of that particular index.
By setting a value for unreadNotificationTime when you store a notification, all newly created notifications will automatically be populated to the unread messages index. By deleting the unreadNotificationTime when a message is read, you the notification from that index. With this schema, there’s no need for any filtering or scan operations. Your index will only contain notifications that are unread, grouped by userId, and sorted by date.
Related
I have added posts to firebase and I am wondering how I can pull the posts chronologically based on when the user has posted them.
My Database is set up like below
The first node after comments is the User ID and then the posts are underneath that. Obviously, these posts are in order, however if a new user posts something in between "posting" and "another 1" ,for example, how would I pull that so it shows up in between.
Is there a way to remove the autoID and just use the userID as a key? The problem I am running into is the previous post is overwritten then.
I am accepting the answer as it is the most thorough. What I did to solve my problem was just create the unique key as the first node and then use the UID as a child and the comment as a child. Then I pull the unique key's as they are in order and find the comment associated with the uid.
The other answers all have merit but a more complete solution includes timestamping the post and denormalizing your data so it can be queried (assuming it would be queried at some point). In Firebase, flatter is better.
posts
post_0
title: "Posts And Posting"
msg: "I think there should be more posts about posting"
by_uid: "uid_0"
timestamp: "20171030105500"
inv_timestamp: "-20171030105500"
uid_time: "uid_0_ 20171030105500"
uid_inv_time: "uid_0_-20171030105500"
comments:
comment_0
for_post: "post_0"
text: "Yeah, posts about posting are informative"
by_uid: "uid_1"
timestamp: "20171030105700"
inv_timestamp: "-20171030105700"
uid_time: "uid_1_20171030105700"
uid_inv_time: "uid_1_-20171030105700"
comment_1
for_post: "post_0"
text: "Noooo mooooore posts, please"
by_uid: "uid_2"
timestamp: "20171030110300"
inv_timestamp: "-20171030110300"
uid_time: "uid_2_20171030110300"
uid_inv_time: "uid_2_-20171030110300"
With this structure we can
get posts and their comments and order them ascending or descending
query for all posts within the last week
all comments or posts made by a user
all comments or posts made by a user within a date range (tricky, huh)
I threw a couple of other key: value pairs in there to round it out a bit: compound values, query-ing ascending and descending, timestamp.
You can not use the userID as key value instead of the autoID, because the key must be unique, thats why Firebase just updates the value and does not add another one with the same key. Normally Firebase nodes are ordered chronologically by default, so if you pull the values, those should be in the right order. However if you wanna make sure about that, you can add a timestamp value and set a server timestamp. After pulling the data you can order it by that timestamp (I think there is actually a timestamp saved automatically by firebase that you can access somehow, but you need to look that up in the documentation). If I got it right, in order to accomplish what you want, you need to change the structure of your database. For example you could maybe use the autoID but save the userID you wanted to use as key as a value if you need that. Hope I got your idea right, if not just be more precise and I will try to help.
Firebase keys are chronological by default - it's built into their key generation algorithm. I think you need to restructure/rethink your data.
Your POSTS database should (possibly) have the comments listed with each post, and then you can duplicate on the user record if needed for faster retrieval if they need to be accessed by user. So something like:
POSTS
- post (unique key)
- title (text)
- date (timestamp)
- comments
- comment (unique key)
- text (text)
- user_id (user key)
- date (timestamp)
When you pull the comments, you shouldn't be pulling them from a bunch of different users. That could result it a lot of queries and a ton of load time. Instead, the comments could be added (chronologically of course) to the post object itself, and also to the user if you want to keep a reference there. Unlike in MySQL, NoSQL databases can have quite a bit of this data duplication.
I'm attempting to use mongodb to implement a simple messaging system between two users in mongo. I want to be able to take two users, user0 and user1, and search for their entry in a collection. If the entry for those two users doesn't exist I want to create it and then add the message that was sent to its message field. If it does exist I just want to push the message to the message field.
I'm not really sure the best way to implement this.
db.privateChat.update(
{between:{$all:['user0', 'user1']}},
{$push:{message:'text'}}, {upsert:true}
)
And other similar entry schemes but they don't work. They produce the error:
"Cannot create base during insert of update. Caused by :ConflictingUpdateOperators Cannot update 'between' and 'between' at the same time"
I can think of other ways to do this producing a symmetric key (where the order of the users don't matter for the purposes of the search) from say adding the hashes together or a query that checks if either messenger0 or messenger1 is either user0 or user1 but these don't seem like great ways of doing it. Is this totally the wrong approach?
Thanks.
I think this could be solved by design.
let say that we have document in collection chats;
chat{
_id,
between[arrayOfIds],
startTime,
events[
{message:{
fromUserId,
timeStamp,
data}
}}
]}
}
then messages will be stored in message object inside chat .
App will be aware of chat _id so there will be no issues when you will have a group chat between more than 2 users.
This approach will allow you to prevent overflowing document size limitation as you could start new chat entry every week, day, etc...
Have a fun!
What is the correct way of storing large lists in PostgreSQL?
I currently have a "user" table, where the "followers" column stores a list of the followers that that user has. This list is stored in JSON, and every time the server wants to add a new user to that list, it retrieves it from the database, appends the new user, and then replaces the old list with the new list.
The problem is that these lists tend to get quite lengthy, which might affect performance. Is it possible to simply append to the list directly via SQL without retrieving it and rewriting it later?
Use a separate table for followers. The table should have at least two columns: userid and followerid. And it's good practice to have a primary key for this table as well, so let's give it a "ufid".
You can do a select to get all the elements and compute the JSON string if your application needs it. But do not work with JSON or any other string representation of the list, as it defeats the purpose of a relational database.
To add a new follower, simply add a new record to the follower table with the userid; deleting and update are also done on the record level without working with the "other records".
If followers is a list of integers which are primary keys to their accounts, make it an integer array int[]. If they are usernames or other words, go with a string array character varying[].
To append to an array column you can do this:
UPDATE the_table SET followers = followers || new_follower WHERE id = user;
I'm working on a voting site and I'm wondering how I should handle votes.
For example on SO when you vote for a question (or answer) your vote is stored, and each time I go back on the page I can see that I already voted for this question because the up/down button are colored.
How do you do that? I mean I've several ideas but I'm wondering if it won't be an heavy load for the database.
Here is my ideas:
Write an helper which will check for every question if a voted has been casted
That's means that the number of queries will depends on the number of items displayed on the page (usually ~20)
Loop on my items get the ids and for each page write a query which will returns if a vote has been casted or NULL
Looks ok because only one query doesn't matter how much items on the page but may be break some MVC/Domain Model design, dunno.
When User log in (or a guest for whom an anonymous user is created) retrieve all votes, store them in session, if a new vote is casted, just add it to the session.
Looks nice because no queries is needed at all except the first one, however, this one and, depending on the number of votes casted (maybe a bunch for each user) can increase the size of the session for each users and potentially make the authentification slow.
How do you do? Any other ideas?
For eg : Lets assume you have a table to store votes and the user who cast it.
Lets assume you keep votes in user_votes when a vote is cast with a table structure something like the below one.
id of type int autoincrement
user_id type int, Foreign key representing users table
question_id type of int, Foreign key representing questions table
Now as the user will be logged in , when you are doing a fetch for the questions do a left join with the user_id in the user_votes table.
Something like
SELECT q.id, q.question, uv.id
FROM questions AS q
LEFT JOIN user_votes AS uv ON
uv.question_id = q.id AND
uv.user_id = <logged_in_user_id>
WHERE <Your criteria>
From the view you can check whether the id is present. If so mark voted, else not.
You may need to change your fields of the questions table and all. I am assuming you store questions in questions table and users in user table so and so. All having the primary key id .
Thanks
You could use a combination of your suggested strategies.
Retrieve all the votes made by the logged in user for recent/active questions only and store them in the session.
You then have the ones that are more likely to be needed while still reducing the amount you need to store in the session.
In the less likely event that you need other results, query for just those as and when you need to.
This strategy will reduce the amount you need to store in the session and also reduce the number of calls you make to your database.
Just based on the information than you've given so far, I would take the second approach: get the IDs of all the items on the page, and then do a single query to get all the user's votes for that list of item IDs. Then pass the collection of the user's item votes to your view, so it can render items differently when the user has voted for that item.
The other two approaches seem like they would tend to be less efficient, if I understood you correctly. Using a view helper to initiate an individual query for each item to check if the user has voted on it could lead to a lot of unnecessary queries. And preloading all the user's voting history at login seems to add unnecessary overhead, getting data that isn't always needed and adding the burden of keeping it up to date for the duration of the session.
Is it possible to invalidate memcahced entries based on values.
In my app, I am assigning user in different groups and I store this mapping in memcached.
key = userID
value = groupID
So multiple userIds map to one groupID.
When I delete a particular group then I want to remove all entries in memcache store which have value as the groupID of the deleted group.
So Essentially I want to delete the entries having particular values. How do I do it.
You cannot get key's by value in memcached. What you could do though is have a key called groupID that has a comma separated userID's. If you want to see who is part of a group then you could get the key called groupID and parse out the userID's. Then if you have a key for each userID you could delete them with the parsed userID's. You could use memcached's append function too to just append the userID's to a groupID key when a new user registers for you system.
I wrote a blog post on maintaining a set a while back that may do what you're looking for.
It's essentially what mikewied is suggesting, but with more words and code samples.