Lets say I have a social media app. There is a Group model that has a field called invitedUsers which is simply an array of user ids that are a part of that group.
On my backend I have a route that a user hits to join that Group. In that route I do something like the following:
group.invitedUsers = lodash.concat(group.invitedUsers || [], userId)
group.save()
where group is the group that the user wants to join and userId is the id of the user that wants to join the group. Upon save everything is updated properly and the user is now a part of the group.
But what happens if two users hit the route at exactly the same time? How does MongoDB ensure that the group will always have both users ids added via the above method. Is there not a chance that group.invitedUsers could be referencing a stale value if both these group.save() are being triggered around the same time?
I've been trying to implement live messaging in my application and I cannot seem to think of a convenient data structure inside Firestore. My current structure looks like this:
collection("conversations").document(id).collection("messages")
Each document holds two attributes user1 and user2 with nicknames of contributors to the conversation. Each document also owns a collection called messages which holds documents where each represents a single message sent with some info.
What I'm trying to do next is to check if the conversation already exists, if not then create it. The problem for me is write a correct query to find out if it exists.
My first idea was: create users array instead which holds nicknames of users and then simply query:
db.collection("conversations").whereField("users", in: ["username1", "username2"])
Problem with this is that it means "where users contains username1 OR username2", but I need it to contain "username1 AND username2".
I tried to be smart and chain the whereField function as following:
db.collection("conversations").whereField("users", arrayContains: "username1").whereField("users", arrayContains: "username2")
Turns out that you cannot use arrayContains more than once in a single query.
After that I came back to the structure as displayed on the screenshot with user1 and user2 and ran a new query:
db.collection("conversations").whereField("user1", isEqualTo: user).whereField("user2", isEqualTo: friend)
This query is ran in a function where user and friend are string parameters holding nicknames of both sender and receiver of the message we're currently sending. Imagine you are sending a message, user is always going to be your nickname and friend the receiver's one. The problem with the query is that you're nickname might be saved under user1 or user2 and receiver's nickname aswell. In either of those situations the conversation exists. How would I have to change the query since I don't know in an advance who will have which position in the query aswell as in Firestore. Running the last query that I included twice while switching user and friend parameter seems very unconvenient.
Any tips or solutions to progress in this problem will be much appreciated!
I currently get to work with DynamoDB and I have a question regarding the structure I should choose.
I setup Twilio for being able to receive WhatsApp messages from guests in a restaurant. Guests can send their feedback directly to my Twilio WhatsApp number. I receive that feedback via webhook and save it in DynamoDB. The restaurant manager gets a Dashboard (React application) where he can see monitor the feedback. While I start with one restaurant / one WhatsApp number I will add more users / restaurants over time.
Now I have one of the following two structures in mind. With the first idea, I would always create a new item when a new message from a guest is sent to the restaurant.
With the second idea, I would (most of the time) update an existing entry. Only if the receiver / the restaurant doesn't exist yet, a new item is created. Every other message to that restaurant will just update the existing item.
Do you have any advice on what's the best way forward?
First idea:
PK (primary key), Created (Epoc time), Receiver/Restaurant (phone number), Sender/Guest (phone number), Body (String)
Sample data:
1, 1574290885, 4917123525993, 4916034325342, "Example Message 1" # Restaurant McDonalds (4917123525993)
2, 1574291036, 4917123525993, 4917542358273, "Example Message 2" # different sender (4917542358273)
3, 1574291044, 4917123525993, 4916034325342, "Example Message 3" # same sender as pk 1 (4916034325342)
4, 1574291044, 4913423525123, 4916034325342, "Example Message 4" # Restaurant Burger King (4913423525123)
Second idea:
{
Receiver (primary key),
Messages: {
{
id,
Created,
From,
Body
}
}
}
Sample data (same data as for first idea, but different structured):
{
Receiver: 4917123525993,
Messages: {
{
Created: 1574290885,
Sender: 4916034325342,
Body: "Example Message 1"
},
{
Created: 1574291036,
Sender: 4917542358273,
Body: "Example Message 2"
},
{
Created: 1574291044,
Sender: 4916034325342,
Body: "Example Message 3"
}
}
}
{
Receiver: 4913423525123,
Messages: {
{
Created: 1574291044,
Sender: 4916034325342,
Body: "Example Message 4"
}
}
}
If I read this correctly, in both approaches, the proposal is to save all messages received by a restaurant as a nested list (the Messages property looks like an object in the samples you've shared, but I assume it is an array since that would make more sense).
One potential problem that I foresee with this is that DynamoDB documents have a limitation on how big they can get (400kb). Agreed this seems like a pretty large number, but you're bound to reach that limit pretty quickly if you use this application for something like a food order delivery system.
Another potential issue is that querying on nested objects is not possible in DynamoDB and the proposed structure would mostly involve table scans for any filtering, greatly increasing operational costs.
Unlike with relational DBs, the structure of your data in document DBs is dependent heavily on the questions you want to answer most frequently. In fact, you should avoid designing your NoSQL schema unless you know what questions you want to answer, your access patterns, and your data volumes.
To come up with a data model, I will assume you want to answer the following questions with your table :
Get all messages received by a restaurant, ordered by timestamp (ascending / descending can be determined in the query by specifying ScanIndexForward = true/false
Get all messages sent by a user ordered by timestamp
Get all messages sent by a user to a restaurant, ordered by timestamp
Consider the following record structure :
{
pk : <restaurant id>, // Partition key of the main table
sk : "<user id>:<timestamp>", // Synthetic (generated) range key of the main table
messageBody : <message content>,
timestamp: <timestamp> // Local secondary index (LSI) on this field
}
You insert a new record of this structure for each new message that comes into your system. This structure allows you to :
Efficiently query all messages received by a restaurant ID using only the partition key
Efficiently retrieve all messages received by a restaurant and sent by a user using pk = <restaurant id> and begins_with(sk, <user id>)
The LSI on timestamp allows for efficiently filtering messages based on creation time.
However, this by itself does not allow you to query all messages sent by a user (to any restaurant, or a specific restaurant). To do that we can create a global secondary index (GSI), using the table's sk property (containing user IDs) as the GSI's primary key, and a synthetic range key that consists of the restaurant ID and timestamp separated by a ':'.
GSI structure
{
gsi_pk: <user Id>,
gsi_sk: "<dealer Id>:<timestamp>",
messageBody : <message content>
}
messageBody is a non key field projected on to the GSI
The synthetic SK of the GSI helps make use of the different key matching modes that DynamoDB provides (less than, greater than, starts with, between).
This GSI allows us to answer the following questions:
Get all messages by a user (using only gsi_pk)
Get all messages by a user, sent to a particular restaurant (ordered by timestamp) (gsi_pk = <user Id> and begins_with(gsi_sk, <restaurant Id>)
The system has a some duplication of data, but that is in line with one of the core ideas of DynamoDB, and most NoSQL databases. I hope this helps!
Storing multiple message in a single record has multiple issues
Size of write to db will increase as we go. (which will translate to money and response time, worst case you may end up hitting 400kb limit.)
Race condition between multiple writes.
No way to aggregate messages by user and other patterns.
And the worse part is that, I don't see any benefit of storing multiple messages together. (Other than may be I can query all of them together, which will becomes a con as size grows, like you will not be able to do get me last 10 reviews, you will always have to fetch all and then fetch last 10.)
Hence go for option where all the messages are stored differently.
I'm currently making a chat module that counts the number of messages that the user has not read. The user has a child "LatestReadMessage" with the ID of the latest message he has seen before exiting the viewController. What I want is to count the ID's below the value of "LatestReadMessage". To make things clearer here is an example:
"LatestReadMessage" has the value of: -LfZ1gZ7EUCosK9JlZ0b
The id's inside of my messages node are:
messages:
-LfZ1fdWF-AVQMnL9b6y
-LfZ1gZ7EUCosK9JlZ0b //the id of LatestReadMessage
-LfZ1kXNajjLNZXBi_Bh
-LfZ1nMx1M8soMYJ5Da7
-LfZ1o_1mUYS8QN3KbyX
-LfZ1q61SMR96l7L0BOb
-LfZ1rdSkmfL8OLVAq7K
What I want to get is the number of ID's below -LfZ1gZ7EUCosK9JlZ0b (e.g -LfZ1kXNajjLNZXBi_Bh, -LfZ1nMx1M8soMYJ5Da7 and etc.) is there a way for me to filter and count the ID's below it?
my flow:
User A selects user B in the user list:
system needs to check if a room for these two users exists, if not create unique room name and then join both users to the room
if exists, then just join users to the room they were already in and populate the chat with previous msges
Now what I am stuck at is how to exactly do it. Few options I am playing with in my head:
a) First how do i create the unique name that ties both users? Sure I can use string combination for both users, for example user A clicks user B --> "A&B", but this won't work when user B clicks user A, because that will be "B&A". I am struggling with creating dynamic unique names that could be applied to both.
b) do I keep an array with the two users info in the specific room saved in DB, and then check the array if user exists in it already? if so just use that room id as the room name? What is the best flow to save created rooms? Do i save by room name, which I guess would act as unique Id as well?
c) should I be checking the DB EVERYTIME user clicks another user to start a chat just to check if a room exists or not?
I know how to create rooms and all that jazz but what I am really struggling with is how to dynamically create room names so that its the same whether A clicks B or B clicks A and how to from a pseudo code level, store created rooms in DB and check for many users.
Here's an idea: Store the room in your database as a document that contains fields user1 and user2, which will contain the IDs of these users. Specifically, ensure that user1 < user2. When you need to query for this document later, you can do db.rooms.findOne({user1: smallerId, user2: largerId}). Then you can either store the room name and not use it in your queries, or you can even generate the displayed room name dynamically at runtime.
This has the benefit of not only guaranteeing the structure of a room document, but making your queries more efficient as well (you're comparing binary vs. comparing strings). There's also the benefit of not breaking the query when a user's name changes.
In general it's recommended that a document A that's associated with a different document B should refer to document B by an immutable ID, rather than by a mutable name. In this case since a room is associated with two users, have room refer to each user's ID.