We are developing an app that basically works similarly to Instagram, but with groups. Users can invite other users to groups. The members in a group share posts containing photos, media etc. A user has a feed with all the posts that have been posted recently in the groups, with paging. The users can also access posts from within the groups.
The feed query is slow due to the nature of our current schema design. In practice, request time will increase as the amount of data increases, which is not very scalable now when we see more users on the platform.
Question 1 How should we model our data, in the best possible way, using MongoDB to make our feed query faster and more scaleable?
Our current backend is built using Parse Server with MongoDB and Node.js
A simplified version of our schema is as follows:
Class/Document
- Attribute
Users
- id
Group
- id
- members (array of user pointer objects)- basically an array of ids
- admins (array of user pointer objects) - basically an array of ids
- createdBy (pointer to user object) - basically an id of the owner of the group
Posts
- id
- groups (array of group pointer objects) - basically an array of ids
So right now, when we want to get the posts in a user's feed we have to go through the following process:
Go through all groups, checking the group's members/admins arrays for the user, and get all groups for the user
Go through all posts, checking the post’s groups array to see if post should be in the feed.
Possible solution to Question 1 (MongoDB - Many-to-many relationship?)
This would mean:
Have a group array (pointers to group objects) in the User class, that points to groups where the user is admin/member or owner. Have, like before, the member/admin arrays and createdBy in the Group class.
Have a post array (pointers to post objects) in the Group class. Have, like before, the groups array in the Post class.
This would translate into the following schema:
Class/Document
- Attribute
Users
- id
- groups (array of group pointer objects, where the user is either admin/member or owner) - basically an array of ids
Group
- id
- members (array of user pointer objects)- basically an array of ids
- admins (array of user pointer objects) - basically an array of ids
- createdBy (pointerObject to user) - basically an id of the owner of the group
- posts (array of post pointer objects) - basically an array of ids
Posts
- id
- groups (array of group pointer objects) - basically an array of ids
Meaning, you would have two-way query ability. Downside is that you have to be careful to keep these arrays consistent with one another.
Question 2 Is it sensible to use MongoDB in the long run for our type of application or should we aim to migrate to, for example,
PostgreSQL? (I’m a bit biased as I’ve been working more with RDMS-
than NoSQL databases)
We will also continue extend our statistics interface for showing different activities and their stats in the groups and we are predicting that we will have more many-to-many situations, or few-to-few relationships. We have started to experiment, adding PostgreSQL to the mix. Mirroring the data in MongoDB with PostgreSQL and use SQL for some of the above queries that didn’t scale well with our current schema design in MongoDB. We are also thinking of gradually moving over to PostgreSQL. However, this might be a premature decision, if we can make everything work well with MongoDB. As we will continue to extend our statistics interface, is it better to move over to using a RDMS like PostgreSQL?
All help is appreciated.
Have a great day!
Related
It's my first time working with FireStore. I'm working on a ridesharing app with Flutter that uses Firebase Auth where users can create trips and offer rides similarly to BlaBlaCar, where other users can send requests to join a ride. I’m having difficulty not only deciding the potential collections and paths to use, but also how to even structure it.
For simplicity at this stage, I want any user to be able to see all trips created, but when they go to their “My Rides” page, they will only see the rides that they’ve participated in. I would be grateful for any kind of feedback.
Here are the options I’ve considered:
Two collections, “Users” and “Trips”. The path would look something like this:
users/uid and trips/tripsId with a created_by field
One collection of “Users” and a sub-collection of “Trips". The path seems to make more sense to me, which would be users/uid/trips/tripId but then I don't know how other users could access all the rides on their home feed.
I'm inclined to go with the first option of two collections. Also very open to any other suggestions or help. Thanks.
I want any user to be able to see all trips created, but when they go
to their “My Rides” page, they will only see the rides that they’ve
participated in
I make the assumption that participating in a ride is either being the author or being a passenger of the ride.
I would go for 2 collections: one for users and one for trips. In a trip document you add two fields:
createdBy with the uid of the creator
participants: an Array where you store the author's uid and all the other participants uids (passengers)
This way you can easily query for:
All the rides
All the rides created by a user
All the rides for which a user is a participant, using arrayContains.
(Regarding the limit of 1 MiB for the maximum size for a document I guess this is not a problem because the number of passengers of a ride shouldn't be so huge that the Array fields size makes the document larger than 1 Mib!)
Note that the second approach with subcollections could also be used since you can query with collections group queries but, based on the elements in your question, I don't see any technical advantage.
With MongoDb how can I restrict access to documents - based on user defined groups?
The scenario is a social app like a blog where users create content and others can comment on it. But I need to allow the content creators to be able to limit access to content (posts) based on specific groups they define (eg Public, Family, Work, Private) where each group also has an access level eg none/read/write.
I'm fairly new to MongoDB/NoSQL. In a relational db sense I would simply create a join table (linking the many-to-many relationship between user-groups and posts then use an inner join when querying the data.
Thanks.
Suppose i have company collection, and company has members, members of company may be registered in my application as a user or may be not a registered user. So now how i can store member details such as first_name, last_name, id of member in case of registered members inside company.members. I have something like this inside company:
Suppose company has two member: member A is a registered member in my application and i have his/her id, profilePic, and member B is not registered, so i have only his/her first_name, last_name.
As usual there is more than 1 answer for this question and it depends on how your application will be used, the simplest way is:
1) on company document set a array with your users straight way, this is good for university projects and to proof of concept
In a real world solution I'd split the documents in 2 or more collections, so:
2) on company document set an array with the object id o the user collection, this is useful in case you need to work often with the company or user collection, a separate collection could be used for each registered or unregistered users
3) big trucks need big engine, so if you really need to work hard with company and user documents, like including new user, companies or even changing them from a group to another, registering a new account and them including them to your system in a way often I would suggest using a collection just for the groups
MongoDB does an amazing jog when we talk about reading operation, so the idea is getting its work easier for that, if you split the writing job mongo will work better on that, so managing little chunks of data each time is best
so I'm working with a database that has multiple collections and some of the data overlaps in the collection . In particular I have a collection called app-launches which contains a field called userId and one called users where the _id of a particular object is actually the same as the userId in app-launches. Is it possible to group the two collections together so I can analyze the data? Or maybe match the the userId in app-launches with the _id in users?
There is no definit answer for your question Jeffrey and none of the experts here can tell you to choose which technique over other just by having this information.
After going through various web pages over internet and mongo documentation and understanding the design patterns used in Mongo over a period of time, How I would design it depends on few things which I can try explaining it here in short.
if you have a One-To-One relation then always prefer to choose Embedding over Linking. e.g. User and its address (assuming user has only one address) thus you can utilize the atomicity (without worrying about transactions) as well easily fetch the records without too and fro to bring other information as in the case of Linking (like in DBRef)
If you have One-To-Many relation then you need to consider whether you can do the stuff by using Embedding (prefer this as explained the benefits in point 1). However, embedding would help you if you always want the information altogether e.g. Post/Comments where your requirement is to get the post and all of its comments by postId let say. But think of a situation where you need to get all the comments (and it related posts) which contains some specific tags in comments. in this case you should prefer Linking Because if you go via Embedding route then you would end up getting all the collection of comments for a post and you have to filter the desired comments.
for a Many-To-Many relations I would prefer two separate entities as well another collection for linking them e.g. Product-Category.
-$
I have two collections - Tickets and Users. Where a user can have one to many tickets. The ticket collection is defined as follows
Ticket = {_id, ownerId, profile: {name}}
The ownerId is used to find all tickets that belong to a specific person. I need to write a query that gets me all users with no tickets.
How can i write this query without having to loop through all users, checking if the userID shows up in any Tickets?
Would a bidirectional storage cause me any performance problems ? For example, if i were to change my users collection and add an array of tickets: [ticketID, ticketID2, ...]?
I'd go with the array of tickets being stored in users. As far as I know, Mongo doesn't really have a way to query one collection based on the (lack of) elements in another collection. With the array, though, you can simply do db.users.find({tickets:[]}).