I am building a personal work/career portfolio web app project, and plan on using MongoDB for my database. (I plan to build the project using MERN stack.) Most of my data is not one-time data (such as education, and work experiences), however I have a few pieces of data (such as my personal summary (the content for my "About Me" section), and skills summary) that are one-time only data (I think "single instance" might be a better fitting term). I would like to store all of the data in a database, and set up an admin-end to manage and edit the data. However, I am not sure how to go about storing the one-time data in my MongoDB database.
One idea I had was to create a collection solely for the one-time data, and only allow the user (me) to update and read the documents in the collection. Another idea I had was placing all of my portfolio data into a single collection called "entries", and giving each "entry" a type (such as "Education", or "Personal Summary"). Then when I retrieve the data from the collection I would gather all the documents with the same value in their type field together. I was thinking of storing each of the types as a constant on my server. However, my biggest concern with both ideas is if they would be considered bad practice of not.
I would be very appreciative if anyone has any advice on how to solve this problem.
I had implemented this a while back on one of my small projects, and again after discussing it over with some professionals I'm in contact with, they said that the best approach would be to create a collection with a single document that contains all the information, like the links, about, etc...
One more thing I, was suggested is that we could use Redis solely for the purpose of storing this type of information as well.
Something that I implemented a long time back similar to the one collection, single doc approach: https://github.com/codelancedevs/Sundar-Clinic/tree/local-backend/src/api/app
Working on a similar approach here: https://github.com/kunalkeshan/Cam-O-Genics-Backend
Hope this is of some help, I'm still learning as to what might be the best approach. Open to any suggestions out there!
Related
I am working on a react-native app using nodejs and mongodb on the backend. In my app users are able to create multiple accounts and there are different account types (Business, Artist, Venue, etc.). For the most part, each account type has the same data and fields in the database. Things like name, location, website. But it is possible for each account type to have a couple pieces of data specific to that account. My question is, should I simply have one "Account" collection in the database that stores all accounts and has an "accountType" field to differentiate each account?
Initially I thought to do the opposite and store each account type in a separate collection, but I found it made the client code pretty messy as I found myself adding a bunch of if statements to determine things like what api endpoint to make requests to, what components to render, and what screens to navigate to, when in reality, it's really just a couple pieces of data that may change from one account type to another.
It seems like having just one "Accounts" collection with an "accountType" field will greatly simplify the code. But maybe there is something I am missing. If anyone has some insight as to which approach may be better for the situation, or some of the pros/cons of each approach, I'd really appreciate the help! Thanks!
Well, the answer clearly depends on how the documents for different types of accounts differ. But, the idea of going with a single collection is fine, also take a look at the Subset Pattern, it's will give you a fine idea, of how to divide data into different collections, depending on their usage.
So I'm in the middle of planning a little web app that will require quite large amounts of data stored on a user level, in one case, the system would take a large object from a system level and make a "user specific" version, a user can have multiple ones of these. Simplest would be to compare it to a form stored in a google spreadsheet, where the user is expected to use the template spreadsheet, then change not only the answers but also the question.
Security wise I am quite OK
In the second case there is requirement to store multiple objects, size about 250k to maybe 3mb, once again on a user specific level, with a potential to move it to a system level so additional users can access it. As an example, say the user can upload pictures, but may not want to share all of them. However, a user may choose to "publish" a small number of them because they are happy with those specific pictures.
What design patterns should I consider using specifically around web apps where the user have decent amounts of data? For example, would it make most sense to use a single large database and have a table that keeps track of resources or create separate tables per user?
I have considered putting it all in a mongo database.
Your approach may be wrong.
If you want to store user based binary data and make it accessible for the user itself or the community, you would need a hierarchic structure like so:
userid1
pic1,pic2,pic3
userid2
pic4,pic5,pic6
community
pic7,pic8
You could then grant read permissions to "community" for all users, and permission for each user to its own directory.
Usually there is nothing wrong using a database to store binary files if you consider partitioning, role permissions and an applicable interface to access the data.
My suggestion is to use a binary repository like Artifactory.
It provides hierarchic structures, simple search queries using HTTP requests and has caching abilities for frequently queried objects.
I also think that http requests are a lot easier to use and also there is an abstraction layer to the data which is more secure.
Artifactory is free.
I am working on an application on which we'll have users and videos.
It's a n-n relationship, a user can be related to several videos, and the same video can be related to several users.
I decided to go with mongoDB on the implementation, though I wasn't familiar with this technology at first, so I run into a problem regarding the document data model (in contraposition with the entity-relation data model).
In this application I'll need to access frequently the videos that are somehow related to a certain user. From this point of view, it would be logical to embed the document 'video' in the document 'user'.
But, I will also need frequent access to video collections, regardless of the users related to them. From this point of view, it seems the data model would be better designed if the the users related to the video were embedded inside its document.
Both designs make sense, and solve a problem, but make the remaining problem quite hard to solve; I would have to perform complex, inefficient queries to actually be able to get both functionalities with any of those two designs.
Right now I think the best decission would be to implement it the same way I would in a relational database (with two different documents for users and videos, and an intermediate document that allows me to know the relations between those two).
I'm really not sure that is the way this problem should be solved in mongoDB, so I would like to ask for advice regarding the data model design.
Thanks in advance.
Do both.
While redundancy should be avoided in a relational database, the same is not true for a document-oriented database. When you have no JOINs, you need to make sure that every common query can be fulfilled with documents from a single collection. Redundancy is usually the only way to achieve this.
The downside is that you now need two queries to update the relation, because both the video and the user document need to be updated. But that's a small price to pay, especially considering that updates are usually not as performance-critical as reads (you can perform them in the background while faking the result on the frontend for the user who requested the update).
this is a question on best practice, i understand that there are a lot of different options for doing this, but i would like your opinions as to how you would approach solving this problem. Please take it as though performance is critical in this system, in other words scalable.
I have recently found the wonders of graph database, so i came up with a theoretical situation where a company wants to manage it's customers relationships, and in order to do so they are going to use neo4j which is great, and allows for really great management of the customers, different staff members and their relationships, which is all great, however the company now wants to create a web based interface which will need authentication, and anyone in the neo4j database should be able to login to the system in order to see how they are related to other people in the company's database, so each user must have a password/email/id associated with their name.
So my question is, in this case scenario, is it best to store the password_hash/password_salt/id/email in a mysql database and then based on the node look it up on the mysql database. Or is it better to store the password_hash/password_salt/id/email in the hash tables inside the nodes.
Also each store has 1000s of products, and they can be stored in the graph database or i can store the products in the mysql database and then look up the product there, and do the changes there, because the products are not related to each other, so no point in storing them in the graph database, so should they be not stored there to improve performance?
So my question boils down to this: is it best for large projects to use a graph database along with the more common rdms database such as mysql? if not, then what is the point at which you start to use these two database systems?
apologies in advance for my lack of knowledge regarding database terminology.
Graph DB is mainly used for maintaining relations. If app has a graph DB that does not mean that app needs to store everything in Graph DB.
Every node request on Graph is in memory and thus if you have unnecessary properties in your node it will be bloated and may make things slower and take more memory.I usually decide what needs to go in graph and what needs to go in DB by very simple rule.
High level property (that defines the relation and other important properties that defines the node) goes in graph whereas additional information goes in RDMS.
For example in FB may be FBID, Name goes in Graph as it defines the relationship of one node with another. But when user clicks on someones facebook ID, he/she gets to see other users DOB, Age , College .All these can go in RDBMS.
PS: RDMS has another advantage, it can be used for quick analytics. I know with graph also you can do that but i am not sure if its as scalable and easy as RDBMS.
Downside to this approach is : You need to maintain two DBS.
Unless you have a proven case for a two-DB solution, I'd say fewer moving parts would keep you more agile, more able to change things quickly. If later you find a use case that is difficult, then weigh up the cost/ benefit of introducing a second storage. A two-DB architecture is not unheard of, but comes with an overhead.
Specific to security, there is no reason why Neo4j or any other reasonable NOSQL solution couldn't do that: http://spring.neo4j.org/docs#tutorial_security
You should use both in case there is data where it does not make much sense to store it in a graph DB such as neo4j/orientDB (and some data would be better off in a graph DB as opposed to a relational DB). Forcing data on one platform may cause issues with performance/scalability down the line.
I'm trying to model a simple, experimental app as I learn Symfony and Doctrine.
My data model requires some flexibility, so I'm currenty looking into the possibility of using either an EAV model, or document store in MongoDB.
Here's my basic requirements:
Users will be able to store and share their favourite things (TV prog, website, song etc).
The list of possible 'things' a user can store is unknown. For example, a user may want to store their favourite animal.
Users can share their favourite things with other users. However, a user can decide what he / she shares with each other user. For example, a user may share their favourite movie with one user, but not another.
A typical user will log in and view all the favourite things from their list of friends, depending on what his friends have decided to share. The user will also update their own favourite things, which will be reflected when each other users views their own profile. Finally, the user may change which of his friends can see what of his favourite thing.
I've worked a lot with Magento, which uses the EAV model extensively. However, I'm adding another layer of complexity by restricting which users can see what information.
I'm instantly drawn to MongoDB as the schemaless format gives me the flexibility I require. However, I'm not sure how easy (or efficient) it will be to access the data once it's saved. I'm also concerned about how changes to the data will be managed, e.g. a user changes their favourite film.
I'm hoping someone can point me in the right direction. This is purely a demo app I'm building to further my knowledge, but I'm treating it like a real-world app where data access times are super-important.
Modelling this kind of app in a traditional relational DB makes me sweat when I think about the crazy number of joins I'd need to get the data for one user.
Thanks for reading this far, and please let me know if I can provide anymore information.
Regards,
Fish
You need to choose a model based on how you need to access the data.
If you just need to filter out some values when viewing the user profile, a single document for each user would work quite well, with each favorite within that having a list of authorized user/group IDs that is applied in the application code. Both read and write are single operations on a known document in this case, so will be fast.
If you need views across multiple profiles though, your main document should probably be the favorite. You'll need to set up the right indexes, but performance shouldn't be a problem.
Actually, the permissions you describe don't add that much complexity to an EAV schema - as long as attributes can have multiple values the permissions list is just one more attribute.