Are there any best practices or better yet existing software, which would manage description of the current schema and schema migrations of an REDIS database?
What I'd expect:
concise, human and machine readable schema
concise, human and machine readable schema migration
optionally, a software that would manage and apply migrations
Of course I'm not looking to get 100% of functionality of tools such as South, only it's subset.
What do I mean by "schema" in a schema-less DB? Assumption is that I'm basically mapping my objects to DB. For example I follow pattern such that I've got set somethings of ids and hashes something:<id> containing values of the fields, some of which might actually be referencing other objects.
So an example of this would look like this:
set authors = { carlson, macedo, olivera }
hashes autor:*
author:carlson = {first_name: Josiah, last_name: Carlson, initial: L}
author:macedo = {first_name: Tiago, last_name: Macedo}
author:olivera = {first_name: Fred, last_name: Olivera}
set books = { rediscook, redisaction }
hashes book:*
book:rediscook = {title: "Redis Cookbook", publisher: "O'Reilly"}
book:redisaction = {title: "Redis in Action", publisher: "Manning"}
sets book:authors:*
book:authors:rediscook = {macedo, olivera}
book:authors:redisaction = {carlson}
What I'm looking for is more formal way to describe above.
Related
I need store and build fast query for next structure:
class Model {
id: number:
alias: string;
schema: Record<string, any>;
}
where schema it's can be:
{
someField: '$model_alias',
otherField: {
nestedField: '$other_model_alias'
}
}
Example data:
{ id: 1, alias: "model_one", schema: { field1: "test", field2: "demo" } }
{ id: 2, alias: "model_second", schema: { someField: "$model_one", otherField: { nestedField: 5 } }
{ id: 3, alias: "model_third", schema: { field5: "$model_second", field6: "$model_one", field7: "$model_fourth" } }
{ id: 4, alias: "model_fourth", schema: { field8: "$model_second" } }
As you can see, json field schema contains fields which may refer to another models with schemas. Thus, there can be a lot of nesting, and relationships can be many-to-many.
Is it possible to achieve such a structure with Postgres or should some alternative be used? I need possible to easy manage structure and very fast queries (get tree children or get tree parents).
Thanks.
Choosing the right db type is tricky at the best of times. Can you provide more information about what sorts of queries you'd be doing? And how big is your dataset?
If your requirement is to exclusively get the parents and children of a model, a relational db such as postgres would do it. If the relations are many-to-many, you'll have a bridging table (https://dzone.com/articles/how-to-handle-a-many-to-many-relationship-in-datab), and will be able to do efficient queries on that.
If you're doing significant, multi-hop traversals between the relationships, you might indeed want to look at a graph database to avoid expensive joins. Postgres even has a plugin that allows this: https://www.postgresql.org/about/news/announcing-age-a-multi-model-graph-database-extension-for-postgresql-2050/
I wouldn't recommend a document store for data that's heavily relational like this, just because managing relationships between documents has to be handled manually by the user, and that's normally more trouble that it's worth.
In a Mongo environment it is beneficial to embed data in documents.
so for example an Employees document:
{
{
userid: 'someid',
username: 'user1'
isManager: true,
subordinates: [
{
userid: 'anotherid',
username: 'user2',
isManager: false
}
],
officeLocation: {
officeId: 'someofficeid',
officeName: 'Some Office'
}
},
{
userid: 'anotherid',
username: 'user2',
isManager: false,
officeLocation: {
officeId: 'someotherofficeid',
officeName: 'Some Other Office'
}
}
}
And the office document:
{
{
officeid: 'someofficeid',
officeName: 'Some Office'
},
{
officeid: 'someotherofficeid',
officeName: 'Some Other Office'
}
}
So lets assume that someone in the company decides that they don't like the name Some Other Office and they want to change it to Some Cool Office.
When they make the change in the office document how do we know to update all the embedded Some Other Office in the employee document as well?
It seems that every time that you take a piece of data from one document and embed it into an object in another document that the link between the two gets broken and then you have to write separate queries to update the data in all the different spots that you embedded that object into.
I like the idea of embedded documents rather than storing references, but without some kind of 2 way data-binding it seems impractical when it comes to updating information.
Is there any way that I would be able to bind the data two ways or is there an easier way to go about modeling my data?
Thanks
It remainds me about the traditional RDBMS systems when you model to normalize/denormalize an information. I'm not sure about the binding, but, if you need the "single true" for an information, the better way is never having the information stored in two different places. So, in your case, it may be better to store the Office information into a separated document and just link it by Id.
I would like to create an eCommerce type of database where I have products and categories for the products using Mongodb and Mongoose. I am thinking of having two collections, one for products and one for categories. After digging online, I think the category should be as such:
var categorySchema = {
_id: { type: String },
parent: {
type: String,
ref: 'Category'
},
ancestors: [{
type: String,
ref: 'Category'
}]
};
I would like to be able to find all the products by category. For example "find all phones." However, the categories may be renamed, updated, etc. What is the best way to implement the product collection? In SQL, a product would contain a foreign key to a category.
A code sample of inserting and finding a document would be much appreciated!
Why not keep it simple and do something like the following?
var product_Schema = {
phones:[{
price:Number,
Name:String,
}],
TV:[{
price:Number,
Name:String
}]
};
Then using projections you could easily return the products for a given key. For example:
db.collection.find({},{TV:1,_id:0},function(err,data){
if (!err) {console.log(data)}
})
Of course the correct schema design will be dependent on how you plan on querying/inserting/updating data, but with mongo keeping things simple usually pays off.
I'm using MongoDB. I know that MongoDB isn't relational but information sometimes is. So what's the most efficient way to reference these kinds of relationships to lessen database load and maximize query speed?
Example:
* Tinder-style "matches" *
There are many users in a Users collection. They get matched to each other.
So I'm thinking:
Document 1:
{
_id: "d3fg45wr4f343",
firstName: "Bob",
lastName: "Lee",
matches: [
"ferh823u9WURF",
"8Y283DUFH3FI2",
"KJSDH298U2F8",
"shdfy2988U2Ywf"
]
}
Document 2:
{
_id: "d3fg45wr4f343",
firstName: "Cindy",
lastName: "Doe",
matches: [
"d3fg45wr4f343"
]
}
Would this work OK if there were, say, 10,000 users and you were on Bob's profile page and you wanted to display the firstName of all of his matches?
Any alternative structures that would work better?
* Online Forum *
I supposed you could have the following collections:
Users
Topics
Users Collection:
{
_id: "d3fg45wr4f343",
userName: "aircon",
avatar: "234232.jpg"
}
{
_id: "23qdf3a3fq3fq3",
userName: "spider",
avatar: "986754.jpg"
}
Topics Collection Version 1
One example document in the Topics Collection:
{
title: "A spider just popped out of the AC",
dateTimeSubmitted: 201408201200,
category: 5,
posts: [
{
message: "I'm going to use a gun.",
dateTimeSubmitted: 201408201200,
author: "d3fg45wr4f343"
},
{
message: "I don't think this would work.",
dateTimeSubmitted: 201408201201,
author: "23qdf3a3fq3fq3"
},
{
message: "It will totally work.",
dateTimeSubmitted: 201408201202,
author: "d3fg45wr4f343"
},
{
message: "ur dumb",
dateTimeSubmitted: 201408201203,
author: "23qdf3a3fq3fq3"
}
]
}
Topics Collection Version 2
One example document in the Topics Collection. The author's avatar and userName are now embedded in the document. I know that:
This is not DRY.
If the author changes their avatar and userName, these change would need to be updated in the Topics Collection and in all of the post documents that are in it.
BUT it saves the system from querying for all the avatars and userNames via the authors ID every single time this thread is viewed on the client.
{
title: "A spider just popped out of the AC",
dateTimeSubmitted: 201408201200,
category: 5,
posts: [
{
message: "I'm going to use a gun.",
dateTimeSubmitted: 201408201200,
author: "d3fg45wr4f343",
userName: "aircon",
avatar: "234232.jpg"
},
{
message: "I don't think this would work.",
dateTimeSubmitted: 201408201201,
author: "23qdf3a3fq3fq3",
userName: "spider",
avatar: "986754.jpg"
},
{
message: "It will totally work.",
dateTimeSubmitted: 201408201202,
author: "d3fg45wr4f343",
userName: "aircon",
avatar: "234232.jpg"
},
{
message: "ur dumb",
dateTimeSubmitted: 201408201203,
author: "23qdf3a3fq3fq3",
userName: "spider",
avatar: "986754.jpg"
}
]
}
So yeah, I'm not sure which is best...
If the data is realy many to many i.e. one can have many matches and can be matched by many in your first example it is usually best to go with relations.
The main arguments against relations stem from mongodb not beeing a relational database so there are no such things as foreign key constraints or join statements.
The trade off you have to consider in those many to many cases (many beeing much more than two) is either enforce the key constraints yourself or manage the possible data inconsistencies accross the multiple documents (your last example). And in most cases the relational approach is much more practical than the embedding approach for those cases.
Exceptions could be read often write seldom examples. For (a very constructed) example when in your first example matches would be recalculated once a day or so by wiping all previous matches and calculating a list of new matches. In that case the data inconsistencies you would introduce could be acceptable and the read time you save by embedding the firstnames of the matches could be an advantage.
But usually for many to many relations it would be best to use a relational approach and make use of the array query features such as {_id :{$in:[matches]}}.
But in the end it all comes down to the consideration of how many inconsistencies you can live with and how fast you realy need to access the data (is it ok for some topics to have the old avatar for a few days if I save half a second of page load time?).
Edit
The schema design series on the mongodb blog might be a good read for you: part1, part2 and part3
I am creating a blog system in Node.js with mongodb as the db.
I have contents like this: (blog articles):
// COMMENTS SCHEMA:
// ---------------------------------------
var Comments = new Schema({
author: {
type: String
},
content: {
type: String
},
date_entered: {
type: Date,
default: Date.now
}
});
exports.Comments = mongoose.model('Comments',Comments);
var Tags = new Schema({
name: {
type: String
}
});
exports.Tags = mongoose.model('Tags',Tags);
// CONTENT SCHEMA:
// ---------------------------------------
exports.Contents = mongoose.model('Contents', new Schema({
title: {
type: String
},
author: {
type: String
},
permalink: {
type: String,
unique: true,
sparse: true
},
catagory: {
type: String,
default: ''
},
content: {
type: String
},
date_entered: {
type: Date,
default: Date.now
},
status: {
type: Number
},
comments: [Comments],
tags: [Tags]
}));
I am a little new to this type of database, im used to MySQL on a LAMP stack.
Basically my question is as follows:
whats the best way to associate the Contents author to a User in the
DB?
Also, whats the best way to do the tags and categories?
In MYSQL we would have a tags table and a categories table and relate by keys, I am not sure the best and most optimal way of doing it in Mongo.
THANK YOU FOR YOUR TIME!!
Couple of ideas for Mongo:
The best way to associate a user is e-mail address - as an attribute of the content/comment document - e-mail is usually a reliable unique key. MongoDB doesn't have foreign keys or associated constraints. But that is fine.
If you have a registration policy, add user name, e-mail address and other details to the users collection. Then de-normalize the content document with the user name and e-mail. If, for any reason, the user changes the name, you will have to update all the associated contents/comments. But so long as the e-mail address is there in the documents, this should be easy.
Tags and categories are best modelled as two lists in the content document, IMHO.
You can also create two indices on these attributes, if required. Depends on the access patterns and the UI features you want to provide
You can also add a document which keeps a tag list and a categories list in the contents collection and use $addToSet to add new tags and categories to this document. Then, you can show a combo box with the current tags as a starting point.
As a final point, think through the ways you plan to access the data and then design documents, collections & indices accordingly
[Update 12/9/11] Was at MongoSv and Eliot (CTO 10gen) presented a pattern relevant to this question: Instead of one comment document per user (which could grow large) have a comment document per day for a use with _id = -YYYYMMDD or even one per month depending on the frequency of comments. This optimizes index creation/document growth vs document proliferation (in case of the design where there is one comment per user).
The best way to associate the Content Authors to a User in the MongoDB, is to take an array in Author Collection which keeps an reference to User. Basically Array because One Content/Book may have multiple Authors i.e. you need to associate one Content to many Users.
The best way for category is to create a different collection in your DB and similarly as above keep a array in Contents.
I hope it helps at-least a little.