Should I use relational database schema (3NF) for designing the data storage for Skygear?
For example, I have to usergroup which contains user, should it be Case A or B below?
A.
User{userId}
Group{groupId, groupName}
Grouping(userId, groupId, role}
B.
Group{userId, groupId, groupName, role}
I think I should be having the following result eventually when passing to the app.
{
"groupName":"Group A",
"groupId":"G123",
"administrator":[
{
"userId":"U123"
},
{
"userId":"U456"
}
],
"creator":{
"userId":"U123"
},
"member":[
]
}
3NF is good for data integrity. Everyone love data integrity, but it may not always the first priority when you design an application.
In the case you provided, I will probably opt for A. B requires update of all Group if groupName is changed.
Beside considering how the data pass to app, try to consider how to update and create too?
When to break 3NF in skygear?
No of user in group.
In 3NF, you will probably use aggregate func. Skygear does not provide it yet, common way it to add an user_count column to group record. And update the number when an user is added/removed.
3NF should be used.
Although skygear does not support table joining. It have a Record Relations Data Type for using a foreign key. (although one level only)
https://docs.skygear.io/guides/cloud-db/data-types/android/
Related
I'm trying to model the following relationships between entities, mainly consisting of a partial, disjoint generalization.
original EERD
'mapped' to relational
Since I didn't need the subclasses to have any particular attributes I decided to use the "single table inheritance" approach, added the "type" field and moved the relationships towards the parent.
After that I had two choices to make:
1- type for the "business type" attribute
2- way to constraint participation to at most one of the 4 relationships based on the type attribute
For the sake of portability and extensibility I decided to implement no.1 as a lookup table (rather than enum or a hardcoded check).
About no.2 I figured the best way to enforce participation and exclusivity constraints on the four relationships would be a trigger.
The problem is that now I'm not really sure how to write a trigger function; for instance it would have to reference values inserted into business type, so I'd have to make sure those can't be changed or deleted in the future.
I feel like I'm doing something wrong so I wanted to ask for feedback before going further; is this a suitable approach in your opinion?
I found an interesting article describing a possible solution: it feels a bit like an 'hack' but it should be working
(it's intended for SQL Server, but it can be easily applied in postgres too).
EDIT:
It consists in adding a type field to the parent table, and then have every child table reference said field along with the parent's id by using a foreign key constraint (a UNIQUE constraint on this pair of fields has to be added beforehand, since FKs must be unique).
Now in order to force the type field to match the table it belongs to, one adds a check constraint/always generated value ensuring that the type column always has the same value
(eg: CHECK(Business_type_id = 1) in the Husbandry table, where 1 represents 'husbandry' in the type lookup table).
The only issue is that it requires a whole column in every subclass, each containing the same generated value repeated over and over (waste of space?), and it may fall apart as soon as the IDs in the lookup table are modified
One of the requirements for our REST interface is that each resource be identifiable by unique fields (aside from the primary identifier). The reason for this is that we want to be able to handle bulk importing data - in which case the client can't know the system generated primary identifiers.
This means we have to be able to reference our resources by unique fields. Using a primary key our read requests look like this:
GET example.com/rest/customers/1
and to get orders related to that customer
GET example.com/rest/customers/1/orders
Now, lets assume two fields in customer identify it uniquely, name ("foo") and businessId ("bar"). Given that, I came up with the following URI to get the orders for this customer:
GET example.com/someotherpath/customers/foo,bar/orders
But I don't like that I have a different path to identify that this is a resource being accessed via unique fields. How would you structure the above query in a RESTful way using unique fields instead of the primary key?
Further, an order looks like this:
{
<SNIP>
"orderId" : "42"
"_links": {
"customer": {
"href" : "rest/customers/1"
"key": [ "foo", "bar" ]
}
},
}
Any issues with allowing client to interchangeably specify href OR key when communicating with the interface?
For the first bit, I just wouldn't do it. If a customer has a unique id (and they should), I wouldn't allow end users to specify N other fields that happen to also uniquely identify the customer. It's messy for the user (which field goes first?) and also messy for you on the back end.
For the second bit, the issue is: what happens when they specify both? Which takes precedence? Are they going to remember? Do you want to have to support both? It's generally a good idea to only allow one way to do any particular thing if you can get away with it.
I have read Eric Evans' Domain Driven Design book and I have been trying to apply some of the concepts.
In his book, Eric talks about aggregates and how aggregate roots should have a unique global id whereas aggregate members should have a unique local id. I have been trying to apply that concept to my database tables and I'm running into some issues.
I have two tables in my PostgreSQL database: facilities and employees where employees can be assigned to a single facility.
In the past, I would lay out the employees table as follows:
CREATE TABLE "employees" (
"employeeid" serial NOT NULL PRIMARY KEY,
"facilityid" integer NOT NULL,
...
FOREIGN KEY ("facilityid") REFERENCES "facilities" ("facilityid")
);
where employeeid is a globally unique id. I would then add code in the backend for access control validation, preventing users of one facility from accessing rows pertaining to other facilities. I have a feeling this might not be the safest way to do it.
What I am now considering is this layout:
CREATE TABLE "employees" (
"employeeid" integer NOT NULL,
"facilityid" integer NOT NULL,
...
PRIMARY KEY ("employeeid", "facilityid"),
FOREIGN KEY ("facilityid") REFERENCES "facilities" ("facilityid")
);
where employeeid is unique (locally) for a given facilityid but needs to be paired with a facilityid to be unique globally.
Concretely, this is what I am looking for:
Employee A (employeeid: 1, facilityid: 1)
Employee B (employeeid: 2, facilityid: 1)
Employee C (employeeid: 1, facilityid: 2)
where A, B and C are 3 distinct employees and...
adding an employee D to facility 1 would give him the keys (employeeid : 3, facilityid: 1)
adding an employee E to facility 2 would give him the keys (employeeid : 2, facilityid: 2)
I see two ways of achieving this:
I could use triggers or stored procedures to automatically generate new employeeids and store the last ids for every facility in another table for quicker access but I am concerned about concurrency issues and ending up with 2 employees from the same facility with the same id.
I could possibly create a new sequence for each facility to manage the employeeids but I fear ending up with thousands of sequences to manage and with procedures to delete those sequences in case a facility is deleted. Is there anything wrong with this? It seems heavy to me.
Which approach should I take? Is there anything I'm missing out on?
I am inferring from your question that you will be running a single database for all facilities, or at least that if you have a local database as the "master" for each facility that the data will need to be combined in a central database without collisions.
I would make the facilityid the high order part of the primary key. You could probably assign new employee numbers using a simple SELECT max(employeeid) + 1 ... WHERE facilityid = n approach, since adding employees to any one facility is presumably not something that happens hundreds of times per second from multiple concurrent sources. There is some chance that this could generate an occasional serialization failure, but it is my opinion that any database access should be through a framework which recognizes those and automatically retries the transaction.
I guess you overstressed the aggregate root concept here. In my understanding of modelling an employee (that depends on your context) an employee is almost always an aggregate root possibly referenced by another aggregate root facility.
Both employee and facility almost always have natural keys. For the employee this is typically some employee id (printed on employee identification badges, or at least maintained in the human resources software system) and facilities have this natural keys too almost always containing some location part and some number like "MUC-1" for facility 1 located in munich. But that all depends on your context. In case employee and facility have this natural keys your database model should be quite clear.
Please help an EF n00b design his database.
I have several companies that produce several products, so there's a many-to-many relationship between companies and products. I have an intermediate table, Company_Product, that relates them.
Each company/product combination has a unique SKU. For example Acme widgets have SKU 123, but Omega widgets have SKU 456. I added the SKU as a field in the Company_Product intermediate table.
EF generated a model with a 1:* relationship between the company and Company_Product tables, and a 1:* relationship between the product and Company_Product tables. I really want a : relationship between company and product. But, most importantly, there's no way to access the SKU directly from the model.
Do I need to put the SKU in its own table and write a join, or is there a better way?
I just tested this in a new VS2010 project (EFv4) to be sure, and here's what I found:
When your associative table in the middle (Company_Product) has ONLY the 2 foreign keys to the other tables (CompanyID and ProductID), then adding all 3 tables to the designer ends up modeling the many to many relationship. It doesn't even generate a class for the Company_Product table. Each Company has a Products collection, and each Product has a Companies collection.
However, if your associative table (Company_Product) has other fields (such as SKU, it's own Primary Key, or other descriptive fields like dates, descriptions, etc), then the EF modeler will create a separate class, and it does what you've already seen.
Having the class in the middle with 1:* relationships out to Company and Product is not a bad thing, and you can still get the data you want with some easy queries.
// Get all products for Company with ID = 1
var q =
from compProd in context.Company_Product
where compProd.CompanyID == 1
select compProd.Product;
True, it's not as easy to just navigate the relationships of the model, when you already have your entity objects loaded, for instance, but that's what a data layer is for. Encapsulate the queries that get the data you want. If you really want to get rid of that middle Company_Product class, and have the many-to-many directly represented in the class model, then you'll have to strip down the Company_Product table to contain only the 2 foreign keys, and get rid of the SKU.
Actually, I shouldn't say you HAVE to do that...you might be able to do some edits in the designer and set it up this way anyway. I'll give it a try and report back.
UPDATE
Keeping the SKU in the Company_Product table (meaning my EF model had 3 classes, not 2; it created the Company_Payload class, with a 1:* to the other 2 tables), I tried to add an association directly between Company and Product. The steps I followed were:
Right click on the Company class in the designer
Add > Association
Set "End" on the left to be Company (it should be already)
Set "End" on the right to Product
Change both multiplicities to "* (Many)"
The navigation properties should be named "Products" and "Companies"
Hit OK.
Right Click on the association in the model > click "Table Mapping"
Under "Add a table or view" select "Company_Product"
Map Company -> ID (on left) to CompanyID (on right)
Map Product -> ID (on left) to ProductID (on right)
But, it doesn't work. It gives this error:
Error 3025: Problem in mapping fragments starting at line 175:Must specify mapping for all key properties (Company_Product.SKU) of table Company_Product.
So that particular association is invalid, because it uses Company_Product as the table, but doesn't map the SKU field to anything.
Also, while I was researching this, I came across this "Best Practice" tidbit from the book Entity Framework 4.0 Recipies (note that for an association table with extra fields, besides to 2 FKs, they refer to the extra fields as the "payload". In your case, SKU is the payload in Company_Product).
Best Practice
Unfortunately, a project
that starts out with several,
payload-free, many-to-many
relationships often ends up with
several, payload-rich, many-to-many
relationships. Refactoring a model,
especially late in the development
cycle, to accommodate payloads in the
many-to-many relationships can be
tedious. Not only are additional
entities introduced, but the queries
and navigation patterns through the
relationships change as well. Some
developers argue that every
many-to-many relationship should start
off with some payload, typically a
synthetic key, so the inevitable
addition of more payload has
significantly less impact on the
project.
So here's the best practice.
If you have a payload-free,
many-to-many relationship and you
think there is some chance that it may
change over time to include a payload,
start with an extra identity column in
the link table. When you import the
tables into your model, you will get
two one-to-many relationships, which
means the code you write and the model
you have will be ready for any number
of additional payload columns that
come along as the project matures. The
cost of an additional integer identity
column is usually a pretty small price
to pay to keep the model more
flexible.
(From Chapter 2. Entity Data Modeling Fundamentals, 2.4. Modeling a Many-to-Many Relationship with a Payload)
Sounds like good advice. Especially since you already have a payload (SKU).
I would just like to add the following to Samuel's answer:
If you want to directly query from one side of a many-to-many relationship (with payload) to the other, you can use the following code (using the same example):
Company c = context.Companies.First();
IQueryable<Product> products = c.Company_Products.Select(cp => cp.Product);
The products variable would then be all Product records associated with the Company c record. If you would like to include the SKU for each of the products, you could use an anonymous class like so:
var productsWithSKU = c.Company_Products.Select(cp => new {
ProductID = cp.Product.ID,
Name = cp.Product.Name,
Price = cp.Product.Price,
SKU = cp.SKU
});
foreach (var
You can encapsulate the first query in a read-only property for simplicity like so:
public partial class Company
{
public property IQueryable<Product> Products
{
get { return Company_Products.Select(cp => cp.Product); }
}
}
You can't do that with the query that includes the SKU because you can't return anonymous types. You would have to have a definite class, which would typically be done by either adding a non-mapped property to the Product class or creating another class that inherits from Product that would add an SKU property. If you use an inherited class though, you will not be able to make changes to it and have it managed by EF - it would only be useful for display purposes.
Cheers. :)
I'm willing to give MongoDB and CouchDB a serious try. So far I've worked a bit with Mongo, but I'm also intrigued by Couch's RESTful approach.
Having worked for years with relational DBs, I still don't get what is the best way to get some things done with non relational databases.
For example, if I have 1000 car shops and 1000 car types, I want to specify what kind of cars each shop sells. Each car has 100 features. Within a relational database i'd make a middle table to link each car shop with the car types it sells via IDs. What is the approach of No-sql? If every car shop sells 50 car types, it means replicating a huge amount of data, if I have to store within the car shop all the features of all the car types it sells!
Any help appreciated.
I can only speak to CouchDB.
The best way to stick your data in the db is to not normalize it at all beyond converting it to JSON. If that data is "cars" then stick all the data about every car in the database.
You then use map/reduce to create a normalized index of the data. So, if you want an index of every car, sorted first by shop, then by car-type you would emit each car with an index of [shop, car-type].
Map reduce seems a little scary at first, but you don't need to understand all the complicated stuff or even btrees, all you need to understand is how the key sorting works.
http://wiki.apache.org/couchdb/View_collation
With that alone you can create amazing normalized indexes over differing documents with the map reduce system in CouchDB.
In MongoDB an often used approach would be store a list of _ids of car types in each car shop. So no separate join table but still basically doing a client-side join.
Embedded documents become more relevant for cases that aren't many-to-many like this.
Coming from a HBase/BigTable point of view, typically you would completely denormalize your data, and use a "list" field, or multidimensional map column (see this link for a better description).
The word "column" is another loaded
word like "table" and "base" which
carries the emotional baggage of years
of RDBMS experience.
Instead, I find it easier to think
about this like a multidimensional map
- a map of maps if you will.
For your example for a many-to-many relationship, you can still create two tables, and use your multidimenstional map column to hold the relationship between the tables.
See the FAQ question 20 in the Hadoop/HBase FAQ:
Q:[Michael Dagaev] How would you
design an Hbase table for many-to-many
association between two entities, for
example Student and Course?
I would
define two tables: Student: student
id student data (name, address, ...)
courses (use course ids as column
qualifiers here) Course: course id
course data (name, syllabus, ...)
students (use student ids as column
qualifiers here) Does it make sense?
A[Jonathan Gray] : Your design does
make sense. As you said, you'd
probably have two column-families in
each of the Student and Course tables.
One for the data, another with a
column per student or course. For
example, a student row might look
like: Student : id/row/key = 1001
data:name = Student Name data:address
= 123 ABC St courses:2001 = (If you need more information about this
association, for example, if they are
on the waiting list) courses:2002 =
... This schema gives you fast access
to the queries, show all classes for a
student (student table, courses
family), or all students for a class
(courses table, students family).
In relational database, the concept is very clear: one table for cars with columns like "car_id, car_type, car_name, car_price", and another table for shops with columns "shop_id, car_id, shop_name, sale_count", the "car_id" links the two table together for data Ops. All the columns must well defined in creating the database.
No SQL database systems do not require you pre-define these columns and tables. You just construct your records in a certain format, say JSon, like:
"{car:[id:1, type:auto, name:ford], shop:[id:100, name:some_shop]}",
"{car:[id:2, type:auto, name:benz], shop:[id:105, name:my_shop]}",
.....
After your system is on-line providing service for your management, you may find there are some flaws in your design of db structure, you hope to add one column "employee" of "shop" for your future records. Then your new records coming is as:
"{car:[id:3, type:auto, name:RR], shop:[id:108, name:other_shop, employee:Bill]}",
No SQL systems allow you to do so, but relational database is impossible for this job.