I'm new to data migrations, so I'm just wondering what the best way would be to go about migrating all of the data from the Big Table (NDB) over to Django Models (Postgres).
On the one hand, I have plenty of 'tables' that have plenty of relations (KeyProperties) and on the other, I must maintain those relations as well as port some over to general relations (GFK).
I'm not even sure how to go about doing this. I know how to create a Postgres Django DB, just not how to maintain things like, KeyProperties linking to image Blogs. How do I copy those images over and also maintain this 'FK' relation? I have quite a bit of data and would really like to maintain the structure of it.
Is there any good documents on database migrations and how its ideally done?
Any help would be appreciated!!!
Create a Postgres table just for the images (using BLOB or bytea types) and use FK relations to it.
The general question of doing database migrations is too broad to answer, please ask a more specific question. You are going to have to write custom code to split apart each entity's properties and convert them into Postgres data types.
Related
I am new to the rails world and I need to use an existing database for a rails project, for example I have a "games" table that does not have the columns created_at and updated_at, if I add them in the postgres terminal, rails would recognize my tables as objects ("Game" for example)? or should I generate the model for each table in my database? what would be the correct way to fetch all tables like a objects and use them from the controllers?
As mentioned in the comment, one version is to use some kind of reverse engineering gem that would automatically generate models for your database tables with all the associations, foreign keys etc.
BUT if you are new to Rails, I would definitely suggest doing that by hand for a couple of reasons:
Those gems will not understand namespacing (a solid guess of mine) e.g. If you have a admin_users, admin_other_table, admin_suggestions tables, the most reasonable class name in ruby would be Admin::User, Admin::OtherTable, Admin::Suggestion, but there is no way that these gems would understand that.
If it makes more sense to name your model different than the table (and then map the correct table with self.table_name=, gems won't help.
If associations need to be defined only one way then your models will get a lot of code in them that is just useless( if you have "user has many articles with a foreign key of author_id" maybe you don't need the user.articles method, but your going to get it anyways.
Probably there are a lot more of these cases. So I would strongly suggest doing it by hand so you have full control of what kind of code is written.
I am curious to know that if Entity framework can create tables in other databases besides MS-SQL ??
Moreover, is there any provision to create XML schema through EF ?
Under the hood Entity Framework uses providers that are specific for different databases. So it depends on a provider whether EF can create tables or not. However, I haven't heard about providers that do not have this possibility. The easiest way to be sure is to write a simple program with a few lines of code.
As to XML schema. Are you asking about using XML files instead of database as the storage for your data? If so, again it depends on the provider. If you want you can theoretically create one that will use XML files. However, I haven't tried to do so and I don't think that it is a good idea. There are technologies that fit here better (see this question).
We are involved in quite a new development in which we are remaking our current web shop platform.
In the current platform we do not use EF6 neither other ORM but store procedures to access to the db, but in the new building is what we do.
We have a doubt regarding database design of the new platform. In the current platform we use several different databases depending on the content of them.
For example, we have dedicated databases to store information for products catalogs other dedicated db for handling orders.
Currently all data access is done through stored procedures, so we have no problem with the links between different databases.
The problem appears to us now when we have started to use EF6. In this case each DB is associated with a context and it is not possible to know data from one context to another
unless we implement directly in the source code these relationships using various contexts. It looks like these means we will lose the power of EF6.
The questions we have are:
Is it a bad design maintaining different databases for the same application using EF6?
in case this is a poor design and choosing for a single database, is the performance going to be optimum even driving hundreds of tables (almost 1000) with several TBytes of information?
in the other hand, in the case of opting for the design in which several bbdd appear (it would be much better in our case), what is the best way to handle them EF6?
Thank you very much for your help!
First of all EF is not written to be cross database. You can't write cross database (cross context) queries, lazy load does not work and so on.
This is a big limitation in your case.
EF could work with several schema (actually I don't use it and I don't like it but is just my opinion).
You can use your stored procedures with EF but as I understand you are thinking to stop to use them.
In my experience I wrote several applications with more than one database but the use of the different databases was very limited. In this cases I use cross database views (i.e. one database per company and some common tables with views in company databases that selects data in common tables). In your case, if the tables are sharded everywhere I don't think this is a way you can choose.
So, in my opinion you could change the approach.
If you have backups problems you could shard the huge tables (I think facts tables and tables with pictures) and create cross database views. BTW, also, cross database referential integrity is not supported in SQL Server so you need to write triggers to check it.
If you need to split different application functions (i.e. WMS, CRM and so on) you can use namespaces without bothering about how tables are stored in the DB.
I am a DBA. I want to know what advantages my Business Objects developers will get when using EF with SQL Server DB which is fully managed using Foreign keys and Primarkey as and when require. As this is our new project and we have to use EF with SQL Server 2008 R2. We have a plan to use Database First Approach. Can anyone tell me what difference my Business Object developer will experience in case If I define all foreign Key relationships in my DB?
Assuming it's setup correctly, when your developers actually create their objects from the database structure, they'll be able to access any related tables rather easily.
It should also make creation of new objects (rows in the tables) easy, as it then shouldn't be possible to create new items that would break the foreign key relationship.
It's also just plain good practice to correctly setup any foreign keys in the database; I'm not sure of any benefit not to.
As a developer that's had to work with data sources that haven't been setup correctly, I can tell you a correctly setup database structure is an amazing experience for a developer.
(As an aside, as a DBA, you may want to take a look at EF. Also take a look at LINQ, one of the items that they'll be using. In particular, Why LINQ beats SQL may help you get a basic understanding, even if you don't agree with the article title :) )
How is database normalization related to ER Modelling??
What comes first??
Or should both be implemented at the same time??
I feel modeling should come first in a highly normalized database design.
Creating the model allows you to think through how the tables will relate to one another and also allows you to envision what tables you'll need to use when writing your join queries.
Using a tool such as MySQL Workbench or Toad Data Modeler , depending on your target database vendor, can even generate SQL commands to build the tables, constraints, and indexes directly from the model. This is useful because it ensures the tables are created exactly as you designed them.
Also, when making changes to the model, some tools like those mentioned above will even allow you to "update" your schema by issuing the necessary statements required to do so.
So in short, for a project with more than one table, I'd always model it first. It also makes it easier for developers to understand how the tables function and relate at a glance rather than having to read through DDL to understand it.
Modeling can even be fun!
A model created with MySQL Workbench:
Hope this helps!