We want to support all of the following scenarios using a EF6 database initializer to create or update an existing database from our EF6 model:
Create and initialize a new database.
Update an existing (legacy) database that has no __MigrationHistory table yet.
Update an existing (migrated) database that already has a __MigrationHistory table.
When using the CreateDatabaseIfNotExists database initializer, it covers only the first scenario. But then we cannot further evolve our data model.
When using the MigrateDatabaseToLatestVersion database initializer with an InitialDatabase migration that creates the base line database, we can support scenario 1 and 3. But it fails to upgrade existing legacy databases because the generated SQL statements are not idempotent. They fail e. g. with a "Table 'xy' already exists" error.
When using the MigrateDatabaseToLatestVersion database initializer with an empty InitialDatabase migration (=empty Up() method body), then we can only support scenario 2 and 3, but we cannot create new database from scratch.
I am looking for a way that combines the best of both worlds and supports all three required scenarios. Unfortunately this seems impossible with the current EF6 design. The problem I am facing is that the DbMigration steps are discovered using reflection and there seems to be no (clean) way to intercept this. What I would like to do is to write an enhanced "hybrid" database initializer that:
1. Checks if the database exists.
2. If not, then execute all migrations (=default behavior of MigrateDatabaseToLatestVersion).
3. Otherwise check if it is already enabled for migrations (i. e. table __MigrationHistory exists).
4. If yes, then execute only the pending migrations (=default behavior of MigrateDatabaseToLatestVersion).
5. Otherwise execute all DbMigrations except the very first "InitialDatabase" migration. This should also create the __MigrationHistory table and include all(!) migrations (including the very first "InitialDatabase" migration).
I don't find a way to implement step 5.
I would be happy with a "hack" that lets me just catch and ignore exceptions from upgrade operations of the first step (with a well-known MigrationId). I already thought about deriving a class from DbMigrator and override the ApplyMigration method with a
try
{
base.ApplyMigration(...)
}
catch
{
...ignore if migrationMetadata.Id=="my well-known id"..
}
block. But this is impossible because the ApplyMigration is internal.
Any ideas on this?
Related
Inside of OnModelCreating, I want to be able to ignore a column if the database is on an older migration EF Core 5 throws an exception if I attempt to read from the database directly, or indirectly by querying the applied migrations. I'm not certian that it's even a good idea, since OnModelCreating is used during the migration 😩, but I'll burn that bridge when I cross it.
There are some examples on how one would do this with EF6, but they don't seem to apply anymore with EF Core.
While Ivan Stoev is right that --generally-- you should model the target database without outside input, the real world isn't always that clear-cut. In my particular case, there are multiple service instances (Azure Functions) that need to read and write to a single database. In order to maintain zero downtime, those Functions need to not read or write columns that don't yet exist.
I solved the problem the way Serge suggested. The database has a known version, populated with seed data that increments with every migration. On startup, the service reads that version with a regular old Microsoft.Data.Sql.SqlConnection. This version is then added to the IServiceCollection as a singleton to be used by the DbContext constructor.
When talking to an older database version, OnModelCreating does things like this:
builder.Entity<Widget>(w =>
{
// another option would be to use the migrations table instead of an integer
if (DatabaseVersion < ContextVersions.WidgetNewPropertyAddedVersion)
{
w.Ignore(w => w.NewProperty);
}
else
{
w.Property(w => w.NewProperty)
.HasDefaultValue(0);
}
});
The startup code also detects if it's been started by the Entity Framework tools and does not read the database version, instead assuming "latest". This way, we do not ignore new properties when building the migration.
Figuring out how to let the service instances know that the database has been upgraded and they should restart to get the new database model is an exercise left up to the reader. :)
I'm using EF6 code-first migrations for existing database but initial DbContext does not fully cover existing schema (since it's massive). So from time to time I have to make updates to the model in database-first style. For example when I need an entity mapping for a table or a column that is already in the database but not reflected in the code I do the following:
Make all change (add new entity, rename the column mapping or add new property)
Scaffold migration representing the latest model snapshot stub_migration
Copy-paste latest serialized model from stub_migration to the last_migration resource file
Delete stub_migration
Revert last_migration in database
Update-Database so that model snapshot in [__MigrationHistory] table would be also updated
I understand that this aproach is a bit hackish and the proper way would be to leave empty stub_migration but this would force lots of empty migrations which I would rather avoid.
Looking at a similar scenario from MSDN article (Option 2: Update the model snapshot in the last migration) I wouldn't imagine that there is an easier way rather than writing power shell script, managed code or both to make it work. But I would rather ask community first before diving deep into it.
So I wonder: is there a simple way to automate generation of new model snapshot in latest migration and reaplying it?
I'm doing something similar. I have a large database and I am using the EF Tools for VS 2013 to reverse engineer it in small parts into my DEV environment. The tool creates my POCOs and Context changes in a separate folder. I move them to my data project, create a fluent configuration and then apply a migration (or turn automigration on).
After a while I want a single migration for TEST or PROD so I roll them up into a single migration using the technique explained here: http://cpratt.co/migrating-production-database-with-entity-framework-code-first/#at_pco=smlwn-1.0&at_si=54ad5c7b61c48943&at_ab=per-12&at_pos=0&at_tot=1
You can simplify the steps for updating DbContext snapshot of the last migration applied to database by re-scaffolding it with Entity Framework:
Revert the last migration if it is applied to the database:
Update-Database -Target:Previous_Migraton
Re-scaffold the last migration Add-Migration The_name_of_the_last_migration which will recreate the last migrations *.resx and *.Designer.cs (not the migration code), which is quite handy.
Those 2 steps are covering 4 steps (2-5) from original question.
You can also get different bahavior depending on what you want by specifying the flags -IgnoreChanges and (or) -Force
And by the way, the major problem with the updating the DbContext snapshot is not how to automate those steps, but how to conditionally apply them to TEST/PROD environments depending on whether you actually want to suppress the warning because you've mapped existing DB-first entities in you DbContext or you want it it to fail the build in case you've created new entities and forgot to create a code-first migration for them.
So, try to avoid those steps altogether and maybe create empty migrations when you just want to map existing tables to your code.
I am a newbie in Codefirst and I do not understand how it works correctly.
I created 3 migrations, migration 1..3 via "Add-Migration" command and issued the relative update with "Update Database".
I have a Configuration.cs file in my Migration directory and custom database initializer (I am working on MySQL) I created in order to seed initial data.
I do not know what happens behind the scenes in the production environment where I do not have any database yet.
Who is responsible to create and update the database? Are the migrations executed one after the other?
Can you suggest me how this process works in production and share useful links?
Regards,
Roberto
Assuming you are using Automatic Migrations - All migrations are called one after another, in the sequence in which they were created. Their Up function is called. Afterwards the Seed function on your Configuration class should fire.
Each migration has a sort of a Time-Stamp inside its file name, like this:
if you are creating lots of databases out of a single context class like I do, and you are afraid of not having control over migrations, you can use manual migrations. It is called DbMigrator.
See my/others answer here how to use it.
Where is stored information about current database state (what migration is applied)? I suppose it can be "dbo.__MigrationHistory" table or this table is just for logging purpose?
If I enabled migration, added migration and updated my database. After that I checked-in code to SVN and another developer checked it out. What this another developer has to do to create/update his own database?
I see such options:
1) Call Update-Database command right away.
2) Do everything from beginning (Enable-Migration, Add-Migration, Update-Database).
3) Do everything but skip Add-Migration step (it is already present and it seems strange to add it once again for every new developer).
Which of my assumption is right or if no one where is the right way?
To retrieve which migrations have been applied to the database, you can use the Get-Migrations command (reference).
Everything depends on how the database is created, which initializer you are using.
This article is worth reading if you are unfamiliar with those.
When using the DropCreateDatabaseAlways initializer, you don't really need to care about updating your database, because your database will be deleted & recreated at each application startup.
When using the DropCreateDatabaseWhenModelChanges initiliazer, your database will be dropped then recreated if EF detects the model has changed at the application startup.
When using the CreateDatabaseIfNotExists initilizer or if no initializer has been defined, your database will be created if it does not already exists. If the database already exists & you added a Migration, you (and every developer retrieving your code) need to use the Update-Database command to update the database.
There is a new initializer introduced with Code-First Migrations: MigrateDatabaseToLatestVersion, this initializer automatically update the database to the latest Migration defined.
See this page's last section: http://msdn.microsoft.com/en-us/data/jj591621.
I need to configure the database created by Entity Framework Code First MigrateDatabaseToLatestVersion class.
Is it possible to influence the database files parameters like size or maxsize? What interests me in particular, is there a way to add database files?
I assume I need to find the moment where the Entity Framework generates the CREATE DATABASE statement and influence it a bit. As far as I understand, the SqlServerMigrationSqlGenerator class is too late because the database already exists at this point.
Edit: According to comment this doesn't work:
You can just add code based migration using Add-Migration command and modify its Up method to execute Sql("ALTER DATABASE ...") and do what ever you want with your database.
Database is created through DbProviderServices - that is a bridging component between EF and specific database server. ObjectContext exposes operations for creating database and generating database creation script. DbMigrator use database creation operation when you execute Update. It checks that if Database exists and if not it uses ObjectContext.CreateDatabase. So if you want to change the process of creating the database you need to implement your own implementation of the migrator to change the way how Update method generates the database (maybe you just need a decorator which will create a database prior to executing DbMigrator.Update operation).