After developing our application for a while we've accumulated quite a bit of EFCore database migrations. Since EFCore adds a snapshot of the entire db model to every migration, this code adds up quite a bit. After analysis about 80% of our compile time is spend on the migrations (compiling + Roslyn analyzers).
So it's time to clean up some old migrations! But what's the best way to do this? There doesn't seem to be any official guidance on it...
We don't need any rollbacks (we only roll forward), so that makes things more simple. We do need to support creating a database from scratch, and updating a database from the last few migrations.
What I've tried:
The nuclear option seems to be to delete all migrations and the model snapshot, and creating a new initial migration. While this is fine, it seems a bit dangerous. With this approach we need to be very careful that every part of the database schema is part of the code model. One edge case we for example ran into is that EFCore doesn't support checked constraints yet. So we added a checked constraint in a migration, but not in the code model. So when creating a new initial migration, the checked constraint was not part of it.
As an experiment, I've tried to delete the model snapshot from all old migrations, since the snapshots are 90% of the code which cause the long compile time. I figured out, that EFCore only uses the snapshot as a compare tool to make a new migration. After deleting the snapshot, the old migrations were however no longer executed when they ran on a fresh database.
So is there any better way to accomplish what I want?
Okay, since asking this question I've experimented quite a bit with this.
It seems for now, the best way to accomplish this is option 1. Option 2 would be much better, but until this EFCore feature is implemented, it's not really doable for my use case (supporting existing dbs with migrations on them, and supporting empty dbs).
Option 1 also has a few pitfalls which I stumbled upon (maybe even more that I haven't stumbled upon).
So this is how I did it:
Create a new initial migration:
Make sure all your existing migrations have been applied to your database. We'll create a new initial migration, so the migrations that haven't been applied will be lost.
Delete your old EFCore migration files, and the database snapshot file.
Create a new Initial migration from your database's current state. (For example via dotnet ef migrations add Initial-PostCleanup.)
This new migration is only compatible with new databases, since it will create all tables (and fail if any of the tables, constraints, etc. already exist). So now we're going to make this migration compatible with the existing database:
Create a SQL script for the new initial migration via dotnet ef migrations script -o script.sql.
Remove the first transaction (until the first GO), which creates the __EFMigrationsHistory table:
IF OBJECT_ID(N'[__EFMigrationsHistory]') IS NULL
BEGIN
CREATE TABLE [__EFMigrationsHistory] (
[MigrationId] nvarchar(150) NOT NULL,
[ProductVersion] nvarchar(32) NOT NULL,
CONSTRAINT [PK___EFMigrationsHistory] PRIMARY KEY ([MigrationId])
);
END;
GO
Remove the last transaction, that inserts the new entry in the __EFMigrationsHistory table:
INSERT INTO [__EFMigrationsHistory] ([MigrationId], [ProductVersion])
VALUES (N'20190704144924_Initial-PostCleanup', N'2.2.4-servicing-10062');
GO
Remove GO commands, since we will put the create script in an IF statement:
Replace GO\r\n\r\n with nothing.
Now open up your migration file (the C# file, not the sql file) and replace the Up method with the following:
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(#"
DECLARE #migrationsCount INT = (SELECT COUNT(*) FROM [dbo].[__EFMigrationsHistory])
IF #migrationsCount = 0
BEGIN
% PASTE YOUR EDITED SQL SCRIPT HERE %
END
");
}
Done! Everything should work now!
Be sure to compare the database schema, and data before and after for the new database. Everything that's not part if your EF Code model is not part of the new database.
A bit late, but we had the same problem in our current project. Above 400 migraitons and 6m lines of code inside .Designer. Here is how we managed to resolve this problem:
MigrationProject.csproj
<PropertyGroup>
...
<DefaultItemExcludes Condition="'$(Configuration)' == 'Debug' ">$(DefaultItemExcludes);Migrations\**\*.Designer.cs</DefaultItemExcludes>
</PropertyGroup>
This way you don't need to reset migrations to a clear state neither delete .Designer files. You can always change configuration to Release and use .Designer files by any means necessary.
To reset all migrations and updates from scratch (Assume no useful data on disk), the following steps may be useful.
(1) Ensure that program.cs file is not optimized for creating/ updating Database by Database.EnsureCreate command, as this command prevents Migrations.
(2) Delete folder Migrations.
(3) dotnet build
(4) dotnet ef database update 0 -c yourContextFile
(5) dotnet ef migrations add init -c yourContextFile
(6) dotnet ef database update -c yourContextFile
I'm fairly new to Entity Framework and I am having trouble getting it to apply my code first migrations to a new database. I think my migrations have gotten into a bad state.
I started out with an existing database, so I created an initial empty migration using
Add-Migration Initial -IgnoreChanges
That worked fine. Then I added a new entity and created another migration
Add-Migration New_Entity
That worked fine. I could apply these migrations to an existing database that didn't even have a __MigrationHistory table and it would be properly updated.
Fast forward a few days and a few commits later. I now have a new "clean" database that I want to apply the migrations to. I got into Visual Studio, point my app.config file at the new database, run
Update-Database
And it tells me
Unable to update database to match the current model because there are
pending changes and automatic migration is disabled. Either write the
pending model changes to a code-based migration or enable automatic
migration. Set DbMigrationsConfiguration.AutomaticMigrationsEnabled to
true to enable automatic migration. You can use the Add-Migration
command to write the pending model changes to a code-based migration.
However, I do not have any pending models changes. Nothing has changed. But just to see what would happen, I ran
Add-Migration x
and it created a new migration that would add every entity in the model to the database. Basically starting over as if I had no migrations. So I deleted that one then re-added with the -IgnoreChanges flag. That gives me another blank one. If then run
Update-Database
It runs the x migration but totally ignores the migrations I created a few days ago (Initial and New_Entity). It's like it doesn't know they exist.
Obviously something is in a bad state but I'm not sure what I can do to get it back. I can't just delete and recreate the migrations because I need a blank one and one with just the new entity. I can't just comment out the entity temporarily because a lot of code already depends on it.
How did I get myself into this mess and how do I get out?
Alright, I got to the bottom of this. Turns out that somehow (I can't recall how at this point) the namespace inside the Configuration.cs file (inside the Migrations folder) was changed. Apparently this is enough to confuse the system into not seeing your other migrations. I thought at first it was simply because it was out of synch with the actual migration classes that had been generated. However, I tried changing the migration classes' namespaces to match the new namespace value of Congiguration.cs but the migrations still failed. It only worked after I reverted Configuration.cs back to its old namespace value.
Not really sure exactly what the deal is here. Maybe the original namespace was stored in the migration resx files and it noticed they didn't match anymore?
I was using EF5 code first and migrations right from the start.But I messed up something
and decided to reset my migrations.
I did enable-migrations -force
Then tried add-migrations xyz
I expected only to see incremental changes (addition of 2 tables)
Instead it tries to recreate every table.I dont want this to happen as it is shared via GIT and I need to push the migration also.
I have deleted all migration history and folder.What I want is a way to do another migration and it should only do the incremental create tables as other tables are already there
You need to do this in two steps:
Comment out your added tables and create an empty migration that updates the meta data to the state before those where added, by using the -ignore-changes switch (see my command reference)
Readd your tables and create a migration. Now it should contain only those two new tables.
Please note that whenever you mess manually with the migrations you need to be careful with the state of the metadata or you can get really nasty surprises.
I am using Entity Framework 5.0 Data migrations along with code first.
When i add a new field to my model and execute the following command in the package manager console.
"Add-migration AddedField"
All I get is an empty migration called "n_AddedField", the up and down methods contain no logic.
I tried a bunch of things, reinstalling the EF nuget package, cleaning my solution, rebuilding, manually removing all generated files and directories.
Then i decided that i would scrap all my migrations and start over, and then it got weird.
After deleting all my migrations, and the migrationhistory table in the database, i recreated the database using the CreateDatabaseIfNotExists initializer. After doing this, I should be able to create a new initial migration. But when i try to create create a new migration, I get an error saying that there are pending migrations, and lists all the migrations that i just deleted from my project.
I have no idea why and how EF still has any recollection of those migrations.
I even tried searching through filecontents looking if the migrations were saved somewhere else or something. But nothing..
Data migrations look really neat when scott hansleman demo's it on stage, but for real work, I'm starting to look for alternatives.
When the project started, we were using EF 4.x and a while back switcted to 5.0, but since the switch i have added a bunch of migrations successfully.
Does anyone have any idea how to solve this problem?
Basically i just want to be able to add migrations, and generate a sql script with the changes.
I had a similar problem where a new migration was not being found, and so update-database was giving me the following error no matter what I did:
Unable to update database to match the current model because there are pending changes and automatic migration is disabled. Either write the pending model changes to a code-based migration or enable automatic migration. Set DbMigrationsConfiguration.AutomaticMigrationsEnabled to true to enable automatic migration.
You can use the Add-Migration command to write the pending model changes to a code-based migration.
Doing a "batch clean" solved my problem, suggesting EF was using an old/invalid assembly from a folder other than the currently selected 'solution configuration (e.g. DEBUG)'.
Hope this helps someone else out there.
oops. In my case I was adding a new root entity not referenced by any other entity. The result was simply that code first had no reason to generate a migration for the entity. Once I added the code into the DbContext (a dbset) it worked like a charm.
The problem in my case was caused by:
Create a migration (successfully)
Decide that I want to re-create it, and delete the migration .cs file
Try to regenerate it, and end up with empty migration's Down and Up functions
In this case, I forgot to also delete the ApplicationDbContextModelSnapshot.cs entries for the model changes. Removing the new mappings in this file solved my problem and it then generated correctly.
Just got the same problem but figured out that my new field was added as a member variable and not a property - it was missing the {get; set;} part and that makes migration skip that field.
May not be your case but it might help someone else.
You're 'out of sync' - Db, migrations, code - and you can expect all sorts of problems like that.
I did this million times (almost:) and it works really well - but you need to go steady, and be meticulous with what you're doing.
You can read through this 'summary' I made - start half-way somewhere (but also check connection).
Code first create tables
...and if it doesn't work I'd suggest you make a small 'repeatable' scenario / model - post exactly what you have.
How migrations work:
Migrations are tied to the 'migration table'.
When Add-Migration is run - it checks against the 'existing database' structure and migration table - and makes the 'difference' (sometimes you get none 'up' 'down' simply as too are in sync).
So, each 'migration' is a complex diff in between your code, existing migrations, and database, and migration table. Short of removing the database nothing else is certain to reset - Db 'migration' table may not be enough - that doesn't guarantee full 'cleanup' (if possible, I always do full Db delete). You also need to delete your code migrations.
Make sure to 'compile' the projects (best make them compile automatically in configuration) after/before where relevant.
Make sure your 'connection' matches.
Once all is in sync - all should work nicely - but you gotta keep it in sync. Unless you plan to delete the Db (testing) - don't remove migrations just like that (you can use Update-Database -0 (I think) to get back to some migration (this is 'zero state').
I had a problem similar to this, where using the -force flag on add-migration to re-scaffold an existing migration stopped working for no apparent reason.
No matter what I did I got the stupid "Unable to generate an explicit migration because the following explicit migrations are pending" error message. After trying nearly everything I could think of and stopping just short of smashing my laptop to pieces, out of desperation I ran enable-migrations again and of course got the "Migrations have already been enabled in project 'Blah.Blah'" message. Tried add-migration -force again and magically it was working.
I have no idea what it changed- must have been some user settings/config file outside of source control. Hopefully this will help someone else.
The batch build -> clean option did not work for me.
I solved the problem by:
Creating a migration with 'Add-Migration NameOfMigration'
Deleting the contents of the up and down functions of the created migration class.
Updating the database by running the migration script (which will just add a row to the _MigrationHistory table with 'Update-Database -Verbose'
The web application now runs successfully, so essentially I had an issue that was fixed by adding meta-data only.
It happened to me and nothing worked. Then i did this on my own and everything works now.
Problem:
I created a Model "Cars". And When I create a migration for it using command "add-migartion AddCarModel", a migratoin was created but it was empty. I tried with different names and also tried delete migration's .cs file but nothing worked. Then I did the following:
Solution:
Follow below steps:
1. Delete all the empty migrations that you created for the Model. (But remember the names of the migrations for step 2)
2. Also delete those migration entries from "_MigrationHistory" table.
3. Comment out you line(s) of your model DB context, (in my case it is "public DbSet Cars{ get; set; }")
4. Clean and Rebuild solution. (Its best that if you batch clean)
5. Make sure that your update command is working and not throwing errors. (Command: "update-database -verbose")
6. Now uncomment line(s) that you commented in step 3.
7. Now create the migration for that model. (I created the migration with same name as before)
Hopefully it works. :-)
I added a new class to my data model to a sub-directory, the resultant namespace was not visible to scaffolding using add-migration.
Fix was to rename the namespace of the new class to conform to the rest of model, and/or add "public virtual DbSet .." etc into your entity context class, which will require you to reference this new namespace, then run add-migration again.
It seems that i managed to solve the problem by moving the models and the context class to another project.
I still have no idea why this happened, and this solution is really no solution at all :(
I had the same problem. Migrations were enabled but they weren't detecting any changes.
My solution was to re-enable migrations using -Force attribute and then everything worked.
Enable-Migrations -ProjectName -StartupProjectName --ConnectionStringName -Force
I had to delete the _MigrationHistory table that is generated by EF. Then I ran add-migration again. Be careful with this though, as it will generate the queries needed from scratch, including tables that are already there.
In my case it was because I had added a secondary context 'ApplicationDbContext' as part of the ASP.net identity stuff. When I ran the 'enable-migrations' command again I got an error that there was more than one context. Once I combined the two things started working again.
Maybe the stupidest of all:
I was adding a migration with the same name as the new object I was creating.
I had to run dotnet ef migrations remove even though I'd deleted the previous migration
If you are using fluent api to set up your configurations for the DbSets then you won't have any problems with it
I'm using EF 5.0 and I would like to start using Code-based migrations
I've used fluent migrator and there is a concept of migration order. Migrations can be migrated/rollback no matter the database's migration version.
Does Entity Framework have similar functionality?
I was planning on keeping multiple migration implementations for each database version (likely tied to sprint number at first).
Why do i want this?
Our continuous integration will migrate the database for each environment. It's likely that our Dev build will only be one version "behind" but when we go to QA or PROD environment the database will be behind by multiple migrations.
Maybe i'm going about this the wrong way, in which case I would love to hear opinions on the best way to do migration with CI.
Yes EF has this functionality.
When you run Add-Migration you'll notice the migration file is prefixed with a timestamp. This is what determines the order, assuming automatic migrations are and always have been disabled.
If you are using a mixture of explicit migrations and automatic migrations then you may notice an additional Source property in the .resx file generated with your migration. This is how EF will determine if it needs to run an automatic migration before it runs your explicit migration.
My experience has taught me these guidelines:
1) Never use automatic migrations.
2) Every developer on your team should ensure they have the latest code before creating a new explicit migration. Sort of obvious, but creating migrations from stale code will result in problems.
3) Developers should make sure that if they write custom SQL in the Up() method of the migration then they write appropriate code (and test it!) to reverse those changes in the Down() method.