EFCore: migrations and initial DB creation - entity-framework-core

I'm using EFCore (recently migrated from EF) to do code-first DBs with coded migrations. EFCore updates the context model snapshot whenever I create a new migration. So when I create a new DB from scratch using
using (var db = new AudmDatabaseContext())
{
if (!db.Database.GetService<Microsoft.EntityFrameworkCore.Storage.IRelationalDatabaseCreator>().Exists())
{
db.Database.EnsureCreated();
}
}
I have the current DB structure. However, when I have an existing system, I need to run migrations
using (var db = new AudmDatabaseContext())
{
if (db.Database.GetService<Microsoft.EntityFrameworkCore.Storage.IRelationalDatabaseCreator>().Exists())
{
var pendingMigrations = db.Database.GetPendingMigrations();
if (pendingMigrations.Count() > 0)
{
db.Database.Migrate();
}
}
}
}
Now, imagine I created a new DB using my DB creation code. Then at some later point, I'm changing the DB schema, so I need to run migrations. But, when I do that, I get all migrations as pending, and the migrator tries to make changes to the DB schema that have already been made - which obviously doesn't work. (note that my initial migration has all code commented out as otherwise it would create the entire DB again but that's already being done with db.Database.EnsureCreated())
So what's the recommended approach here? I need to be able to create new DBs for new installations for my software, and I need to be able to migrate (so I actually set up my app to execute migrations automatically on startup to never miss any changes).
The most obvious that comes to mind would be to have some kind of flag that tells the migrator to run the migrations but actually not execute any sql code when doing the initial create - so a newly created DB would already appear to be fully migrated to the migrator.. and when at some later point I have new pending migrations, only these new pending migrations that have been added to the code since I created the initial DB would be executed.
I'm just curious what other people are doing since I seem to be unable to find the recommended approach to this problem in the official documentation.

Related

Is there a way to query the database before or during OnModelCreating?

Inside of OnModelCreating, I want to be able to ignore a column if the database is on an older migration EF Core 5 throws an exception if I attempt to read from the database directly, or indirectly by querying the applied migrations. I'm not certian that it's even a good idea, since OnModelCreating is used during the migration 😩, but I'll burn that bridge when I cross it.
There are some examples on how one would do this with EF6, but they don't seem to apply anymore with EF Core.
While Ivan Stoev is right that --generally-- you should model the target database without outside input, the real world isn't always that clear-cut. In my particular case, there are multiple service instances (Azure Functions) that need to read and write to a single database. In order to maintain zero downtime, those Functions need to not read or write columns that don't yet exist.
I solved the problem the way Serge suggested. The database has a known version, populated with seed data that increments with every migration. On startup, the service reads that version with a regular old Microsoft.Data.Sql.SqlConnection. This version is then added to the IServiceCollection as a singleton to be used by the DbContext constructor.
When talking to an older database version, OnModelCreating does things like this:
builder.Entity<Widget>(w =>
{
// another option would be to use the migrations table instead of an integer
if (DatabaseVersion < ContextVersions.WidgetNewPropertyAddedVersion)
{
w.Ignore(w => w.NewProperty);
}
else
{
w.Property(w => w.NewProperty)
.HasDefaultValue(0);
}
});
The startup code also detects if it's been started by the Entity Framework tools and does not read the database version, instead assuming "latest". This way, we do not ignore new properties when building the migration.
Figuring out how to let the service instances know that the database has been upgraded and they should restart to get the new database model is an exercise left up to the reader. :)

Entity Framework Core updating database withing project

I have made a blazor project that is hooked up to a database that pulls information via a form. I have also gone ahead and added a migration for this DB. Now our production database has new tables being made every 90 days. My question is for EF Core is there something I can write in the code that would update my solution with the newest Db changes and runs once a day? I have read up on EF Core scripts but could not find any concrete examples. Any advice would be grealy appreciated.
Assuming it is a Blazor Server app, in your Program.cs you can migrate your database every time the application starts (if there are new migrations) by doing something like this:
var host = CreateHostBuilder(args).Build();
using var scope = host.Services.CreateScope();
var services = scope.ServiceProvider;
var context = services.GetRequiredService<YourDbContext>();
await context.Database.Migrate(); // or MigrateAsync();
host.Run();

Recommended way to clean old Entity Framework Core migrations

After developing our application for a while we've accumulated quite a bit of EFCore database migrations. Since EFCore adds a snapshot of the entire db model to every migration, this code adds up quite a bit. After analysis about 80% of our compile time is spend on the migrations (compiling + Roslyn analyzers).
So it's time to clean up some old migrations! But what's the best way to do this? There doesn't seem to be any official guidance on it...
We don't need any rollbacks (we only roll forward), so that makes things more simple. We do need to support creating a database from scratch, and updating a database from the last few migrations.
What I've tried:
The nuclear option seems to be to delete all migrations and the model snapshot, and creating a new initial migration. While this is fine, it seems a bit dangerous. With this approach we need to be very careful that every part of the database schema is part of the code model. One edge case we for example ran into is that EFCore doesn't support checked constraints yet. So we added a checked constraint in a migration, but not in the code model. So when creating a new initial migration, the checked constraint was not part of it.
As an experiment, I've tried to delete the model snapshot from all old migrations, since the snapshots are 90% of the code which cause the long compile time. I figured out, that EFCore only uses the snapshot as a compare tool to make a new migration. After deleting the snapshot, the old migrations were however no longer executed when they ran on a fresh database.
So is there any better way to accomplish what I want?
Okay, since asking this question I've experimented quite a bit with this.
It seems for now, the best way to accomplish this is option 1. Option 2 would be much better, but until this EFCore feature is implemented, it's not really doable for my use case (supporting existing dbs with migrations on them, and supporting empty dbs).
Option 1 also has a few pitfalls which I stumbled upon (maybe even more that I haven't stumbled upon).
So this is how I did it:
Create a new initial migration:
Make sure all your existing migrations have been applied to your database. We'll create a new initial migration, so the migrations that haven't been applied will be lost.
Delete your old EFCore migration files, and the database snapshot file.
Create a new Initial migration from your database's current state. (For example via dotnet ef migrations add Initial-PostCleanup.)
This new migration is only compatible with new databases, since it will create all tables (and fail if any of the tables, constraints, etc. already exist). So now we're going to make this migration compatible with the existing database:
Create a SQL script for the new initial migration via dotnet ef migrations script -o script.sql.
Remove the first transaction (until the first GO), which creates the __EFMigrationsHistory table:
IF OBJECT_ID(N'[__EFMigrationsHistory]') IS NULL
BEGIN
CREATE TABLE [__EFMigrationsHistory] (
[MigrationId] nvarchar(150) NOT NULL,
[ProductVersion] nvarchar(32) NOT NULL,
CONSTRAINT [PK___EFMigrationsHistory] PRIMARY KEY ([MigrationId])
);
END;
GO
Remove the last transaction, that inserts the new entry in the __EFMigrationsHistory table:
INSERT INTO [__EFMigrationsHistory] ([MigrationId], [ProductVersion])
VALUES (N'20190704144924_Initial-PostCleanup', N'2.2.4-servicing-10062');
GO
Remove GO commands, since we will put the create script in an IF statement:
Replace GO\r\n\r\n with nothing.
Now open up your migration file (the C# file, not the sql file) and replace the Up method with the following:
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(#"
DECLARE #migrationsCount INT = (SELECT COUNT(*) FROM [dbo].[__EFMigrationsHistory])
IF #migrationsCount = 0
BEGIN
% PASTE YOUR EDITED SQL SCRIPT HERE %
END
");
}
Done! Everything should work now!
Be sure to compare the database schema, and data before and after for the new database. Everything that's not part if your EF Code model is not part of the new database.
A bit late, but we had the same problem in our current project. Above 400 migraitons and 6m lines of code inside .Designer. Here is how we managed to resolve this problem:
MigrationProject.csproj
<PropertyGroup>
...
<DefaultItemExcludes Condition="'$(Configuration)' == 'Debug' ">$(DefaultItemExcludes);Migrations\**\*.Designer.cs</DefaultItemExcludes>
</PropertyGroup>
This way you don't need to reset migrations to a clear state neither delete .Designer files. You can always change configuration to Release and use .Designer files by any means necessary.
To reset all migrations and updates from scratch (Assume no useful data on disk), the following steps may be useful.
(1) Ensure that program.cs file is not optimized for creating/ updating Database by Database.EnsureCreate command, as this command prevents Migrations.
(2) Delete folder Migrations.
(3) dotnet build
(4) dotnet ef database update 0 -c yourContextFile
(5) dotnet ef migrations add init -c yourContextFile
(6) dotnet ef database update -c yourContextFile

How do I use Entity Framework in Code First Drop-Create mode?

I'm using Entity Framework v4. I have followed the instructions in the Nerd Dinner tutorial. I'm currently in development mode (not released to any higher environments) and would like for tables to be recreated on each new deployment, since the models are still highly volatile and I don't care to retain data. However, this does not occur. Tables are not created/modified, or anything happening to the DB. If I move to a migrations model by using the Package Manager commands: enable-migrations, add-migration (initial), this works and uses my migrations. However, since I don't yet want to have granular migrations and only want my initial create script, I am forced to delete the migrations folder, redo the commands (enable-migrations, add-migration) and delete the database manually, every time I change anything.
How do I get the drop/create behavior of code first to occur?
Use DropCreateDatabaseAlways initializer for your database. It will always recreate database during first usage of context in app domain:
Database.SetInitializer(new DropCreateDatabaseAlways<YourContextName>());
Actually if you want to seed your database, then create your own initializer, which will be inherited from DropCreateDatabaseAlways:
public class MyInitializer : DropCreateDatabaseAlways<YourContextName>
{
protected override void Seed(MagnateContext context)
{
// seed database here
}
}
And set it before first usage of context
Database.SetInitializer(new MyInitializer());
If the database already exists and you want to make changes to your model, you use DropCreateDatabaseIfModelChanges<YourContextName>

How to skip previous ef migrations on a new database that already contains those changes?

The scenario I'm having problems with is as follows:
Code First model created using EF 4.1 and corresponding database generated
EF upgraded to 4.3.1 and a migration added (using IgnoreChanges because no model changes) to start using migrations (__MigrtionHistory table created and EdmMetadata dropped) - let's call it 'InitialMigration'
Model is changed, say a new property is added and a migration that adds a corresponding column is generated (let's call this one 'NewPropertyMigration') and then applied to a Dev database (using our version of Migrate To Latest initialization strategy)
When code is promoted to production the database there is updated with new column as expected
Then when a brand new database is created in Dev and because it is based on latest model it will include the new column right after it is created but when the initialization strategy is run it still finds 'InitialMigration' and 'NewPropertyMigration' as pending but they both fail because EdmMetadata is not there so nothing to be removed and new column is already there so can't add
When I check __MigrationHistory table on this new database it only contains 'InitialCreate' entry so that would explain why the other two migrations are considered as pending. But I can't see how they would ever get into this table without being applied and at the same time they don't really need to be applied because database already contains any changes they cover.
What am I missing?
I'll just add that it seems a bit suspicious that the Model column in 'InitialCreate' is different to the one in 'NewPropertyMigration' even though they represent the same model. Could that be the cause?
Update:
This is the code I use to create new database and apply any migrations automatically at the same time
public class MigratePaymentsToLatestVersion<TContext> : IDatabaseInitializer<TContext> where TContext : DbContext
{
public void InitializeDatabase(TContext context)
{
// Create a brand new database if it doesn't exist but still apply any pending migrations
context.Database.CreateIfNotExists();
var migrator = new DbMigrator(configuration);
migrator.Update();
}
}
Update 2: Simpler scenario showing the same problem
While investigating this further I've been able to reproduce the behaviour in a much simpler scenario described below:
Create simple model and context
In a test application add one entity - with all default settings database is automatically created and object added
Enable migrations - 'InitialCreate' migration is generated
Update-Database - command returns no pending migrations because 'InitialCreation' is already in __MigrationHistory
Delete database and re-run test application - database is re-created automatically again
At this point I can't add any additional migrations or run update-database because 'InitialCreation' migration is seen as pending but cannot be applied because all entities already exist
I have it sorted now. The crucial bit I was missing seems to be related to the way you move from 4.1 to 4.3. I have been following steps from ef-4-3-migrations-and-existing-database. What seems to work better is the procedure described in the SO question how-to-use-migrations-on-an-existing-db.
If you compare both of them you'll see that one relies on -IgnoreChanges when you do first migration (called 'InitialMigration') while the other one creates a full 'InitialCreate' migration that contains your entire model at that point in time. Two important consequences of the latter approach are:
- when creating a brand new database InitialCreate migration, which contains full definition of the model, is used to create database instead of 'the other code' (not sure exactly but guessing that this is the part that is needed when migrations are not enabled) that generates database based on the model
- new database is created with up-to-date model and with all migrations listed in __MigrationHistory
So with the InitialCreate approach I am able to apply migrations to existing databases (for them the InitialCreate is simply skipped because an entry in history is added manually as part of the procedure) and at the same time I am able to create brand new databases and add new migrations to them without getting an error that model and db are out-of-sync.
I hope that helps people who, like me, followed the first procedure.