End of long day of testing various scenarios where I don't have to recreate a production database...
We started off with EF, and didn't get wise enough during development to move from automatic migrations to named migrations. Now I'm trying to rewind the clock, and create an initial migration that aligns with the production database.
Is this possible to align a model has with an automatic migration has in the migration table?
Should I just create an empty migration to get started with named migrations? My only problem with this is how to create the DB when a new developer joins... I could simply restore the db, and then apply migrations, but that ruins a beautiful EF migration story!
Delete the production DB, create, and write a script to re-import the data (sounds hacky).
Another wrinkle - the DB was created with EF5, and we are now developing with EF6.
Thanks in advance for your help.
It should be possible:
Delete the __MigrationHistory table
Delete any migrations in your project
Disable automatic migrations in your migrations configuration class
Add-Migration InitialCreate
Update-Database -Script
Execute the portion of the script that creates the __MigrationHistory table and inserts a row into it
Repeat steps 1 & 6 for any other existing databases
I also strongly recommend reading Code First Migrations in Team Environments.
If you don't like step 7 of bricelam's answer. Then keep reading.
Because upgrading from automatic migrations to manual can also be automated. And you don't need to delete the migrationhistory table like bricelam says.
In this case I assume you have one development machine, and multiple other (development) machines where databases already exist.
On your development machine you move your database into a safe location (keeping it safe to test your automatic upgrade procedure). (You might want to make sure your code is in the same exact state as all the database which you want to upgrade).
Delete any migrations in your project (you should't have any yet)
Disable automatic migrations in your migrations configuration class
Add-Migration InitialCreate
Put a backup (copy!) of your database back into place
On startup of your application (or your upgrade app) you do something like:
using (var db = new YourDatabaseContext())
{
InitialCreate.SkipInitialCreate = db.Database.Exists();
}
And in your InitialCreate you add something like:
public static bool SkipInitialCreate = false;
public override void Up()
{
// Skip initial create
if (SkipInitialCreate)
return;
This assumes that if a database exists it has the same structure as the one you had on your dev machine. Of course this could be way more complex than that for you. For more control you could inspect DbMigrator object for your config, and skip more than one migration. (would have been nice if you could query the hash of the model in the DbMigrator...)
After the code in step 4 add something like:
var configuration = new YourDatabaseConfiguration();
var migrator = new DbMigrator(configuration);
migrator.Update();
Start your application on your dev machine to test the upgrade. Your history table should have an automaticmigration record and a initialcreate record.
If everything works out you should be able to simple deploy and run the app on machines where the db already exists.
One very important aspect (and maybe the reason bricelam says you needed to delete the history table) is that if the AutomaticMigration record in that table is newer than one of your manual migrations you are going to have a Bad time. Because the migrator uses the dates to sort out the work it needs to do.
So if there are systems out there still automatically upgrading to the newest auto-upgrade model, then you are in a bit of a pickle.
Related
I have a PostgreSQL db that is used by a Nest.Js / Prisma app.
We changed the name of a field in the Prisma schema and added a new field.
Now, when we want to update the PostreSQL structure, I'm running, as suggested by Prisma, the following commands:
npx prisma generate
and then
npx prisma migrate dev --name textSettings-added --create-only
The idea is to use the --create-only flag to review the migration before it is actually made.
However, when I run it I get a list of the changes to be made to the DB and the following message:
We need to reset the PostgreSQL database "my_database" at "my_db_name#cluster.us-east-1.rds.amazonaws.com:5432".
Do you want to continue? All data will be lost.
Of course I choose not to continue, because I don't want to lose the data. Upon inspection I see that the migration file actually contains DROP TABLE for the tables that I simply wanted to modify. How to avoid that?
So how do I run the update without affecting the data?
UPDATE:
I saw that running with --create-only creates a migration which can then be implemented on the DB level using prisma migrate dev, however, in that migration file there are still some commands that drop my previous tables because of some new parameters inside. How can I run prisma migration without deleting my PostgreSQL data?
UPDATE 2:
I don't want Prisma to drop my tables when I just updated them. The migration file generated, however, drops them and then alters them. Do you know what's the best procedure to avoid this drop? I saw somewhere I could first manually update the DB with the new options and then run the migration, so Prisma can find a way to update it, but that seems too manual to me... Maybe some other ideas?
For some cases like renaming tables or columns, Prisma's generated migration files need to be updated if they already contain data.
If that applies to your use case, Prisma's docs suggest to:
Make updates to the prisma schema
Create migration file without applying it (--create-only flag)
Update the migration script to remove the drops and instead write your custom query (e.g. RENAME <table_name> TO <new_name>)
Save and apply the migration (npx prisma migrate dev)
Note that those changes can lead to downtime (renaming a field or model), for which they have outlined the expand and contract pattern.
It might be a Prisma bug: https://github.com/prisma/prisma/issues/8053
I also recently had this situation. It probably should not try to run migration if you only want to create migration file.
But overall it is expected with Prisma to recreate your db sometimes. If you migration is breaking then it will be required to reset the data anyway when you apply it.
I suggest you to create some seeding script so you could consistently re-create the database state, it's very useful for your development environment.
More info
We wish to get rid of 100s of migration classes as DB schema in production is final.
Here are the steps I followed:
Delete Migrations folder.
Add-Migration -??
What command line switches, could help us?
EDIT:
If all goes well Up() method of migration should be empty right? For
example following is wrong generation on Add-Migration. Because if we
execute the project we will get duplicate table errors.
public partial class Sanity : DbMigration
{
public override void Up()
{
CreateTable(
"dbo.AccountPreferences",
c => new
{
AccountID = c.Guid(nullable: false),
}
.... for 1000s of tables
}
}
A clean migration would be something: when trying Add-Migration on subsequent changes, should not be getting any error.
Unable to generate an explicit migration because the following
explicit migrations are pending: [201712281054591_Sanity]. Apply the
pending explicit migrations before attempting to generate a new
explicit migration.
As you can see if we happen to execute Update-Database will get table already exist error.
Are we forced to always retains all migration copies?
See if this can help:
MVC3 and Code First Migrations - "model backing the 'blah' context has changed since the database was created"
Entity framework code first - how to run Update-Database for production database
How to delete and recreate from scratch an existing EF Code First database
note:
I'm writing this from memory, if you have issues let me know and I'll recheck exactly.
Also, my knowledge on this is from slightly older versions of EF as I haven't done much work there recently, but I doubt much has changed.
From what I can tell, if you want to...
a) keep the db,
b) clean your project migrations,
c) have the 2 'match', be in sync:
do the following:
- Remove the migration folder (your project)
- Run Add-Migration Initial - then should add one migration
- caution: it is safe but do backup, change connection string etc. before the next step
- Run Update-Database -Script - that doesn't update the db but creates the SQL script, including the migration table
- find the INSERT INTO [__MigrationHistory] records, just run those (on your db), insert them into the database
...then test with Add-Migration again, to see if it is going to make anything, should yield no new migrations, empty one.
Please read through the first link above and adjust approach as needed.
I'm sure there might be easier, shorter ways to do this (via PM console) but unaware of it at the moment.
Open your database.
Clear table __MigrationHistory
Remove migrations in the folder
Run Add-Migration MigrationName
Almost the same as accepted one, but no scripting the initial migration.
Drop the __MigrationHistory db table
Remove all the migration files in the Migrations folder
Run Add-migration Initial in Package Manager Console
Comment out the code inside of the Up method in the Initial Migration
Run Update-database in PM Console (just to create a Migration Entry)
Remove comments in the Initial migration
Wonder how long it will be "final" ?
Use:
Add-Migration Initial
After removing the migrations folders
How do you get EF back in Sync with code without losing data when Update-database returns the following message
Error Message: System.Data.SqlClient.SqlException (0x80131904): There
is already an object named '' in the database.
I originally wrote this as a self-answering question as I had struggled with the proble for some time, as had a few colleagues, but unfortunately, my answer was deleted and I can't recover it.
Since it's a situation that I suspect can happen several times as people try to "clean up" old migrations, I thought I'd document it with step by step instructions.
Description of the situation we found ourselves in:
We couldn't can't create a new local database because the init script was incomplete, and couldn't apply updates to the production database because the migration scripts create tables that already exist. And, we didn't want to delete production data.
Symptom: Can't run Update-Database because it's trying to run the creation script and the database already has tables with the same name.
Error Message: System.Data.SqlClient.SqlException (0x80131904): There
is already an object named '' in the database.
Problem Background:
To understand this in more detail, I'd recommend watching both videos referenced here:
https://msdn.microsoft.com/en-us/library/dn481501(v=vs.113).aspx
To summarise, EF understands where the current database is at compared to where the code is at based on a table in the database called dbo.__MigrationHistory. When it looks at the Migration Scripts, it tries to reconsile where it was last at with the scripts. If it can't, it just tries to apply them in order. This means, it goes back to the initial creation script and if you look at the very first part in the UP command, it'll be the CreeateTable for the table that the error was occurring on.
Solution: What we need to do is to trick EF into thinking that the current database is up to date, while "not" applying these CreateTable commands since the production database already exists. Once production DB is set, we still need to be able to create local databases as well.
Step 1: Production DB clean
First, make a backup of your production db. In SSMS, Right-Click on the database, Select "Tasks > Export Data-tier application..." and follow the prompts.
Open your production database and delete/drop the dbo.__MigrationHistory table.
Step 2: Local environment clean
Open your migrations folder and delete it. I'm assuming you can get this all back from git if necessary.
Step 3: Recreate Initial
In the Package Manager, run "Enable-Migrations" (EF will prompt you to use -ContextTypeName if you have multiple contexts).
Run "Add-Migration Initial -verbose". This will Create the initial script to create the database from scratch based on the current code.
If you had any seed operations in the previous Configuration.cs, then copy that across.
Step 4: Trick EF
At this point, if we ran Update-Database, we'd be getting the original error. So, we need to trick EF into thinking that it's up to date, without running these commands. So, go into the Up method in the Initial migration you just created and comment it all out.
Step 5: Update-Database
With no code to execute on the Up process, EF will create the dbo.__MigrationHistory table with the correct entry to say that it ran this script correctly. Go and check it out if you like.
Now, uncomment that code and save.
You can run Update-Database again if you want to check that EF thinks its up to date. It won't run the Up step with all of the CreateTable commands because it thinks it's already done this.
Step 6: Confirm EF is ACTUALLY up to date
If you had code that hadn't yet had migrations applied to it, this is what I did...
Run "Add-Migration MissingMigrations"
This will create practically an empty script. Because the code was there already, there was actually the correct commands to create these tables in the initial migration script, so I just cut the CreateTable and equivalent drop commands into the Up and Down methods.
Now, run Update-Database again and watch it execute your new migration script, creating the appropriate tables in the database.
Step 7: Re-confirm and commit.
Build, test, run. Ensure that everything is running then commit the changes.
Step 8: Let the rest of your team know how to proceed.
When the next person updates, EF won't know what hit it given that the scripts it had run before don't exist. But, assuming that local databases can be blown away and re-created, this is all good. They will need to drop their local database and add create it from EF again. If they had local changes and pending migrations, I'd recommend they create their DB again on master, switch to their feature branch and re-create those migration scripts from scratch.
I am working on a web project using ASP.Net 5 and EF7.
I have imported all the tables from existing database onto my models in my project. However, I am having problems in regards with migrations.
I have created the initial migration, made some changes on particular entity, create another migration following the changes I have made and now want to apply the changes on the database.
After running this command below:
dnx ef database update [Migration]
the dnx is trying to apply the 'initial' migration with all the entities which are already in database and this causes an error as below:
{ There is already an object named ['EntityName'] in the database. }
Can you please advise on how to do the migration on the existing database?
Thanks
Saeed
In EF6 you would run a migration with the -IgnoreChanges flag and it would take a snapshot of the model without any Up() code. This is missing from EF 7(EF Core) as indicated here.
The workaround for now is delete or comment out the code for existing database objects from the Up() code of the migration and then update-database. Subsequent migrations will then include only the changes.
After 2 days I found a way for EFCore that is not in google and internet!
How My steps works?
When you have a database with 10 table and you have data in tabales that you don't want to clear's data.and after this you will create new models in your code first and runnig to existing database and you will get error "Invalid object name 'tableName'." for query to new tables and you want to create migrations and do update it to existing database but first migration will create all queries for old and new tables if you run update-database you will get "There is already an object named ['EntityName'] in the database." for your initial migration.
How fix it?
Delete all migrations and snapshot in you database project
Delete __EFMigrationsHistory table in your existing database (if exist)
Run in Package manager console:
Note before run: This code will create new context and models of your existing database to Data folder so don't forgot to check you have not Data folder in your project.
Scaffold-DbContext "your connection string"
Microsoft.EntityFrameworkCore.SqlServer -OutputDir Data
Run in Package manager console:
Note before run: Create first migration for initial database with Data folder context (OldDataBaseContext is inside of your Data folder that created by step 2)
Add-Migration initial -Context OldDataBaseContext
Delete all code in Up method inside of created initial migaration in step 3
Run in Package manager console:
Note before run: Update databse with Data folder context (OldDataBaseContext is inside of your Data folder that created by step 2)
Update-Database -Context OldDataBaseContext
Delete data folder that created in Step 2
Go to snapshot and initial migration classes and change the deleted context from Data folder to main context that is exist in your database project (just fix it for build)
Run:
Note before run: Add migration for main context that have new database changes
Add-Migration newUpdate
Run:
Update-Database
I Hope this help someone.
If you are strongly believe, that new (EF7) database schema will match your old database schema (including index and key names) - you may run 'Initial' migration to empty (temporary) database and then copy __EFMigrationHistory table from there to your real database.
Otherwise, I recommend you create empty database using migration and use sql insert into ... select commands to copy data from your old database. Without this, you will receive exceptions when upgrading database later - for example changing index will lead to DropIndex and CreateIndex migration commands, and DropIndex will fail because there is no index with this name (index exist with other, pre-EF7 name).
In my projects - old (from EF6) and new database schemes are different, and I used second option.
I started off my project by using Entity Framework Code First. When I was ready I uploaded my database and code to my host provider. Everything worked.
I need to add a new field to one of my classes and I don't want to loose the data in the database. Thus, I tried following some blog posts about using Code First Migrations. I did the following:
I backed up my remote (production) database.
I attached this database locally
I added the property to my class
PM> Enable-Migrations
PM> Add-Migration AddSortOrderToCar
PM> Update-Database
At this point I created a .bak file of the local database and then used that file to 'restore' to the remote one.
Lastly, I published the code to the remote site.
When I visit the site I get the following error message:
The model backing the 'blahblah' context has changed since the database was created. Consider using Code First Migrations to update the database.
What am I doing wrong?
From my experience that suggests that migration table is out of sync (even if your data isn't), and that's been part of the db schema now (since 4.3 I think - under system tables).
There could be many reasons and ways to experience that error , but most of the time...
The problematic part is some combination of manually backing/restoring the full database with code changes alongside - I'm not entirely certain as to why always.
In short, even if Db-s are the same migration table data might not be - and hash comparison may fail (still full restore sounds like good enough - but you have 'two sides').
What works for me is to use
Update-Database -Script
That creates a script with a 'migration difference',
which you can manually apply as an SQL script on the target server database (and you should get the right migration table rows inserted etc.).
If that still doesn't work - you can still do two things...
Remove the migration table (target - under system tables) - as per http://blogs.msdn.com/b/adonet/archive/2012/02/09/ef-4-3-automatic-migrations-walkthrough.aspx comments in there - that should fail back to previous behavior and if you're certain that your Db-s are the same - it's just going to 'trust you',
As a last resort I used - make a Update-Database -Script of the full schema (e.g. by initializing an empty db which should force a 'full script'),
find the INSERT INTO [__MigrationHistory] records,
just run those, insert them into the database,
and make sure that your databases - and code match,
that should make things run in sync again.
(disclaimer: this is not a bullet proof to work at all times, you may need to try a few things given your local scenarios - but should get you in sync)
I think in step 6 you need to run Update-Database -Verbose
Also this link is very helpful for updating database in EF with Scaffolding
http://www.asp.net/mvc/overview/older-versions/hands-on-labs/aspnet-mvc-4-entity-framework-scaffolding-and-migrations