EF Core Migrations string replacement - entity-framework-core

I have a scenario where I deploy to different environments (development, testing and production), and each of these databases use different table name prefixes, along with different schema names.
Currently, in my CD pipeline I generate an idempotent script, and do string replacement on my 'variables' in my migration, however I'd like to utilize the dotnet ef database update command which makes it tricky as all my past migrations have been generated with a variable name within their table names, etc.
Is it possible that I can read from an environment file what that variable is while creating my DbContext and map/replace/infer what the actual table prefixes are when running the command line?
Edit 1:
To make matters a little more complicated, we're using an Oracle database
When generating our migrations, we have assistive functions that grant access to certain database users to view the newly added tables. These users too have different names per environment.
Reworking the users, table names, etc isn't in scope of this - I'm aware of the inefficiencies

Why don't you simply replicate the entire code?
When you scaffold/migrate you get partial classes, isn't it?
So, here the steps:
set the environment for Production, with a specific BUILD
variable
prepare the migration to the production database. You
will get at least the migration partial auto-class.
surround the entire body of the migration (having practically an empty file) within #if PRODUCTION ... #endif
rename the file with some _PRODUCTION suffix in the name. (it's partial class, it will be seen anyway)
prepare for DEVELOPMENT as in step 1.
migrate again, you will get again part of the partial class (the one that disappeared because the build variable changed.
surround everything with another #if DEVELOPMENT ... #endif and
change the file name suffixing it with _DEVELOPMENT.cs
loop unil you finish all your enviroments
do the same compiler selection for the variables holding the names of the users. You can be smart here, there should not be anything auto-generated in the class holding the users, so you can define that only class with parts of the names changing again with insertion/removals with #if and #endif.
I never happened to fall in such a weird case, but this would be the first solution I would try. The codebase is not growing as in any instant there is only one enviroment selected, the users get managed and EF can generate its automatic parts independently from your code.
Some side notes:
a sandbox or a clone of the database would be more suitable for testing. After you tested the development enviroment, will you test again the production one from scratch?
Changing the names of the users with just a prefix to differentiate the environment (without having all the details, I may not catch the full meaning of this) seems to bring small value to the effort of separation.
EDIT:
I don't change the previous text because it may drive you on theright path, but it is likely that you should not rename the migration file, otherwise the next migration will create anothe file with the base name. Just surround the entire migration class with the compiler directives, and let EF create another class in the same file at steps 2-3 and 5-6.
cheers
Alex

Without modifying too much your current flow you can do your string replacement before creating the migration and then, for each environment, use a different folder:
dotnet ef migrations add InitialCreate --context DbContext --output-dir Migrations/DevMigrations
dotnet ef migrations add InitialCreate --context DbContext --output-dir Migrations/ProdMigrations
But a cleaner approach would be: as you already know the differences between the various environments you could add that specific information in different db context and manage different DB schema/table names in the code itself of the dedicated DbContext. So in each DBContext you can put default schema name, link the entities to the correct table using a dedicated OnModelCreating and all the info that is different among the environments and then you can evolve each of these migrations:
dotnet ef migrations add InitialCreate --context DevContext --output-dir Migrations/DevMigrations
dotnet ef migrations add InitialCreate --context ProdContext --output-dir Migrations/ProdMigrations
To add the common relations/definitions that are the same for every environment you can use a dedicated IEntityTypeConfiguration to group your "business related" EF core configurations in a common place (the relations for example). This will avoid code duplication and let you have the "infrastructure related" EF core configurations defined in each dedicated DbContext (schema and table names), as they are actually different.
update To address the requirement to assign permissions you can add an empty migration where you can put sql scripts that perform whatever you need
So at the very end, in your pipeline, you can infer from the name of the stage/environment the name of DbContext and the folder where to save/fetch these migrations.
If you follow any of these approaches probably you will need to "sync" the already existing DB, see here and here,in the "Add your baseline migration" part

Related

EF Core Migrations manual edits possible?

I am using EF Core 2.0 in my sample project with some value object configurations. I modify the code and generate migrations via CLI command line. In the last migration rather than adding a new database table as it should, it is trying to rename existing tables to each other and create an extra table for existing one. I could not figure out the reason for it.
Issue is, since with EF Core the snapshot is a separate auto-generated file from the migration itself I don't want to modify the snapshot.
I only want to modify the migration script so that it will not rename multiple tables, and then generate the snapshot from the migrations I created.
I did not see any command for this in the CLI - is it such a bad practice to modify the scaffolded migration and regenerate or am I missing some obvious new link where how to manually modify migration scripts is explained?
Thanks a bunch.
Update 1: After comments, added info about the snapshot from this link.
Because the current database schema is represented in code, EF Core doesn't have to interact with the database to create migrations. When you add a migration, EF determines what changed by comparing the data model to the snapshot file. EF interacts with the database only when it has to update the database. +
I examined my generated snapshot code from source control. It exactly has added one extra table as what I needed.
The migration script to generate this is hectic at best - renaming multiple tables to each other and then warning that this could break causing multiple issues.
Since this is a sample project for me with only mock data as of now at least, I decided to go for it and not break the automated scripts. I am willing to lose some mock data at this stage rather than wasting time on it.
If this were in a production database I would be extremely careful to manually create the same result with intervention modifying both the scaffold and the migration file.
I am accepting this one as an answer (basically saying current EF Core does not support it to the best of my current knowledge) since there is no other candidate now - I will be more than glad to accept if any better answer shows up.

EF Core Why Do Migrations need names?

What is the point in given migrations names?
dotnet ef migrations add {MigrationName}
The file that it generates seems to start with the timestamp the migration was create, so whats the value of the name?
For the same reason source control usually requires commit messages--to help developers differentiate between them.
The dotnet ef migrations add command is always going to require a name, but you could override the internal IMigrationsIdGenerator to ignore it and just use the timestamp.
But since you have to specify them anyway, you might as well just use a sequence like M1, M2... if you don't feel they're important.
because that name gets store for possible rollback.
Also, with the command posted it has NOTHING to do with entityframework below 6.3
reference table _EntityMigration in your database...

Development process for Code First Entity Framework and SQL Server Data Tools Database Projects

I have been using Database First Entity Framework (EDMX) and SQL Server Data Tools Database Projects in combination very successfully - change the schema in the database and 'Update Model from Database' to get them into the EDMX. I see though that Entity Framework 7 will be dropping the EDMX format and I am looking for a new process that will allow me to use Code First in Combination with Database Projects.
Lots of my existing development and deployment processes rely on having a database project that contains the schema. This goes in source control is deployed along with the code and is used to update the production database complete with data migration using pre and post deployment scripts. I would be reluctant to drop it.
I would be keen to split one big EDMX into many smaller models as part of this work. This will mean multiple Code First models referencing the same database.
Assuming that I have an existing database and a database project to go with it - I am thinking that I would start by using the following wizard to create an initial set of entity and context classes - I would do this for each of the models.
Add | New Item... | Visual C# Items | Data | ADO.NET Entity Data Model | Code first from database
My problem is - where do I go from there? How do I handle schema changes? As long as I can get the database schema updated, I can use a schema compare operation to get the changes into the project.
These are the options that I am considering.
Make changes in the database and use the wizard from above to regenerate. I guess that I would need to keep any modifications to the entity and/or context classes in partial classes so that they do not get overwritten. Automating this with a list of tables etc to include would be handy. Powershell or T4 Templates maybe? SqlSharpener (suggested by Keith in comments) looks like it might help here. I would also look at disabling all but the checks for database existence and schema compatibility here, as suggested by Steve Green in the comments.
Make changes in code and use migrations to get these changes applied to the database. From what I understand, not having models map cleanly to database schemas (mine don't) might pose problems. I also see some complaints on the net that migrations do not cover all database object types - this was also my experience when I played around with Code First a while back - unique constraints I think were not covered. Has this improved in Entity Framework 7?
Make changes in the database and then use migrations as a kind of comparison between code and the database. See what the differences are and adjust the code to suit. Keep going until there are no differences.
Make changes manually in both code and the database. Obviously, this is not very appealing.
Which of these would be best? Is there anything that I would need to know before trying to implement it? Are there any other, better options?
So the path that we ended up taking was to create some T4 templates that generate both a DbContext and our entities. We provide the entity T4 a list of tables from which to generate entities and have a syntax to indicate that the entity based on one table should inherit from the entity based on another. Custom code goes in partial classes. So our solution looks most like my option 1 from above.
Also, we started out generating fluent configuration in OnModelCreating in the DbContext but have swapped to using attributes on the Entities (where attributes exist - HasPrecision was one that we had to use fluent configuration for). We found that it is more concise and easier to locate the configuration for a property when it is right there decorating that property.

Update model snapshot of last migration in Entity Framework and reapplying it

I'm using EF6 code-first migrations for existing database but initial DbContext does not fully cover existing schema (since it's massive). So from time to time I have to make updates to the model in database-first style. For example when I need an entity mapping for a table or a column that is already in the database but not reflected in the code I do the following:
Make all change (add new entity, rename the column mapping or add new property)
Scaffold migration representing the latest model snapshot stub_migration
Copy-paste latest serialized model from stub_migration to the last_migration resource file
Delete stub_migration
Revert last_migration in database
Update-Database so that model snapshot in [__MigrationHistory] table would be also updated
I understand that this aproach is a bit hackish and the proper way would be to leave empty stub_migration but this would force lots of empty migrations which I would rather avoid.
Looking at a similar scenario from MSDN article (Option 2: Update the model snapshot in the last migration) I wouldn't imagine that there is an easier way rather than writing power shell script, managed code or both to make it work. But I would rather ask community first before diving deep into it.
So I wonder: is there a simple way to automate generation of new model snapshot in latest migration and reaplying it?
I'm doing something similar. I have a large database and I am using the EF Tools for VS 2013 to reverse engineer it in small parts into my DEV environment. The tool creates my POCOs and Context changes in a separate folder. I move them to my data project, create a fluent configuration and then apply a migration (or turn automigration on).
After a while I want a single migration for TEST or PROD so I roll them up into a single migration using the technique explained here: http://cpratt.co/migrating-production-database-with-entity-framework-code-first/#at_pco=smlwn-1.0&at_si=54ad5c7b61c48943&at_ab=per-12&at_pos=0&at_tot=1
You can simplify the steps for updating DbContext snapshot of the last migration applied to database by re-scaffolding it with Entity Framework:
Revert the last migration if it is applied to the database:
Update-Database -Target:Previous_Migraton
Re-scaffold the last migration Add-Migration The_name_of_the_last_migration which will recreate the last migrations *.resx and *.Designer.cs (not the migration code), which is quite handy.
Those 2 steps are covering 4 steps (2-5) from original question.
You can also get different bahavior depending on what you want by specifying the flags -IgnoreChanges and (or) -Force
And by the way, the major problem with the updating the DbContext snapshot is not how to automate those steps, but how to conditionally apply them to TEST/PROD environments depending on whether you actually want to suppress the warning because you've mapped existing DB-first entities in you DbContext or you want it it to fail the build in case you've created new entities and forgot to create a code-first migration for them.
So, try to avoid those steps altogether and maybe create empty migrations when you just want to map existing tables to your code.

EF codefirst migrations how it works in a production environment

I am a newbie in Codefirst and I do not understand how it works correctly.
I created 3 migrations, migration 1..3 via "Add-Migration" command and issued the relative update with "Update Database".
I have a Configuration.cs file in my Migration directory and custom database initializer (I am working on MySQL) I created in order to seed initial data.
I do not know what happens behind the scenes in the production environment where I do not have any database yet.
Who is responsible to create and update the database? Are the migrations executed one after the other?
Can you suggest me how this process works in production and share useful links?
Regards,
Roberto
Assuming you are using Automatic Migrations - All migrations are called one after another, in the sequence in which they were created. Their Up function is called. Afterwards the Seed function on your Configuration class should fire.
Each migration has a sort of a Time-Stamp inside its file name, like this:
if you are creating lots of databases out of a single context class like I do, and you are afraid of not having control over migrations, you can use manual migrations. It is called DbMigrator.
See my/others answer here how to use it.