How can I override SQL scripts generated by MigratorScriptingDecorator - entity-framework

Using Entity Framework 4.3.1 Code first, and Data Migrations.
I have written a utility to automatically generate the Migration scripts for a target database, using the MigratorScriptingDecorator.
However, sometimes when re-generating the target database from scratch, the generated script is invalid, in that it declares a variable with the same name twice.
The variable name is #var0.
This appears to happen when there are multiple migrations being applied, and when at least two result in a default constraint being dropped.
The problem occurs both when generating the script form code, and when using the Package Manager console command:
Update-Database -Script
Here are the offending snippets form the generated script:
DECLARE #var0 nvarchar(128)
SELECT #var0 = name
FROM sys.default_constraints
WHERE parent_object_id = object_id(N'SomeTableName')
and
DECLARE #var0 nvarchar(128)
SELECT #var0 = name
FROM sys.default_constraints
WHERE parent_object_id = object_id(N'SomeOtherTableName')
I would like to be able to override the point where it generates the SQL for each migration, and then add a "GO" statement so that each migration is in a separate batch, which would solve the problem.
Anyone have any ideas how to do this, or if I'm barking up the wrong tree then maybe you could suggest a better approach?

So with extensive use of ILSpy and some pointers in the answer to this question I found a way.
Details below fo those interested.
Problem
The SqlServerMigrationSqlGenerator is the class ultimately responsible for creating the SQL statements that get executed against the target database or scripted out when using the -Script switch in the Package Manager console or when using the MigratorScriptingDecorator.
Workings
Examining the Genearate method in the SqlServerMigrationSqlGenerator which is responsible for a DROP COLUMN, it looks like this:
protected virtual void Generate(DropColumnOperation dropColumnOperation)
{
RuntimeFailureMethods
.Requires(dropColumnOperation != null, null, "dropColumnOperation != null");
using (IndentedTextWriter indentedTextWriter =
SqlServerMigrationSqlGenerator.Writer())
{
string value = "#var" + this._variableCounter++;
indentedTextWriter.Write("DECLARE ");
indentedTextWriter.Write(value);
indentedTextWriter.WriteLine(" nvarchar(128)");
indentedTextWriter.Write("SELECT ");
indentedTextWriter.Write(value);
indentedTextWriter.WriteLine(" = name");
indentedTextWriter.WriteLine("FROM sys.default_constraints");
indentedTextWriter.Write("WHERE parent_object_id = object_id(N'");
indentedTextWriter.Write(dropColumnOperation.Table);
indentedTextWriter.WriteLine("')");
indentedTextWriter.Write("AND col_name(parent_object_id,
parent_column_id) = '");
indentedTextWriter.Write(dropColumnOperation.Name);
indentedTextWriter.WriteLine("';");
indentedTextWriter.Write("IF ");
indentedTextWriter.Write(value);
indentedTextWriter.WriteLine(" IS NOT NULL");
indentedTextWriter.Indent++;
indentedTextWriter.Write("EXECUTE('ALTER TABLE ");
indentedTextWriter.Write(this.Name(dropColumnOperation.Table));
indentedTextWriter.Write(" DROP CONSTRAINT ' + ");
indentedTextWriter.Write(value);
indentedTextWriter.WriteLine(")");
indentedTextWriter.Indent--;
indentedTextWriter.Write("ALTER TABLE ");
indentedTextWriter.Write(this.Name(dropColumnOperation.Table));
indentedTextWriter.Write(" DROP COLUMN ");
indentedTextWriter.Write(this.Quote(dropColumnOperation.Name));
this.Statement(indentedTextWriter);
}
}
You can see it keeps track of the variables names used, but this only appears to keep track within a batch, i.e. a single migration. So if a migratin contains more than one DROP COLUM the above works fine, but if there are two migrations which result in a DROP COLUMN being generated then the _variableCounter variable is reset.
No problems are experienced when not generating a script, as each statement is executed immediately against the database (I checked using SQL Profiler).
If you generate a SQL script and want to run it as-is though you have a problem.
Solution
I created a new BatchSqlServerMigrationSqlGenerator inheriting from SqlServerMigrationSqlGenerator as follows (note you need using System.Data.Entity.Migrations.Sql;):
public class BatchSqlServerMigrationSqlGenerator : SqlServerMigrationSqlGenerator
{
protected override void Generate
(System.Data.Entity.Migrations.Model.DropColumnOperation dropColumnOperation)
{
base.Generate(dropColumnOperation);
Statement("GO");
}
}
Now to force the migrations to use your custom generator you have two options:
If you want it to be integrated into the Package Manager console, add the below line to your Configuration class:
SetSqlGenerator("System.Data.SqlClient",
new BatchSqlServerMigrationSqlGenerator());
If you're generating the script from code (like I was), add a similar line of code to where you have your Configuration assembly in code:
migrationsConfiguration.SetSqlGenerator(DataProviderInvariantName,
new BatchSqlServerMigrationSqlGenerator());

Related

Entity Framework Code First post migration step?

Is there some way to add a post migration method to Code First EF migration?
All the stored proc's are in the Visual Studio project. Right now there is an approach to load the stored proc resource from the file and put it into it's own migration:
protected override void Up(MigrationBuilder migrationBuilder)
{
var script = ScriptMgr.LoadStoredProc( "StoredProcThatChanged.sql" );
migrationBuilder.Sql( script );
}
There is a weak link in this process: Each time the script changes (StoredProcThatChanged.sql) a new migration needs to be created to make sure it executes again. The problem is the previous migration is also loading the same file. When generating a new script, the process reads in the one file both times, effectively changing the previous migration. Which is a classic no-no.
This would be resolved if there is a post migration method where ALL stored proc's can be reapplied to the DB. Is such a step possible? If so, now is it done?
I have been digging into the efcore source code and it looks like it is possible, not ideal, but there might be a way...
It looks like efcore has an interface called IMigrator. It contains the method string GenerateScript(...). The implementation of it, class Migrator, has comments all over the place saying that it's implementation of GenerateScript is internal and subject to change. But... It looks to me like I can achieve my end goal:
class MyMigrator : Microsoft.EntityFrameworkCore.Migrations.Internal.Migrator
{
public string GenerateScript(
string? fromMigration = null,
string? toMigration = null,
MigrationsSqlGenerationOptions options = MigrationsSqlGenerationOptions.Default)
{
var result = base.GenerateScript( fromMigration, toMigration, options);
results += MyPostSteps(...);
return results;
}
}
Will this work and does anyone know how I might go about replacing the default Migrator with the MyMigrator?

Entity Framework telling me the model backing the context has changed

I have a weird problem with Entity Framework code first migrations. I've been using EF and code first migrations on a project for months now and things are working fine. I recently created a new migration and when running Update-Database a restored backup of my database I get this error:
The model backing the context has changed since the database was
created. Consider using Code First Migrations to update the database
The migration does something like the following:
public override void Up()
{
using (SomeDbContext ctx = new SomeDbContext())
{
//loop through table and update rows
foreach (SomeTable table in ctx.SomeTables)
table.SomeField = DoSomeCalculation(table.SomeField);
ctx.SaveChanges();
}
}
I'm not using the Sql() function because DoSomeCalculation must be done in C# code.
Usually when I get something like this is means that I have updated my model somehow and forgot to create a migration. However that's not the case this time. The weird thing is that the error isn't even occurring on a migration that I created a few days ago and had been working fine.
I looked a quite a few articles about this and they all seems to say call
Database.SetInitializer<MyContext>(null);
Doing that does seem to work, but my understanding (based on this article) is that doing that will remove EF's ability to determine when the database and model are out of sync. I don't want to do that. I just want to know why it thinks they are out of sync all of a sudden.
I also tried running Add-Migration just to see if what it thought changed about the model but it won't let me do that stating that I have pending migrations to run. Nice catch 22, Microsoft.
Any guesses as to what's going on here?
I'm wondering if maybe the fact that migration listed above is using EntityFramework is the problem. Seems like maybe since it's not the latest migration anymore, when EF gets to it tries to create a SomeDbContext object it checks the database (which is not fully up to date yet since we're in the middle of running migrations) against my current code model and then throws the "context has changed" error.
It's possibly related to your using EF within the migration. I'm not sure how you're actually managing this, unless you've set a null database initialiser.
If you need to update data within a migration, use the Sql function, e.g.
Sql("UPDATE SomeTable SET SomeField = 'Blah'");
You should note that the Up() method is not actually running at the time of doing the migration, it's simply used to set up the migration which is then run later. So although you may think you've done something in the migration above the bit where you're using EF, in reality that won't have actually run yet.
If you cannot refactor your calculation code so it can be written in SQL, then you would need to use some mechanism other than migrations to run this change. One possibility would be to use the Seed method in your configuration, but you would need to be aware that this does not keep track of whether the change has been run or not. For example...
internal sealed class Configuration : DbMigrationsConfiguration<MyContext>
{
public Configuration()
{
AutomaticMigrationsEnabled = false;
}
protected override void Seed(MyContext context)
{
// Code here runs any time ANY migration is performed...
}
}
I tried replacing the EntityFramework code with regular ADO.NET code and it seems to work. Here is what it looks like:
public override void Up()
{
Dictionary<long, string> idToNewVal = new Dictionary<long, string>();
using (SqlConnection conn = new SqlConnection("..."))
{
conn.Open();
using (SqlCommand cmd = new SqlCommand("SELECT SomeID, SomeField FROM SomeTable", conn))
{
SqlDataReader reader = cmd.ExecuteReader();
//loop through all fields, calculating the new value and storing it with the row ID
while (reader.Read())
{
long id = Convert.ToInt64(reader["SomeID"]);
string initialValue = Convert.ToString(reader["SomeField"]);
idToNewVal[id] = DoSomeCalculation(initialValue);
}
}
}
//update each row with the new value
foreach (long id in idToNewVal.Keys)
{
string newVal = idToNewVal[id];
Sql(string.Format("UPDATE SomeTable SET SomeField = '{0}' WHERE SomeID = {1}", newVal, id));
}
}

Modified Entity Workspace is ignored when generating SQL

I have an Entity Model (EDMX) file and EF 4.3.1. I am trying to make a run-time modification to the EDMX (change the store:Schema of the tables/entitySets used in generating the query). I am using code based on the EF Model Adapter project by B. Haynes.
It appears that I can make the changes to the XML just fine using the schema model adapter, and load it into a metadata workspace and then pass it to the connection. However, when the query is generated by the DbContext/EF framework code, it uses the old value for the schema.
Create a new MyEntities
Load the EDMX medata data manually
Replace the "store:Schema" value with the new desired value
Create the metadata workspace from the modified XML
Return a new EntityConnection using that modified workspace
Query the data (from x in db.Table select x)
This is the basics of what is going on. We create our dbContext by creating a new EntityConnection based on the modified workspace and the connection. There is also some provider wrapping and such going on, for logging, etc. Sorry if that's confusing.
public MyEntities(): base( this.Create("name=MyEntitiesConnStr"), true)
{
}
public static DbConnection Create(string connectionString)
{
var ecsb = ConnectionHelper.ResolveConnectionStringDetails(connectionString);
var workspace = GetModifiedEntityWorkspace(ecsb);
var storeConnection = DbProviderFactories.GetFactory(ecsb.Provider).CreateConnection();
Debug.Assert(storeConnection != null, "storeConnection != null");
storeConnection.ConnectionString = ecsb.ProviderConnectionString;
var wrappedConnection = MyWrappedConnetion.WrapConnection(storeConnection);
_log.Debug("Creating new entity connection");
var newEntityConnection = new EntityConnection(workspace, wrappedConnection);
WireEvents(wrappedConnection);
return newEntityConnection;
}
private static MetadataWorkspace GetModifiedEntityWorkspace(EntityConnectionStringBuilder ecsb)
{
// instantiate manager class
// read all XML items from the embedded resources
// change the store:schema to the real one for this environment
// <EntitySet Name="..." store:Type="Tables" store:Schema="SCM" store:Name="TBLX">
// create new MetadataWorksspace(ssdl,cdl,...)
}
Any idea where/why it is still getting the old Schema value for the query? I think it worked right with EF 4.0,
Turns out the problem was with the <DefiningQuery> element under the entity set.
This element contains a definition of the base query used to define the entity. Perhaps something changed and now they refer to that for speed reasons. It is necessary to modify that query as well, and then the schema change will take effect.
<EntitySet Name="MYTABLE" store:Type="Tables" store:Schema="MYSCHEMA" ...>
<DefiningQuery>
SELECT MYTABLE.COLUMN [...REPEAT..]
FROM MYSCHEMA.MYTABLE AS MYTABLE
</definingQuery>
So, changing "MYSCHEMA" in both those locations fixes it. Just the store:Schema element is not enough.

Entity SQL doesn't work if edmx file in another project

I'm trying to use Enitity SQL to query data, but if the edmx file in another project, there will be an exception thrown. Below is my test steps.
Create a Class Library project and add an edmx file to it, create from database.
Create a Console Application, add the Class Library project to reference and copy the app.config file to this project.
Write the code as below
using (NorthwindEntities context = new NorthwindEntities())
{
string queryString = #"SELECT VALUE cus
FROM NorthwindEntities.Customers AS cus
WHERE cus.ID > 10";
ObjectQuery<Customers> cusQuery =
context.CreateQuery<Customers>(queryString);
List<Customers> cusList = cusQuery.ToList();
}
When I run the Console Application project, an exception is thrown: "'ID' is not a member of type 'NorthwindModel.Customers' in the currently loaded schemas."
It seems the schema doesn't loaded into the project, anyone has ideas?
Addional question: in this query, I select all the properties of this type, if I only select some of the properties, how to return an anonymous type of ObjectQuery?
Any suggestions are appreciate.
This is not an EF problem.
You have written the SQL statement by hand, you are not using the table definition that is in the EDMX file.
If you try to execute the SQL statement in the Query prompt of SQL Server, you will see that it fails there as well.
Try something like this:
var cus = from customers in context.Customers select customers;
var cusList = cus.ToList();

Can not find out the function for stored procedure in Entity Framework

Based on a database myDB, I generate edmx for all table and compile the project. Then I create stored procedure myProc in myDB. Then I update the model by "Update Model from database" in the node Stored Procedure and add myProc. It is fine. Then "Create a function import" on myProc. It is fine. Then I compiled the project, it is fine.
The return type for this import function is scalars(string) because myProc return a string.
Then I want to use this function for this stored procedure, but I can find out the function.
How to find out the matching function and call it in code?
In EF 3.5 only functions that return Entities show up in ObjectServices.
I.e. importing pulls the Function into the Conceptual Model, but NOT into the code-generation.
We have addressed this problem in 4.0.
In the meantime you can call the function using eSQL.
i.e. something like this (pseudo code):
EntityConnection connection = ctx.Connection as EntityConnection;
//Open the connection if necessary
connection.Open()
//Create the command
EntityCommand command = new EntityCommand();
command.CommandText = "Function(#p1,#p2");
command.Parameters.Add(...);
command.Parameters.Add(...);
command.Connection = connection;
string s = command.ExecuteScalar().ToString();
Hope this helps
Alex