I am trying to add an sql statement to an up-method of a migration for my current project. The database is an Ms Access database. The migrations get applied during run-time.
The situation is as follows:
I have a base Initial-create migration, which in my case is assumed to be already applied. Due to the nature of this application we have a table A which contains some kind of a foreign key, but without any sql-constraints defined. This means the foreign-key relationship is designed via program code and not in sql means an foreign key relationship. The key is an string and if there is no foreign-element the value is empty.
Now we want to add a new migration which enforces this relationship via sql-constraints. This works just fine via standard ef-core migration code, but the problem comes when the migration gets applied to a non empty database. The sql foreign key would need all the empty strings in table A to be null (otherwise we get an exception)
The seemingly easy solution was to add the following statement in the up-method of the new migration:
UPDATE A SET ForeignKeyColumn = NULL WHERE ForeignKeyColumn & \"\" = \"\""
But this results in the following exception:
System.Data.OleDb.OleDbException (0x80040E14): The database engine could not lock table 'A' because it is already in use by another person or process.
at System.Data.OleDb.OleDbCommand.ExecuteCommandTextErrorHandling(OleDbHResult hr)
at System.Data.OleDb.OleDbCommand.ExecuteCommandTextForSingleResult(tagDBPARAMS dbParams, Object& executeResult)
at System.Data.OleDb.OleDbCommand.ExecuteCommandText(Object& executeResult)
at System.Data.OleDb.OleDbCommand.ExecuteCommand(CommandBehavior behavior, Object& executeResult)
at System.Data.OleDb.OleDbCommand.ExecuteReaderInternal(CommandBehavior behavior, String method)
at System.Data.OleDb.OleDbCommand.ExecuteNonQuery()
at EntityFrameworkCore.Jet.Data.JetCommand.ExecuteNonQueryCore()
at EntityFrameworkCore.Jet.Data.JetCommand.<>c.<ExecuteNonQuery>b__40_0(Int32 _, JetCommand command)
at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
at EntityFrameworkCore.Jet.Data.JetCommand.ExecuteNonQuery()
at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQuery(RelationalCommandParameterObject parameterObject)
at Microsoft.EntityFrameworkCore.Migrations.MigrationCommand.ExecuteNonQuery(IRelationalConnection connection, IReadOnlyDictionary`2 parameterValues)
at Microsoft.EntityFrameworkCore.Migrations.Internal.MigrationCommandExecutor.ExecuteNonQuery(IEnumerable`1 migrationCommands, IRelationalConnection connection)
at Microsoft.EntityFrameworkCore.Migrations.Internal.Migrator.Migrate(String targetMigration)
at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.Migrate(DatabaseFacade databaseFacade)
at X.Infrastructure.Setup.Migrate(IFactory`1 pDatabaseContextFactory, String pDatabasePath)
at X.COM.XCOMWrapper.Setup(ISettingsProvider pSettingsProvider)
However if we remove this sql statement from the migraition code and execute it as follows, before the call of context.Database.Migrate():
var dbConnection = context.Database.GetDbConnection();
dbConnection.Open();
using (var transaction = dbConnection.BeginTransaction())
{
var updateForeignKeyReferences= dbConnection.CreateCommand();
updateForeignKeyReferences.CommandText = "UPDATE A SET ForeignKeyColumn = NULL WHERE ForeignKeyColumn & \"\" = \"\"";
updateForeignKeyReferences.ExecuteNonQuery();
transaction.Commit();
}
dbConnection.Close();
It works just fine.
Is my approach of using the sql code in the up-method completly wrong? What are possible reasons for this? And most important, how can I fix this? The second approach is my current work-around for this problem but I fear that this means in the long run I can not use the migrations mechanism and have to go for a custom solution (or another framework). I would prefer to just stick with ef core.
Important:
This application works with an legacy application and we have to insert the application history via sql code on the inital startup. For this we create an transaction and simple create the history table and insert the initially created table. This works just fine, and the transactions as well as the commands should all be closed. The table A is never touched by this functions.
Using migrationBuilder.Sql("UPDATE `A` SET `ForeignKeyColumn` = NULL WHERE `ForeignKeyColumn` = ''") is the correct procedure.
It should execute fine.
Unfortunately, there seems to be an issue, where Jet still holds a lock on the table used in the UPDATE command, when the CREATE INDEX statement is executed (that has been generated for your new navigation property and is part of the Up() migration method).
This is only an issue, if both statements are executed inside the same transaction (which is the case for migrations by default). Otherwise, no lock is held and the CREATE INDEX statement succeeds.
The simplest way to fix this issue, is to set the migrationBuilder.Sql() parameter suppressTransaction to true.
This will execute the statement outside of the rest of the transaction, and not lock the table:
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(
"UPDATE `A` SET `ForeignKeyColumn` = NULL WHERE `ForeignKeyColumn` = ''",
suppressTransaction: true);
migrationBuilder.CreateIndex(/* ... */);
migrationBuilder.AddForeignKey(/* ... */);
}
The other way, which is able to execute the UPDATE statement inside a transaction, is to execute the command in its own dedicated migration:
Add an empty migration. Add your migrationBuilder.Sql() call to the empty Up() method of this migration.
Add the actual migration (containing the CreateIndex() and AddForeignKey() operations).
Apply both migrations to your database.
Related
We are trying to implement a multi tenant architecture in our Web API based application. We are using RLS in SQL Server, and Subscription_Id is, what is given to each subscriber. We have set the default value for Subscription_Id in SQL Server, so while I am calling db.SaveChanges(), I just want to ignore the Subscription_Id going to the SQL Server from the API.
I tried setting the value of Subscription_Id in the SaveChanges() override method but got stuck here.
public override int SaveChanges()
{
var objectType = selectedEntity.CurrentValues.ToObject();
Guid value = new Guid("54E720FC-616B-44C6-8485-5F2185FD7B4C");
PropertyInfo propertyInfo =
objectType.GetType().GetProperty("Subscription_Id");
ChangeTracker.Entries().FirstOrDefault()
.CurrentValues.ToObject().GetType()
.GetProperty("Subscription_Id")
.SetValue(objectType, Convert.ChangeType(value, propertyInfo.PropertyType), null);
return base.SaveChanges();
}
My advice is that you shouldn't modify your SaveChanges() code for this.
A recommended way of using RLS is making the TenantId columns transparent to your EF model and your code, so you don't need to define Tenant ID or navigation properties in your entities. This way you don't need to change your SaveChanges() code, or to explicitly manage and set Subscription_Id values anywhere in your code other that when opening the DB connection.
What you need to do is manually setting a default value constraint in the Subscription_Id columns in your database, with a default value based on the current session Subscription_Id parameter. The value will be set when inserting the records, and implicitly used to filter any subsequent queries and commands at database level.
In case of a new column:
ALTER TABLE SomeEntityTable ADD Subscription_Id nvarchar(128)
DEFAULT CAST(SESSION_CONTEXT(N'UserId') AS nvarchar(128))
In case of an existing column:
ALTER TABLE SomeEntityTable
ADD DEFAULT CAST(SESSION_CONTEXT(N'UserId') AS nvarchar(128)
FOR Subscription_Id
If the column had a previous different DEFAULT value it would be good to also delete its associated obsolete DEFAULT constraint. More info about updating default values in existing columns can be found here.
These columns should not be included in your model. You should not have properties for them in your entity classes. If you are using Database First you should make sure you exclude/ignore these columns when updating your model from your database.
How to do this if you are using EF Code First: you can manually include AlterColumn (or CreateColumn) instructions in a code migration after you generate it with Add-Migration. Do it for every entity table:
public override void Up()
{
AlterColumn("dbo.SomeEntityTable", "Subscription_Id",
c => c.String(
nullable: false,
maxLength: 128,
defaultValueSql: "CAST(SESSION_CONTEXT(N'UserId') AS nvarchar(128))"));
}
(It would be good to add also a Down() method removing the column.)
Warning: Be careful when running this migration if you already have existing records in the tables with an empty Subscription_Id column value (or if you are adding a new Subscription_Id column to a table that already have records). The empty column will be filled with the value of the Subscription_Id in the connection that is executing the migration, which probably will be wrong (you probably don't want all the existing records to be associated to that specific subscription). In that case you may want to include explicit UPDATE instructions with the right Subscription_Id values in your Up() method, with the Sql() method. Something like this:
Sql("UPDATE SomeEntitiesTable SET Subscription_Id= '19bc9b0d-28dd-4510-bd5e-d6b6d445f511' WHERE Id IN (1, 2, 5)");
With Code First you should also remove the Subscription_Id properties from your model classes. If you can't, at least add explicit Ignore() instructions in your configuration code for the Subscription_Id columns, you don't want them in your EF mappings.
Note: I'm assuming here that you created a RLS policy in your DB that uses UserId parameter in SESSION_CONTEXT, and that your application code is setting that value when opening the DB connection, via a DbConnectionInterceptor or something similar.
This page contains more info.
In my postgres database, the keys for my tables are serial data types that the postgres database autoincrements. Using C#, Npgsql, Entity Framework 6, WPF, linq and EF dbsets (any extension methods?), is there a way to implement a repository method such as:
public virtual void Add(TEntity entity, string addrow)
{NpgsqlCommand cmd = new NpgsqlCommand(addrow, DbConnect);
cmd.ExecuteNonQuery();}
The viewmodel would pass in the string argument something like:
_addRow = string.Format("insert into mytable ('descrip') values ( " + DescripProperty + ") returning 'mytable_id' into mytableID");
Is this the right way to insert a postgres row / record that has an autoincrement key? Or do I need to utilize a PL/pgSQL - SQL Procedural Language function to first lock the table and do the insert from that server side? Thank you in advance.
EDIT ---------------------------
For clarification, I pass the EF DbContext into the genericrepository constructor. This code is in genericrepository class:
private readonly DbSet<TEntity> _aquery;
public DbSet<TEntity> AQuery;
// CTOR - inject dbcontext/entities
public GenericRepository(MyContextClass context)
{
if (context == null)
throw new ArgumentNullException("context");
_theDbContext = context;
_aquery = _theDbContext.Set<TEntity>();
AQuery = _aquery;
}
So I can do AQuery.AsQueryable, etc.
Your SQL query is correct in the sense that the serial key will get autogenerated by PostgreSQL and then return the new ID - there's no need for pl/pgsql or any explicit locking - PostgreSQL's serial mechanism is atomic and will work. However, several comments on your code:
You're passing Entity Framework entirely and doing things with raw SQL. The whole point of an ORM is to generate this kind of statement for you so you don't have to, and EF6 is perfectly capable of doing so.
Concatenating a parameter into your SQL query is a bad idea and opens the door to SQL injection. You should use a parameter instead.
You don't seem to be actually using the returned ID, so there's little use for the returning clause (although you may have omitted the relevant code)
What is the correct approach while updating a record in CFE ?
We are using webapi controllers, based on this link, the client-side is developped thanks to AngularJS.
I am having troubles updating a record that already exists and my code fails with an CodeFluentDuplicateException.
I read here that EntityState shouldn't be changed manually. When I want to update a server version with the client changes, shall I consider:
Taking the server version and then applying changes made by client ?
Ask the client version to Save() ?
Any other approach ?
Thanks for your answer,
CodeFluentDuplicateException means that you are inserting a record that already exists in the database. This occurs when the stored procedure executes an INSERT statement instead of an UPDATE.
CodeFluent Entities don't use the EntityState to choose whether the entity must be created or updated in the database. Depending on your model, it uses the RowVersion property (insert if null; update otherwise). If there are no way to choose, the stored procedure executes an UPDATE and when no rows are updated it inserts the row. For instance:
CREATE PROCEDURE [dbo].[Role_Save]
(
#Role_Id [uniqueidentifier],
#Role_Name [nvarchar] (256),
)
AS
SET NOCOUNT ON
IF(#_rowVersion IS NOT NULL)
BEGIN
UPDATE [Role] SET
[Role].[Role_Name] = #Role_Name
WHERE (([Role].[Role_Id] = #Role_Id) AND ([Role].[_rowVersion] = #_rowVersion))
END
ELSE
BEGIN
INSERT INTO [Role] (
[Role].[Role_Id],
[Role].[Role_Name])
VALUES (
#Role_Id,
#Role_Name)
END
So in your case I would check the code of the generated stored procedure to understand why it tries to insert the record instead of updating it.
In fact you can change the EntityState manually if you need to, but there are only a few reasons to do it.
In our project we have necessity of adding some predefined data to DB. I think the best way and concept is using for that EF Migrations (not Seed method).
But we have a big troubles with adding related data to DB:
For Example:
Suppose we have 2 tables:
Users:
Id (PK auto increment)
Name
RoleId
Roles:
Id (PK auto increment)
Name
Let's suppose that we need to add User(Name = 'John', RoleId = (Id of role that name is 'Admin')).
How can we do it? It would be great if we find a solution that allows us to execute pure SQL SELECT script which not uses Entities of Code First because they can be modified or removed.
For DELETE, INSERT, UPDATE can be used Sql(...) method but what about SELECT?
You cannot have a context into the migration.
Logically first are ran the migrations to Update the DB Schema, then you can have a context to work with the data via it. If your DB does not match the model, or even the table is still not there, you cannot use it in EF.
I had to look into the EF code (and also because was curious). Practically the Sql() method in the DbMigration class in several levels below just adds the SQL string into a list of queries that should be executed into the transaction and moves on. It does not executes it when it is called. So in short EF just fills in a list of codes lines that should be executed in the end at once. And it seems correct if you try to walk in all paths of what you can do with the C# code in the migration code.
The question is quite good actually, unfortunately still I didn't found any better solution rather than using pure ADO.
Another option is to generate more custom SQL queries, and use T-SQL more widely.
For your case as you want to insert the user and set the groupId looking by the name, it can be used with inner select:
INSERT INTO Users (Name, GroupId)
VALUES ('John', RoleId = (SELECT Id FROM Roles WHERE Name = 'Admin')).
For my issue, I had to a bit do more sophisticated execution - the following does the same as the AddOrUpdate method of the DbSet, using the IF statement:
IF EXISTS (SELECT * FROM Table1 WHERE Column1='SomeValue')
UPDATE Table1 SET (...) WHERE Column1='SomeValue'
ELSE
INSERT INTO Table1 VALUES (...)
I found it here: http://blogs.msdn.com/b/miah/archive/2008/02/17/sql-if-exists-update-else-insert.aspx
I'm using good old LINQ for this:
public override void Up()
{
using (var dc = new DbContext("your connection string or name"))
{
var ids = dc.Database.SqlQuery<int>("SELECT id FROM sometable WHERE somefield={0}", 42).ToArray();
...
}
}
Using LINQ is better, even for usual migrations, because, there is a bug in DbMigration.Sql method, it ignores arguments: How to pass parameters to DbMigration.Sql() Method
Using Spring Data JPA I have the next flow inside the same transaction (REQUIRES_NEW) :
Remove a set of user's predictions with this Spring Data JPA repository method.
#Query(value = "DELETE FROM TRespuestaUsuarioPrediccion resp WHERE resp.idEvento.id = :eventId AND resp.idUsuario.id = :userId")
#Modifying
void deleteUserPredictions(#Param("userId") int userId, #Param("eventId") int eventId);
Insert the new user's predictions and save the master object (event).
eventRepository.save(event);
When this service finishes, the commit is made by AOP but only works in first attemp...not in the next ones...
How can I manage this situation without iterating over event's predictions entries and updating each one inside?
UPDATE
I tried with that and it doesn't work (the adapter inserts the objects I remove before):
#Transactional(propagation=Propagation.REQUIRES_NEW, rollbackFor=PlayTheGuruException.class)
private void updateUserPredictions(final TUsuario user, final TEvento event, final SubmitParticipationRequestDTO eventParticipationRequestDTO)
{
eventRepository.deleteUserPredictions(user.getId(), event.getId());
EventAdapter.predictionParticipationDto2Model(user, event, eventParticipationRequestDTO);
eventRepository.save(event);
}
Hibernate changed order of the commands. It works in below order :
Execute all SQL and second-level cache updates, in a special order so that foreign-key constraints cannot be violated:
1. Inserts, in the order they were performed
2. Updates
3. Deletion of collection elements
4. Insertion of collection elements
5. Deletes, in the order they were performed
And that is exactly the case. When flushing, Hibernate executes all inserts before delete statements.
The possible option are :
1. To call entityManager.flush() explicitly just after the delete.
OR
2. Wherever possible update existing rows and from rest create ToBeDeleted List. This will ensure that existing records are updated with new values and completely new records are saved.
PostgreSQL (and maybe other databases as well) have the possibility to defer the constraint until the commit. Meaning that it accepts duplicates in the transaction, but enforces the unique constraint when committing.
ALTER TABLE <table name> ADD CONSTRAINT <constraint name> UNIQUE(<column1>, <column2>, ...) DEFERRABLE INITIALLY DEFERRED;