EF Code First Model with Properties that Hold Encrypted Data - entity-framework

I've got some encrypted code in a table in the database which I am maintaining. This is circa 2012, so no "Always On" encryption. 3 columns contain encrypted data.
If I reverse engineer an EF domain, the Model which is created for that table contains properties for those columns which have a type byte[]. This is to be expected, as the columns are varbinary. So, it looks like this:
class Person
{
public byte[] FirstName { get; set; } // FirstName
}
Is there an elegant way to do some kind of EF mapping/configuration such that the FirstName class has a type of string and that it decrypts automagically by the framework? I realize I can just instantiate a Person object using sql, but it would be nice to offload this processing to the framework.
I've seen one of two solutions around where people are basically using a sql query for every property. They decorate the property with an Encrypt attribute and iterate the properties of every property. But with a sql query for every property for every object in a list - that does not exactly scale.
Has anyone "solved" this issue before?
Note: to retrieve the data, you first need to send a sql statement akin to:
OPEN SYMMETRIC KEY SomeKey DECRYPTION BY CERTIFICATE SomeCertificate
Thanks

In this answer I'm going to set out the things you need to do to deal with encrypted columns in EF. So, the columns in question will have a type of VARBINARY(MAX). Lets say you table looks something like this:
CREATE TABLE dbo.Person
(
SomeId int NOT NULL,
CreatedByUserId uniqueidentifier NULL,
CreatedUtcDate datetimeoffset(7) NULL,
Rowversion timestamp NULL,
FirstName varbinary(MAX) NULL,
LastName varbinary(MAX) NULL
)
Step 1 - Create a View which returns the decrypted columns. The view should basically be identical to your table, but for the columns which hold encrypted data, it should return the decrypted data. It would looks something like this:
CREATE VIEW [dbo].[v_Person]
AS
SELECT [SomeId]
,[CreatedByUserId]
,[CreatedUtcDate]
,[RowVersion]
,CONVERT(NVARCHAR(50),DECRYPTBYKEY([FirstName])) [FirstName]
,CONVERT(NVARCHAR(50),DECRYPTBYKEY([LastName])) [LastName]
FROM [dbo].[Person]
Step 2 - Create your domain model Person class with string as the relevant property type, not byte[] (note the select statement in the View above where we have cast the decrypted columns to NVARCHAR).
public class Person
{
public int SomeId { get; set; }
public string FirstName { get; set; } // string, not binary
public string LastName { get; set; } // string, not binary
public Guid CreatedByUserId { get; set; }
public DateTime CreatedUtcDate { get; set; }
public int SomeForeignKeyId { get; set; }
}
Step 3 - We need to set up a mapping for that Domain class. (The solution I am setting out here is for EF6. I am aware that EF Core does not support separate mapping files yet, so this would need to be done in the OnModelCreating event of you DbContext). Create a mapping class for you domain object which looks like this:
public class PersonMap : EntityTypeConfiguration<Person>
{
public PersonConfiguration(string schema)
{
ToTable("v_Person", schema); // note we map to the View
HasKey(x => x.SomeId);
// ... other properties elided for brevity
Property(x => x.FirstName)
.HasColumnName(#"FirstName")
.HasColumnType("nvarchar")
.IsOptional()
.HasMaxLength(50);
Property(x => x.LastName)
.HasColumnName(#"LastName")
.HasColumnType("nvarchar")
.IsOptional()
.HasMaxLength(50);
// Foreign keys
HasRequired(a => a.LogbookEntry)
.WithOptional(b => b.Person)
.WillCascadeOnDelete(false);
MapToStoredProcedures(p =>
p.Insert(i => i.HasName("Insert_Person"))
.Update(u => u.HasName("Update_Person"))
.Delete(d => d.HasName("Delete_Person")));
}
}
Note how we mapped to the view, v_Person, and not the raw table.
Also note the call to MapToStoredProcedures, which I explain next.
Step 4 - The last step is to create some stored procedures for your Insert, Update and Deletes. When you invoke SaveChanges, these will be invoked by EF and the relevant stored proc will be invoked depending on which EntityState the entity has. I won't set out all 3, but an example of the Update stored proc might look something like:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[Update_Person]
#SomeId INT,
#CreatedByUserId UNIQUEIDENTIFIER,
#CreatedUtcDate DATETIME,
#RowVersion_Original timestamp,
#FirstName NVARCHAR(50),
#LastName NVARCHAR(50) = NULL
AS
BEGIN
SET NOCOUNT ON;
DECLARE #CertKey NVARCHAR(7) = 'CertKey';
UPDATE PersonDetail
SET
FirstName = ENCRYPTBYKEY(KEY_GUID(#CertKey), #FirstName),
LastName = ENCRYPTBYKEY(KEY_GUID(#CertKey), #LastName)
WHERE SomeId = #SomeId
SELECT SomeId, RowVersion
FROM PersonDetail
WHERE SomeId = #SomeId
END
Feel free to comment if you have done it a better way.
Cheers

Related

EF Core 2.0/2.1 - How to efficiently handle large, infrequently accessed columns?

I have a table as follows:
CREATE TABLE MyTable
(
ID INT NOT NULL PRIMARY KEY,
NAME VARCHAR(50) NOT NULL,
LARGEBLOB VARBINARY(MAX) NULL
)
With an Entity defined as:
public class Entity
{
public int Id {get;set;}
public string Name {get;set;}
public virtual byte[] LargeBlob {get;set;}
}
99% of my use cases involve displaying ID and NAME only.
1% of the time I need LARGEBLOB.
Is there any way I can mark LargeBlob as Lazily Loaded so to avoid
huge wasted data transfers? Alternatively, are there other ways of
achieving the same outcome?
I tried splitting into 2 tables with a 1->[0|1] relationship as follows:
CREATE TABLE MyTable
(
ID INT NOT NULL PRIMARY KEY,
NAME VARCHAR(50) NOT NULL,
LARGEBLOBID INT NULL
)
CREATE TABLE MySubTable
(
ID INT PRIMARY KEY,
LARGEBLOB VARBINARY(MAX) NOT NULL
)
with entities
public class Entity
{
public int Id { get; set; }
public string Name { get; set; }
public virtual LargeBlob LargeBlob { get; set; }
}
public class LargeBlob
{
public int Id { get; set; }
public virtual byte[] Blob { get; set; }
}
That did work in so far as lazy loading was concerned, but I tried all manner of inverse relationship / foreign key tags, HasOne, OwnsOne, OnDelete(Cascade) in all kinds of combinations, but I couldn't achieve what I wanted to achieve. Just to recap, that would be:
Blob is loaded only when the LargeBlob property is actually derefenced.
If entity.LargeBlob property gets set to a new LargeBlob, the (now "orphaned" ) old LargeBlob gets deleted from the database.
If the entity gets deleted, the related large blob gets deleted.
Quick Update re: versions &c
Note: I'm using VS 2017 15.6.2, .net core 2.0, with EF core 2.1 (to get at least the possibility of some lazy loading). Nuget packages:
<PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.6" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Proxies" Version="2.1.0-preview1-final" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="2.1.0-preview1-final" PrivateAssets="All" />
I tried splitting into 2 tables with a 1->[0|1] relationship as follows
But by putting the FK in the Entity you actually did the opposite - [0|1]->1 relationship.
To get the desired relationship, the FK must be at LargeBlog. It could be a separate property (column), but the most appropriate is to use the Id property as both PK and FK (the so called shared PK association). You can do it with the following fluent configuration:
modelBuilder.Entity<Entity>()
.HasOne(e => e.LargeBlob)
.WithOne()
.HasForeignKey<LargeBlob>(e => e.Id);
Once you do that, since the whole purpose of doing it was to get separate controllable (eager, explicit or lazy when available) load behavior, it can be seen that the separate table is not really needed - the "entity" containing the blob data can be embedded inside the same table using the table splitting which is achieved by simply adding the following to the above configuration:
modelBuilder.Entity<Entity>().ToTable("MyTable");
modelBuilder.Entity<LargeBlob>().ToTable("MyTable");
Note that while the most logical choice seems to be owned type, unfortunately currently owned types are always loaded (similar to EF6 complex types), so they cannot be used to achieve controllable load behavior.
You should only select the columns you need to save bandwidth:
var entity = await dbContext.Entities
.Where(...)
.Select(e => new
{
Id = e.Id,
Name = e.Name,
LargeBlob = null,
})
.FirstOrDefaultAsync();
and whenever you really need the LargeBlob column, load it manually
entity.LargeBlob = await dbContext.Entities
.Where(e => e.Id == entity.Id)
.Select(e => e.LargeBlob)
.SingleOrDefaultAsync();
You can delete an entity without loading the whole entity, just the Id (and the concurrency token, if present on the entity) suffices
var entity = new Entity { Id = removeEntityId };
dbContext.Entities.Remove(entity);
await dbContext.SaveChangesAsync();

EF Core set Id to Int.MinValue and try to insert in database

I am using EF Core and I have a problem when I save a new entity.
Here is my model class
[Column("Id")]
public int ID { get; set; }
[Required]
[Column("Pratica", TypeName = "varchar(10)")]
public string PRATICA { get; set; }
[Column("Anno")]
public int ANNO { get; set; }
[Required]
[Column("Variante", TypeName = "varchar(2)")]
public string VARIANTE { get; set; }
Here I create and initialize a new PRAT object:
var prat = new PRAT();
prat.PRATICA = "Prova";
prat.ANNO = 2000;
prat.VARIANTE = "0";
context.PRAT.Add(prat);
context.SaveChangesAsync();
Just after the context.PRAT.Add(prat) line if I check prat.ID member I get something like -2147482647
After context.SaveChangesAsync I get the error "Cannot insert explicit value for identity column in table 'Prat' when IDENTITY_INSERT is set to OFF"
This is the generated SQL statement:
INSERT INTO [Prat] ([Id], [Anno], [Pratica], [Variante]) VALUES (#p0, #p1, #p2, #p3);
As you can see the Id Field is added to the list of fields, but this field is Identity!
If, before context.SaveChangesAsync() I set
prat.ID = 0
the generated SQL Statement is
INSERT INTO [Prat] ([Anno], [Pratica], [Variante]) VALUES (#p0, #p1, #p2);
And all works fine.
Thank you.
I think you need to configure your model with the DatabaseGenerated attribute, or configure it with fluent api
...
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
[Column("Id")]
public int ID { get; set; }
...
The primary key property is of type int, by convention EF Core assumes that the database will use the SQL
IDENTITY command to create a unique key when a new row is added. So you must define your database column as identity column.
For anyone still dealing with this, the other answers are insufficient. Primary keys for ints, shorts, guids etc in EF core are automatically generated.
The DatabaseGeneratedOption.Identity is for columns that are not primary keys.
The real problem is that somewhere in your code (potentially your database seeder if you have one) is pushing entities with manually entered primary keys.
For example:
_context.Jobs.Add(
new Job()
{
JobId = 1,
Name = "Truck Driver",
},
);
_context.SaveChanges();
Doing so tells ef core that you will be supplying primary keys for that entity and it will not know how to generate them. I am unsure why this is because you would think ef core could just grab the max value primary key and add 1 but I think the PK value generation code under the hood is the same for all primary key datatypes (including guid where max value isn't a thing).
Anyways, remove the code where you are manually inserting primary keys and the Add functionality should work as expected.

Add Column Name Convention to EF6 FluentAPI

This question was asked here 4 years ago: EF Mapping to prefix all column names within a table I'm hoping there's better handling these days.
I'm using EF6 Fluent API, what I'll call Code First Without Migrations. I have POCOs for my models, and the majority of my database column names are defined as [SingularTableName]Field (e.g., CustomerAddress db column maps to Address field in Customers POCO)
Table:
CREATE TABLE dbo.Customers (
-- ID, timestamps, etc.
CustomerName NVARCHAR(50),
CustomerAddress NVARCHAR(50)
-- etc.
);
Model:
public class Customer
{
// id, timestamp, etc
public string Name {get;set;}
public string Address {get;set;}
}
ModelBuilder:
modelBuilder<Customer>()
.Property(x => x.Name).HasColumnName("CustomerName");
modelBuilder<Customer>()
.Property(x => x.Address).HasColumnName("CustomerAddress");
Goal:
What I'd really like is to be able to say something like this for the FluentAPI:
modelBuilder<Customer>().ColumnPrefix("Customer");
// handle only unconventional field names here
// instead of having to map out column names for every column
With model-based code-first conventions this has become very simple. Just create a class that implements IStoreModelConvention ...
class PrefixConvention : IStoreModelConvention<EdmProperty>
{
public void Apply(EdmProperty property, DbModel model)
{
property.Name = property.DeclaringType.Name + property.Name;
}
}
... and add it to the conventions in OnModelCreating:
modelBuilder.Conventions.Add(new PrefixConvention());

Hydrate non required string properties from null to empty string

If I have a model class
public class Foo
{
public string Property1 { get; set; } = string.Empty;
... other properties omitted for brevity
}
It gets save to the database as null since it is not a required property. But now when I retrieve the entity that property is null.
I have a lot of not required string properties on this entity and I don't want to have to do a lot of null checking on those properties, I just want them to be rehydrated as empty strings if they are null in the database. I don't mind them being stored in the db as null but when the entity is retrieved from the db I would like to have it mapped back to empty string if it is null in the db.
I'm thinking this must be a fairly common scenario and was wondering if I'm missing some simple way to do it with a configuration setting or something. I am fairly new to using EF.
In my OnModelCreating for this property I have:
.HasMaxLength(50)
.HasDefaultValue(string.Empty)
but it still gets stored as null and rehydrated as null. Again I don't mind it being stored as null but I want it hydrated as empty string if it is null in the db.
I tried modifying the model class like this:
private string property1 = string.empty;
public string Property1
{
get { return property1; }
set { property1 = value ?? string.Empty; }
}
thinking that EF must have to use the setter but this still resulted in a null property. So far the only way I've been able to solve it by making the property like this:
private string property1 = string.empty;
public string Property1
{
get { return property1 ?? string.empty; }
set { property1 = value ?? string.Empty; }
}
I'd really rather not have to make all my properties like that.
Can anyone suggest a cleaner way to do this or correct me if I'm doing something wrong or unusual or thinking about it wrong. Am I missing some easier way to achieve this?
I don't want to make the property required since empty string would not satisfy that case either.
The question's original Entity Framework version was EF7, the first ef-core version that was renamed to EF-core 1 later. The described behavior considerably differs from the current EF-core version (2.1.1).
Let me mention the two key points:
1
It gets save to the database as null since it is not a required property. But now when I retrieve the entity that property is null.
That's not what happens currently.
Take a simple class:
public class Item
{
public int ID { get; set; }
public string Name { get; set; }
public string Code { get; set; } = string.Empty;
}
When adding an Item of which only Name is set, the following SQL is executed:
exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [Items] ([Code], [Name])
VALUES (#p0, #p1);
SELECT [ID]
FROM [Items]
WHERE ##ROWCOUNT = 1 AND [ID] = scope_identity();
',N'#p0 nvarchar(4000),#p1 nvarchar(50)',#p0=N'',#p1=N'Item1'
The empty string is part of the insert statement. Also, when retrieving the item from the database, its Code is an empty string.
2
In my OnModelCreating for this property I have:
.HasMaxLength(50)
.HasDefaultValue(string.Empty)
but it still gets stored as null and rehydrated as null.
That, too, is different now. Take the same class but now with:
public string Code { get; set; } // removed "= string.Empty;"
...and mapping for Code:
modelBuilder.Entity<Item>().Property(p => p.Code).HasMaxLength(50).HasDefaultValue(string.Empty);
...then this is the resulting table:
CREATE TABLE [Items] (
[ID] int NOT NULL IDENTITY,
[Name] nvarchar(50) NOT NULL,
[Code] nvarchar(50) NULL DEFAULT N'', -- Default constraint
CONSTRAINT [PK_Items] PRIMARY KEY ([ID])
);
As you see, the mapping instruction is translated into a database default.
When adding an Item of which only Name is set, the following SQL is executed:
exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [Items] ([Name])
VALUES (#p0);
SELECT [ID], [Code]
FROM [Items]
WHERE ##ROWCOUNT = 1 AND [ID] = scope_identity();
',N'#p0 nvarchar(50)',#p0=N'Item1'
So EF inserts the item and reads back the generated default value in order to keep the tracked entity in sync with the database. Likewise, an item read from the database later has an empty string in Code.
These findings confirm that EF7 was a very immature version of EF-core (although I didn't confirm that it really displayed the described behavior). There have been more, and more profound, breaking changes since it. I hope we will soon forget about these early EF-core versions. Since version 2.0, EF-core is finally developing into a production-ready ORM.

EF 4.1 Code First. Table-per-type inheritance with different primary key name from its base class' primary key name

Given this:
create table Location(
LocationId int identity(1,1) not null primary key,
Address nvarchar(max) not null,
City nvarchar(max) null,
State nvarchar(max) not null,
ZipCode nvarchar(max) not null
);
create table Park(
ParkId int not null primary key references Location(LocationId),
Name nvarchar(max) not null
);
I tried this mapping:
modelBuilder.Entity<Location>();
modelBuilder.Entity<Park>().ToTable("Park");
modelBuilder.Entity<Park>().Property(x => x.LocationId).HasColumnName("ParkId");
Unfortunately that didn't work.
using (var db = new Ef())
{
var park = new Park { Name = "11th Street Park", Address = "801 11th Street", City = "Aledo", State = "TX", ZipCode = "76106" };
db.Set<Location>().Add(park);
db.SaveChanges();
}
It has this error:
The property 'LocationId' is not a declared property on type 'Park'.
Verify that the property has not been explicitly excluded from the
model by using the Ignore method or NotMappedAttribute data
annotation. Make sure that it is a valid primitive property.
How should I map Park entity so its LocationId property fall to ParkId column?
I have this mapping by the way:
public class Location
{
public virtual int LocationId { get; set; }
public virtual string Address { get; set; }
public virtual string City { get; set; }
public virtual string State { get; set; }
public virtual string ZipCode { get; set; }
}
public class Park : Location
{
public virtual string Name { get; set; }
}
If it could help, this is possible in EF 4.0 (via designer), just followed the steps in Chapter 2-11 of Entity Framework 4.0 Recipes, Problem Solution Approach. Now I'm trying it on code first via EF 4.1
[EDIT]
If I change the ParkId to LocationId, things are ok. However, with designer approach, it is possible to map the LocationId to ParkId of table Park; I want to achieve the same thing with code first
create table Park(
LocationId int not null primary key references Location(LocationId),
Name nvarchar(max) not null
);
As I know (and I tried it multiple times) code first doesn't support this => your derived type should use same column names for primary key.
This problem can be described very simply: Current fluent mapping implementation doesn't allow overriding mapping rules from parent entity => parent entity defines names of primary key columns in all derived entities.
IMO the most probable reason is that it was really designed as code first where you don't have existing database and you do not have to bother with database naming - it was up to EF to define names as it needed. Once DbContext API was released people started to use it with existing database massively. But here comes a problem: Initial use cases didn't count with this so some scenarios which are pretty easily done in EDMX are not possible. This is one of them.
Here is a workaround for this issue:
Create a view for the derived table and map your entity class that view. Rename the key column in your view so that it matches the key column in the base table.
eg:
base table User (UserID, FirstName, LastName)
derived table Manager (ManagerID, DepartmentID)
Entity Framework fails to update Manager as the key column is different!
solution:
create view UserManager
as
select
ManagerID as UserID,
DepartmentID
from Manager
Then map the Manager class to the UserManager view, instead of to the Manager table.