We embark on a journey to migrate an existing EF4 project to EF7.
We stumbled upon a leak that will lead to many undetected bugs (no build errors).
In the EF4 project, we rely on Lazy Loading to get the data of related entities. For example:
var items = db.Items.GetXXXById(id);
var filteredItems = (from item in items
where item.StatusId == 1
select item).ToList();
var childrens = from item in filteredItems
where item.StatusId == 1
from children in item.Childrens
select children;
Debug.WriteLine(childrens.Count()); // This returned 0 on EF7 but the correct value on EF4.
The above code will return a correct value on EF4 but it returns zero on EF7.
To mitigate the above, we will have to add an "Include" to eager load the children. For example:
var items = db.Items.GetXXXById(id);
var filteredItems = (from item in items
where item.StatusId == 1
select item).Include(i => i.Children).ToList();
var childrens = from item in filteredItems
where item.StatusId == 1
from children in item.Childrens
select children;
Debug.WriteLine(childrens.Count()); // Both EF4 and EF7 returns the correct value.
But, without the build error thrown, we will have to check all our queries to figure out what we need to "Include". Is there a way or a tool that helps detect such "leak"?
You can configure Lazy loading with proxies in your DbContext class like this:
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder
.UseLazyLoadingProxies()
.UseSqlServer(myConnectionString);
This setting will work like EF4 that is lazy loading enabled by default.
IMHO, It's not a best practice for many circumstances. You should consider turn off lazy loading by default and taking care of the relationship between entities to avoid N+1 problem.
Related
I have a EF core code first web api application.
There is a Products entity and a UserProductsRating child entity (with a one to many relationship)
I also wanted to have an average review score (and be able to select/sort based on it) so created a view to do this (using the method described in this answer
[https://stackoverflow.com/a/18707413][1])
So the migration for my View looks like:
protected override void Up(MigrationBuilder migrationBuilder)
{
string script =
#"
CREATE VIEW AverageProductRating AS
SELECT u.ProductId, AVG(CAST(u.Rating AS FLOAT)) as AverageRating
FROM dbo.UserRatings u GROUP BY u.ProductId";
migrationBuilder.Sql(script);
}
protected override void Down(MigrationBuilder migrationBuilder)
{
string script = #"DROP VIEW dbo.AverageProductRating";
migrationBuilder.Sql(script);
}
Then there is an AverageRating entity on top.
This all works fine and allows me to create queries like:
var top5Products = _db.Products.Include(x => x.AverageProductRating)
.Where(x => x.AverageProductRating != null)
.OrderByDescending(x => x.AverageProductRating.AverageRating)
.Take(5);
The problem occurs when I get to my Unit/Integration Tests. I am using the InMemoryDatabase with EnsureCreated to set up a testing instance/seed data.
var options = new DbContextOptionsBuilder<ProductsContext>()
.UseInMemoryDatabase(Guid.NewGuid().ToString())
.EnableSensitiveDataLogging()
.Options;
var context = new ProductsContext(options);
context.Database.EnsureCreated();
When I run tests against this the AverageProductRating Entity always has zero rows (I'm not sure if the View gets created at all or not)
I think this may be to do with restrictions in SQL in the inmemory db or the way migrations are run, but I'm not sure.
Any suggestions on how to work round this would be welcome
Thanks
I have a complex and big object graph that I want to insert in database by using a DbContext and SaveChanges method.
This object is a result of parsing a text file with 40k lines (around 3MB of data). Some collections inside this object have thousands of items.
I am able to parse the file correctly and add it to the context so that it can start tracking the object. But when I try to SaveChanges, it says:
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while updating the entries. See the inner exception for details. ---> System.Data.SqlClient.SqlException: String or binary data would be truncated.
I would like to know if there is a smart and efficient way of discovering which object is causing the issue. It seems that a varchar field is too little to store the data. But it's a lot of tables and fields to check manually.
I would like to get a more specific error somehow. I already configured an ILoggerProvider and added the EnableSensitiveDataLogging option in my dbContext to be able to see which sql queries are being generated. I even added MiniProfiler to be able to see the parameter values, because they are not present in the log generated by the dbContext.
Reading somewhere in the web, I found out that in EF6 there is some validation that happens before the sql is passed to the database to be executed. But it seems that in EF Core this is not available anymore. So how can I solve this?
After some research, the only approach I've found to solve this, is implementing some validation by overriding dbContext's SaveChanges method. I've made a merge of these two approaches to build mine:
Implementing Missing Features in Entity Framework Core - Part 3
Validation in EF Core
The result is...
ApplicationDbContext.cs
public override int SaveChanges(bool acceptAllChangesOnSuccess)
{
ValidateEntities();
return base.SaveChanges(acceptAllChangesOnSuccess);
}
public override async Task<int> SaveChangesAsync(bool acceptAllChangesOnSuccess, CancellationToken cancellationToken = new CancellationToken())
{
ValidateEntities();
return await base.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
}
private void ValidateEntities()
{
var serviceProvider = this.GetService<IServiceProvider>();
var items = new Dictionary<object, object>();
var entities = from entry in ChangeTracker.Entries()
where entry.State == EntityState.Added || entry.State == EntityState.Modified
select entry.Entity;
foreach (var entity in entities)
{
var context = new ValidationContext(entity, serviceProvider, items);
var results = new List<ValidationResult>();
if (Validator.TryValidateObject(entity, context, results, true)) continue;
foreach (var result in results)
{
if (result == ValidationResult.Success) continue;
var errorMessage = $"{entity.GetType().Name}: {result.ErrorMessage}";
throw new ValidationException(errorMessage);
}
}
}
Note that it's not necessary to override the other SaveChanges overloads, because they call these two.
The Error tells you that youre writing more characters to a field than it can hold.
This error for example would be thrown when you create a given field as NVARCHAR(4) or CHAR(4) and write 'hello' to it.
So you could simply check the length of the values you read in to find the one which is causing your problem. There is at least on which is too long for a field.
I'm using EF5 and attaching a disconnected graph of POCO entities to my context, something like this:-
using (var context = new MyEntities())
{
context.Configuration.AutoDetectChangesEnabled = false;
context.MyEntities.Attach(myEntity);
// Code to walk the entity graph and set each entity's state
// using ObjectStateManager omitted for clarity ..
context.SaveChanges();
}
The entity "myEntity" is a large graph of entities, with many child collections, which in turn have their own child collections, and so on. The entire graph contains in the order of 10000 entities, but only a small number are usually changed.
The code to set the entity states and the actual SaveChanges() is fairly quick (<200ms). It's the Attach() that's the problem here, and takes 2.5 seconds, so I was wondering if this could be improved. I've seen articles that tell you to set AutoDetectChangesEnabled = false, which I'm doing above, but it makes no difference in my scenario. Why is this?
I am afraid that 2,5 sec for attaching an object graph with 10000 entities is "normal". It's probably the entity snapshot creation that takes place when you attach the graph that takes this time.
If "only a small number are usually changed" - say 100 - you could consider to load the original entities from the database and change their properties instead of attaching the whole graph, for example:
using (var context = new MyEntities())
{
// try with and without this line
// context.Configuration.AutoDetectChangesEnabled = false;
foreach (var child in myEntity.Children)
{
if (child.IsModified)
{
var childInDb = context.Children.Find(child.Id);
context.Entry(childInDb).CurrentValues.SetValues(child);
}
//... etc.
}
//... etc.
context.SaveChanges();
}
Although this will create a lot of single database queries, only "flat" entities without navigation properties will be loaded and attaching (that occurs when calling Find) won't consume much time. To reduce the number of queries you could also try to load entities of the same type as a "batch" using a Contains query:
var modifiedChildIds = myEntity.Children
.Where(c => c.IsModified).Select(c => c.Id);
// one DB query
context.Children.Where(c => modifiedChildIds.Contains(c.Id)).Load();
foreach (var child in myEntity.Children)
{
if (child.IsModified)
{
// no DB query because the children are already loaded
var childInDb = context.Children.Find(child.Id);
context.Entry(childInDb).CurrentValues.SetValues(child);
}
}
It's just a simplified example under the assumption that you only have to change scalar properties of the entities. It can become arbitrarily more complex if modifications of relationships (children have been added to and/or deleted from the collections, etc.) are involved.
I have an entity with self reference (generated by Entity Designer):
public MyEntity: EntityObject
{
// only relevant stuff here
public int Id { get...; set...; }
public MyEntity Parent { get...; set...; }
public EntityCollection<MyEntity> Children { get...; set...; }
...
}
I've written a stored procedure that returns a subtree of nodes (not just immediate children) from the table and returns a list of MyEntity objects. I'm using a stored proc to avoid lazy loading of an arbitrary deep tree. This way I get relevant subtree nodes back from the DB in a single call.
List<MyEntity> nodes = context.GetSubtree(rootId).ToList();
All fine. But when I check nodes[0].Children, its Count equals to 0. But if I debug and check context.MyEntities.Results view, Children enumerations get populated. Checking my result reveals children under my node[0].
How can I programaticaly force my entity context to do in-memory magic and put correct references on Parent and Children properties?
UPDATE 1
I've tried calling
context.Refresh(ClientWins, nodes);
after my GetSubtree() call which does set relations properly, but fetches same nodes again from the DB. It's still just a workaround. But better than getting the whole set with context.MyEntities().ToList().
UPDATE 2
I've reliably solved this by using EF Extensions project. Check my answer below.
You need to assign one end of the relationship. First, divide the collection:
var root = nodes.Where(n => n.Id == rootId).First();
var children = nodes.Where(n => n.Id != rootId);
Now, fix up the relationship.
In your case, you'd do either:
foreach (var c in children)
{
c.Parent = root;
}
...or:
foreach (var c in children)
{
root.Children.Add(c);
}
It doesn't matter which.
Note that this marks the entities as modfied. You'll need to change that if you intend to call SaveChanges on the context and don't want this saved.
The REAL solution
Based on this article (read text under The problem), navigation properties are obviously not populated/updated when one uses stored procedures to return data.
But there's a nice manual solution to this. Use EF Extensions project and write your own entity Materilizer<EntityType> where you can correctly set navigation properties like this:
...
ParentReference = {
EntityKey = new EntityKey(
"EntityContextName.ParentEntitySetname",
new[] {
new EntityKeyMember(
"ParentEntityIdPropertyName",
reader.Field<int>("FKNameFromSP")
)
})
}
...
And that's it. Calling stored procedure will return correct data, and entity object instances will be correctly related to eachother. I advise you check EF Extensions' samples, where you will find lots of nice things.
I have the following code snippet.
// Here I get an entity out of an EntityCollection<TEntity> i.e ContactSet. Entity is obtained and not null.
ProjectContact obj = ((Project)projectDataGrid.SelectedItem).ContactSet
.Where(projectContact => projectContact.ProjectId == item.ProjectId &&
projectContact.ContactId == item.ContactId).First();
// And the next line I just check whether ContactSet contains the queried entity that i.e. obj.
bool found = ((Project)projectDataGrid.SelectedItem).ContactSet.Contains(obj);
but found is always false. How can that be?
edit: Matt thank you for your guidance but let me make it a bit more clear since I haven't given out the full source code.
I have three tables in my database:
Project, Contact and ProjectContact and there's a many-to-many relationship between Project and Contact table through the ProjectContact table, although ProjectContact table has some extra columns other than Project and Contact table keys, and that's why I get an extra entity called ProjectContact if I use ADO.NET entity framework's entity designer generated code.
Now at some point I get a Project instance within my code by using a linq to entities query i.e:
var item = (from project in myObjectContext.Project.Include("ContactSet.Contact")
orderby project.Name select project).FirstOrDefault();
Note that ContactSet is the navigational property of Project to ProjectContact table and Contact is the navigational property of ProjectContact to Contact table.
Moreover the queried Project in question i.e. "item" has already some ProjectContacts in its item.ContactSet entity collection, and ContactSet is a standard EntityCollection implementation generated by entity designer.
On the other hand, ProjectContact overrides Equals() and GetHashCode() etc. but if I use that overriden implementation within an EqualityComparer then Project.ContactSet.Contains returns true so I'm guessing there's no problem with that but now the tricky part comes along. Assume that I have the following snippet:
using(SomeObjectContext myObjectContext = new SomeObjectContext())
{
var projectQueryable = from project in myObjectContext.Project.Include("ContactSet.Contact") orderby project.Name select project;
ObservableCollection<Project> projects = new ObservableCollection<Project>(projectQueryable.ToList());
var contactQueryable = from contact in myObjectContext.Contact select contact;
ObservableCollection<Contact> contacts = new ObservableCollection<Contact>(contactQueryable.ToList());
Project p = projects[0];
Contact c = contacts[0];
//Now if I execute the code below it fails.
ProjectContact projectContact = new ProjectContact();
projectContact.Contact = c;
projectContact.Project = p;
projectContact.ContactId = c.Id;
projectContact.ProjectId = p.Id;
projectContact.Role = ContactRole.Administrator; // This corresponds to the column in ProjectContact table and I do manual conversion within the partial class since EF doesn't support enums yet.
p.ContactSet.Add(projectContact); // This line might be unnecessary but just to be on the safe side.
// So now p.ContactSet does indeed contain the projectContact and projectContact's EntityState is Added as expected. But when I execute the line below without saving changes it fails.
bool result = p.ContactSet.Remove(projectContact); // result == false and projectContact is still in the p.ContactSet EntityCollection.
//Now if I execute the line below
myObjectContext.Delete(projectContact);
//Now projectContact's EntityState becomes Detached but it's still in p.ContactSet.
// Also note that if I test
bool exists = p.ContactSet.Contains(projectContact);
// It also returns false even if I query the item with p.ProjectContact.Where(...) it returns false.
}
Since everything occurs within the same ObjectContext I think I am missing something about EntityCollection.Remove(). But it seems still odd that ContactSet.Contains() returns false for an item obtained via direct Where query over ContactSet. In the end the question becomes:
How do you really Remove an item from an EntityCollection without persisting to the database first. Since a Remove() call after Add() apparently fails.
This looks like it should work, Some ideas:
Does ProjectContact override Object.Equals()? Or perhaps the ContactSet implements ICollection, and there may be a bug in the implementation of ICollection.Contains()?