Code First Entity Framework Slow When Properties are Missing - frameworks

I'm using Entity Framework 4.3.1 Code First.
I have a pretty simple expression and entity model.
using (var PMCtx = new PMContext("PMEntities"))
{
var results =
PMCtx.Fetch<vwSDHOriginalMW>()
.Where(x => x.DT >= StartDate && x.DT < EndDate)
.ToList();
return results;
}
public class vwSDHOriginalMW : IEntityObject, IPMContext
{
public int Schedule { get; set; }
public DateTime DT { get; set; }
public int HE { get; set; }
public Decimal OrgMW { get; set; }
public Decimal DELIVERMW { get; set; }
public string NERCCode { get; set; }
public string NERCCodeStatus { get; set; }
public int SDHSDHID { get; set; }
}
This was taking 15 seconds every time, not just the first time. The model is mapped to a view in a Sql Server 2008 database. I output the query that EF was sending, and ran it in SSMS and it took a fraction of a second.
Why is this so slow in Entity Framework?
IEntityObject appears to be a marker interface so that the original programmer could be sure these were the only that get put into the generic.
EDIT 1
Fetch ends up going through some layer wrappers to get to the data layer where it does this:
private DbSet<TEntity> FetchSet<TEntity>()
where TEntity : class, IEntityObject
{
Type PassedType = typeof(TEntity);
if (!CheckedTypes.Any(x => x.FullName == PassedType.FullName))
if (!PassedType.GetInterfaces().Any(x => CtxInterfaces.Contains(x)))
throw new ArgumentException("Type passed is not a DbSet type of constructed context.");
else
CheckedTypes.Add(PassedType);
return privateContext.Set<TEntity>();
}
Cleaned up example of the query EF is sending
SELECT [Schedule],
[DT],
[HE],
[OrgMW],
[DELIVERMW],
[NERCCode],
[NERCCodeStatus],
[SDHSDHID],
[ScheduleDeliveryHourHistoryID]
FROM [vwSDHOriginalMW]
WHERE ([DT] >= '2/17/2013') AND ([DT] < '2/21/2013')
EDIT 2
The view in the database actually had one more column than my entity model had properties.
I added the property to the model.
public class vwSDHOriginalMW : IEntityObject, IPMContext
{
public int Schedule { get; set; }
public DateTime DT { get; set; }
public int HE { get; set; }
public Decimal OrgMW { get; set; }
public Decimal DELIVERMW { get; set; }
public string NERCCode { get; set; }
public string NERCCodeStatus { get; set; }
public int SDHSDHID { get; set; }
//missing property
public int ScheduleDeliveryHourHistoryID { get; set; }
}
After adding the property yesterday, it sped up tremendously for a while, ran in 4 seconds instead of 15. But today it's slow again, and nothing has changed.
UPDATE:
I have narrowed it down a little further. There are two methods that I can use that end up using the same FetchSet. The one that I am using returns an IQueryable instead of an IEnumerable. This seems normal, and since I am filtering afterward, most desirable. However the method that returns IQueryable takes 15 seconds while the IEnumerable takes less than a second. (I am calling ToList() on both) FetchAll turns out just to be a wrapper that calls Fetch and returns IEnumerable instead of IQueryable
public IQueryable<TEntity> Fetch<TEntity>() where TEntity : class, Common.IEntityObject
{
return privateContext.Fetch<TEntity>();
}
public IEnumerable<TEntity> FetchAll<TEntity>() where TEntity : class, Common.IEntityObject
{
return privateContext.FetchAll<TEntity>();
}
If I change
IEnumerable<vwSDHOriginalMW> results =
PMCtx.Fetch<vwSDHOriginalMW>()
.Where(x => x.DT >= StartDate && x.DT < EndDate)
.ToList();
to
IEnumerable<vwSDHOriginalMW> results =
PMCtx.Fetch<vwSDHOriginalMW>()
.ToList()
.Where(x => x.DT >= StartDate && x.DT < EndDate);
it is fast. But this isn't acceptable, because it seems like I would want my where clause to be passed to the database. In this case on a dev environment the view is only 180 rows, but it has potential to be millions, so I definitely don't want to return all my results into memory before I filter them.

After much digging and many headaches, I figured out that the view was referencing a view on a different database instance that referenced a table that was missing a non-clustered index. This caused the execution plan to get cached incorrectly. After adding the index on the other database:
USE [OTHERDATABASE]
GO
CREATE NONCLUSTERED INDEX [IX_ScheduleEnergyProfileJoin]
ON [dbo].[WTXS_ScheduleEnergyProfile] ([SEQSDR])
INCLUDE ([SEQSEPI],[StartDate],[EndDate])
GO
Then clearing the execution plan cache on the database with the view I'm using:
USE [MYDATABASE]
DBCC FREEPROCCACHE
DBCC DROPCLEANBUFFERS
The query is running quickly. So it turns out that the SQL that EF said it was using was probably not the sql that was getting sent to the database. Moral of the story is I should have gone through whatever hoops to get profiling permissions on this database instead of relying on the following to output the SQL that would actually be sent.
var sql = ((System.Data.Entity.Infrastructure.DbQuery<vwSDHOriginalMW>)results).ToString();

Related

GroupBy Expression failed to translate

//Model
public class Application
{
[Key]
public int ApplicationId { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime ConfirmedDate { get; set; }
public DateTime IssuedDate { get; set; }
public int? AddedByUserId { get; set; }
public virtual User AddedByUser { get; set; }
public int? UpdatedByUserId { get; set; }
public virtual User UpdatedByuser { get; set; }
public string FirstName { get; set; }
public string MiddleName { get; set; }
public string LastName { get; set; }
public string TRN { get; set; }
public string EmailAddress { get; set; }
public string Address { get; set; }
public int ParishId { get; set; }
public Parish Parish { get; set; }
public int? BranchIssuedId { get; set; }
public BranchLocation BranchIssued { get; set; }
public int? BranchReceivedId { get; set; }
public BranchLocation BranchReceived {get; set; }
}
public async Task<List<Application>> GetApplicationsByNameAsync(string name)
{
if (string.IsNullOrEmpty(name))
return null;
return await _context.Application
.AsNoTracking()
.Include(app => app.BranchIssued)
.Include(app => app.BranchReceived)
.Include(app => app.Parish)
.Where(app => app.LastName.ToLower().Contains(name.ToLower()) || app.FirstName.ToLower()
.Contains(name.ToLower()))
.GroupBy(app => new { app.TRN, app })
.Select(x => x.Key.app)
.ToListAsync()
.ConfigureAwait(false);
}
The above GroupBy expression fails to compile in VS Studio. My objective is to run a query filtering results by name containing a user given string and then it should group the results by similar TRN numbers returning a list of those applications to return to the view. I think I am really close but just cant seem to figure out this last bit of the query. Any guidance is appreciated.
Error being presented
InvalidOperationException: The LINQ expression 'DbSet<Application>
.Where(a => a.LastName.ToLower().Contains(__ToLower_0) || a.FirstName.ToLower().Contains(__ToLower_0))
.GroupBy(
source: a => new {
TRN = a.TRN,
app = a
},
keySelector: a => a)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()
UPDATE
Seems it is definitely due to a change in how .net core 3.x and EF core play together since recent updates. I had to change it to client evaluation by using AsEnumerable() instead of ToListAsync(). The rest of the query given by Steve py works with this method. I was unaware even after reading docs how the groupby really worked in LINQ, so that has helped me a lot. Taking the query to client side eval may have performance issues however.
The GroupBy support in EF core is a joke.
This worked perfectly on the server in EF6
var nonUniqueGroups2 = db.Transactions.GroupBy(e => new { e.AccountId, e.OpeningDate })
.Where(grp => grp.Count() > 1).ToList();
In EF core it causes an exception "Unable to translate the given 'GroupBy' pattern. Call 'AsEnumerable' before 'GroupBy' to evaluate it client-side." The message is misleading, do not call AsEnumerable because this should be handled on the server.
I have found a workaround here. An additional Select will help.
var nonUniqueGroups = db.Transactions.GroupBy(e => new { e.AccountId, e.OpeningDate })
.Select(x => new { x.Key, Count = x.Count() })
.Where(x => x.Count > 1)
.ToList();
The drawback of the workaround is that the result set does not contain the items in the groups.
There is an EF Core issue. Please vote on it so they actually fix this.
Based on this:
I want to group by TRN which is a repeating set of numbers eg.12345, in the Application table there may be many records with that same sequence and I only want the very latest row within each set of TRN sequences.
I believe this should satisfy what you are looking for:
return await _context.Application
.AsNoTracking()
.Include(app => app.BranchIssued)
.Include(app => app.BranchReceived)
.Include(app => app.Parish)
.Where(app => app.LastName.ToLower().Contains(name.ToLower()) || app.FirstName.ToLower()
.Contains(name.ToLower()))
.GroupBy(app => app.TRN)
.Select(x => x.OrderByDescending(y => y.CreatedAt).First())
.ToListAsync()
.ConfigureAwait(false);
The GroupBy expression should represent what you want to group by. In your case, the TRN. From there when we do the select, x represents each "group" which contains the Enumarable set of Applications that fall under each TRN. So we order those by the descending CreatedAt date to select the newest one using First.
Give that a shot. If it's not quite what you're after, consider adding an example set to your question and the desired output vs. what output / error this here produces.
I experience a similar issue where I find it interesting and stupid at the same time. Seems like EF team prohibits doing a WHERE before GROUP BY hence it does not work. I don't understand why you cannot do it but this seems the way it is which is forcing me to implement procedures instead of nicely build code.
LMK if you find a way.
Note: They have group by only when you first group then do where (where on the grouped elements of the complete table => does not make any sense to me)

EF6:How to include subproperty with Select so that single instance is created. Avoid "same primary key" error

I'm trying to fetch (in disconnected way) an entity with its all related entities and then trying to update the entity. But I'm getting the following error:
Attaching an entity of type 'Feature' failed because another entity of the same type already has the same primary key value.
public class Person
{
public int PersonId { get; set; }
public string Personname { get; set }
public ICollection Addresses { get; set; }
}
public class Address
{
public int AddressId { get; set; }
public int PersonId { get; set; }
public string Line1 { get; set; }
public string City { get; set; }
public string State { get; set; }
public Person Person { get; set; }
public ICollection<Feature> Features { get; set; }
}
// Many to Many: Represented in database as AddressFeature (e.g Air Conditioning, Central Heating; User could select multiple features of a single address)
public class Feature
{
public int FeatureId { get; set; }
public string Featurename { get; set; }
public ICollection<Address> Addresses { get; set; } // Many-To-Many with Addresses
}
public Person GetCandidate(int id)
{
using (MyDbContext dbContext = new MyDbContext())
{
var person = dbContext.People.AsNoTracking().Where(x => x.PersonId == id);
person = person.Include(prop => prop.Addresses.Select(x => x.Country)).Include(prop => prop.Addresses.Select(x => x.Features));
return person.FirstOrDefault();
}
}
public void UpdateCandidate(Person newPerson)
{
Person existingPerson = GetPerson(person.Id); // Loading the existing candidate from database with ASNOTRACKING
dbContext.People.Attach(existingPerson); // This line is giving error
.....
.....
.....
}
Error:
Additional information: Attaching an entity of type 'Feature' failed because another entity of the same type already has the same primary key value.
It seems like (I may be wrong) GetCandidate is assigning every Feature within Person.Addresses a new instance. So, how could I modify the GetCandidate to make sure that the same instance (for same values) is bing assisgned to Person.Addresses --> Features.
Kindly suggest.
It seems like (I may be wrong) GetCandidate is assigning every Feature within Person.Addresses a new instance. So, how could I modify the GetCandidate to make sure that the same instance (for same values) is bing assisgned to Person.Addresses --> Features.
Since you are using a short lived DbContext for retrieving the data, all you need is to remove AsNoTracking(), thus allowing EF to use the context cache and consolidate the Feature entities. EF tracking serves different purposes. One is to allow consolidating the entity instances with the same PK which you are interested in this case, and the second is to detect the modifications in case you modify the entities and call SaveChanges(), which apparently you are not interested when using the context simply to retrieve the data. When you disable the tracking for a query, EF cannot use the cache, thus generates separate object instances.
What you really not want is to let EF create proxies which hold reference to the context used to obtain them and will cause issues when trying to attach to another context. I don't see virtual navigation properties in your models, so most likely EF will not create proxies, but in order to be absolutely sure, I would turn ProxyCreationEnabled off:
public Person GetCandidate(int id)
{
using (MyDbContext dbContext = new MyDbContext())
{
dbContext.Configuration.ProxyCreationEnabled = false;
var person = dbContext.People.Where(x => x.PersonId == id);
person = person.Include(prop => prop.Addresses.Select(x => x.Country)).Include(prop => prop.Addresses.Select(x => x.Features));
return person.FirstOrDefault();
}
}

EF 4.1 Issue retrieving original and current values

I am using EF 4.1 and lazy loading. I have below entities:
public abstract class PersonBase
{
[Key(), Required]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
....
[ForeignKey("Quantity")]
public virtual int? QuantityId { get; set; }
public virtual Quantity Quantity { get; set; }
....
}
public class ConcretePerson : PersonBase
{
....
}
public class Quantity
{
[Key(), Required]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
[Required]
public virtual float QuantityA { get; set; }
[Required]
public virtual float QuantityB { get; set; }
[Required]
public virtual float QuantityC { get; set; }
}
IDbSet<Quantity> Quantities;
IDbSet<ConcretePerson> ConcretePersons;
IDbSet<PersonBase> Persons;
so in my code I perform below:
using (DataBaseContext context = new DataBaseContext())
{
IQueryable<ConcretePerson> concretePersonCollection = context.ConcretePersons.Where(<condition>);
foreach (ConcretePerson concretePerson in concretePersonCollection)
{
...
concretePerson.Quantity.QuantityA = new_quantity_A;
concretePerson.Quantity.QuantityB = new_quantity_B;
concretePerson.Quantity.QuantityC = new_quantity_C;
...
}
...
DbEntityEntry<ConcretePerson> entityEntry;
Quantity quantity;
foreach (ConcretePerson concretePerson in concretePersonCollection)
{
entityEntry = context.Entry<ConcretePerson>(concretePerson);
if (entityEntry.State == System.Data.EntityState.Modified)
{
quantity = ((ConcretePerson)entityEntry.CurrentValues.ToObject()).Quantity;
}
else
{
quantity = concretePerson.Quantity;
}
...
}
...
context.SaveChanges();
}
Note that I only perform SaveChanges at the end so database is not updated until this point is reached.
I have problems within the second foreach:
1.- When entityEntry.State is modified it happens that ((ConcretePerson)entityEntry.CurrentValues.ToObject()).Quantity is null but
((ConcretePerson)entityEntry.CurrentValues.ToObject()).QuantityId is correct (contains the correct value)
Why? How to get this different from null with the current values (neither original values nor database values), just current values?
2.- If I check directly the Quantity by performing concretePerson.Quantity it is not null but
concretePerson.Quantity contains the current values (the ones updated in the first foreach),
not the original ones (the values before updating in the first foreach). Should not concretePerson.Quantity
contain the original values (before updating in the first foreach) instead? because I have not
performed any context.savechanges between the two foreach loops.
3.-Context.SaveChanges is not saving the changes done to the database and is not raising any error. <---- This point is solved, I was pointing to a different context, no using the same (now I am using the same).
That is an odd way of trying to get values... try this.
concretePerson.Quantity will be the local copy of the entity so it will have whichever value you assigned to it.
In the first foreach you are actually modifying each of the items in the collection (even if it is not saved to the database yet it is still in memory, otherwise how would EF know what to save to the database?).
In the second you are actually checking the same collection to see if entities have been modified (which they have) and then getting the current value. However the current value for quantity will be the same as .quantity because you have modified the entity. If you check the original value for modified entries you will see that it is different.
Basically CurrentValue is the value of the in memory entity (if you change the property CurrentValue changes). OriginalValue is "usually the entity's property values as they were when last queried from the database"
Nevermind ;p

Using the Entry<TEntity>().CurrentValues.SetValues() is not updating collections

I have not run into this before, because I usually handled collections by them selves instead of modifying them directly on the entity.
public class Schedule: BaseEntity
{
public Guid Id {get;set;}
public virtual int? DayOfTheWeekTypeId { get; set; }
public virtual DayOfTheWeekType DayOfTheWeekType { get; set; }
public virtual ICollection<Instructor> Instructors { get; set; }
public DateTime? StartDateTime { get; set; }
public DateTime? EndDateTime { get; set; }
public string SpecialInstructions { get; set; }
}
Mapping class:
public ScheduleMapping()
{
HasMany(c => c.Instructors).WithMany().Map(m => { m.MapLeftKey("ScheduleId");
m.MapRightKey("InstructorId");
m.ToTable("Schedule_Instructors");
});
HasOptional(s => s.DayOfTheWeekType).WithMany().HasForeignKey(s => s.DayOfTheWeekTypeId).WillCascadeOnDelete(false);
Property(s => s.SpecialInstructions).IsMaxLength();
}
This is my update method:
public virtual void Update(TEntity entity)
{
if (entity == null)
throw new ArgumentNullException("entity");
//this is the original persisted entity
var persistedEntity = _repository.GetById(entity.Id);
if(originalEntity != null)
{
entity.Id = persistedEntity.Id;
UnitOfWork.ApplyCurrentValues<TEntity>(originalEntity,entity);
UnitOfWork.Commit();
}
}
This is the Method that handled the "merge"
public void ApplyCurrentValues<TEntity>(TEntity original, TEntity current) where TEntity : class
{
base.Entry<TEntity>(original).CurrentValues.SetValues(current);
}
If I modify the Instructors collection then try to apply the update, it seems to keep my original values. I have tried loading the the Schedule entity prior to the update and make my changes, but sometimes that causes a PK error (on the Instructors collection) in entity framework. As if it is trying to add an entity with the same key. So, instead I am rebuilding the Schedule entity (including the ID) manually and then updating it. When I do that I do not get any more errors, however, the Instructors collections doesn't change. I am thinking because CurrentValues. SetValues is being applied based on the persisted entity and not my updated version. the Should I handle my updates differently or do I need to manully
SetValues never updates navigation properties. When you execute your code it only knows about changes in simple / complex properties of the entity passed to your Update method. EF even don't know about related entities of the entity passed to your Update method.
You must manually tell EF about each change in your object graph - EF doesn't have any resolution mechanism for object graphs.

Entity Framework Code First Update by Stub Entity

Is it possible in EF Code-First to update without querying the entire row in db by using stub objects,...
e.g.
public class Dinner
{
public int DinnerID { get; set; }
public string Title { get; set; }
public DateTime EventDate { get; set; }
public string Address { get; set; }
public string Country { get; set; }
public string HostedBy { get; set; }
}
Dinner dinner = dinner.DinnerID = 1;
dinner.HostedBy = "NameChanged"
nerdDinners.SaveChanges();
will the code above create an Update Statement which will make the following columns null for the row of DinnerID 1 ?
Title, EventDate, Address, Country
Is there a way or method like "PropertyModified" = true, then the rest make them = false, so that HostedBy is the only one that will be updated?
I think you are looking for ApplyCurrentValues
public void UpdateDinner(Dinner existingDinner)
{
var stub = new Dinner { DinnerId = existingDinner.DinnerId };
ctx.Dinners.Attach(stub);
ctx.ApplyCurrentValues(existingDinner);
ctx.SaveChanges();
}
ApplyCurrentValues copies the scalar values from the existing object to the object in the graph (in the above case - the stub entity).
From the Remarks section on MSDN:
Any values that differ from the original values of the object are marked as modified.
Is that what your after?
To build on Paul's answer, the following will work when you are using EF Model or Database First:
context.ObjectStateManager.GetObjectStateEntry(dinner).SetModifiedProperty("HostedBy");
I think you are looking for the Attach() method.
Attaching and Detaching Objects
Try this maybe, it is specific to EF Code First which seems to do it differently than just EF.
var dinner = context.Dinners.Find(1);
context.Entry(dinner).Property(d => d.HostedBy).IsModified = true;
context.SaveChanges();
From ADO.NET team blog
"Marking a property as modified forces an update to be send to the database for the property when SaveChanges is called even if the current value of the property is the same as its original value."