EF Core - Change Tracking child records - entity-framework-core

I'm creating a change log on my records and ChangeTracking only reads records from the main class.
I've already added slow loading and the child classes are being loaded inside my EntityEntry, but I can't capture them through entry.Properties
My class
private void OnBeforeSaveChanges()
{
ChangeTracker.DetectChanges();
var auditEntries = new List<AuditEntry>();
foreach (var entry in ChangeTracker.Entries())
{
if (entry.Entity is AuditLog || entry.State == EntityState.Detached || entry.State == EntityState.Unchanged)
continue;
var auditEntry = new AuditEntry(entry)
{
TableName = entry.Entity.GetType().Name,
UserId = _aplicationUser.GetId
};
auditEntries.Add(auditEntry);
foreach (var property in entry.Properties)
{
string propertyName = property.Metadata.Name;
if (property.Metadata.IsPrimaryKey())
{
auditEntry.KeyValues[propertyName] = property.CurrentValue;
continue;
}
switch (entry.State)
{
case EntityState.Added:
auditEntry.AuditType = AuditType.Create;
auditEntry.NewValues[propertyName] = property.CurrentValue;
break;
case EntityState.Deleted:
auditEntry.AuditType = AuditType.Delete;
auditEntry.OldValues[propertyName] = property.OriginalValue;
break;
case EntityState.Modified:
if (property.IsModified)
{
auditEntry.ChangedColumns.Add(propertyName);
auditEntry.AuditType = AuditType.Update;
auditEntry.OldValues[propertyName] = property.OriginalValue;
auditEntry.NewValues[propertyName] = property.CurrentValue;
}
break;
}
}
}
foreach (var auditEntry in auditEntries)
{
var auditDto = auditEntry.ToAudit();
AuditLogs.Add(new AuditLog(auditDto.Id, auditDto.UserId, auditDto.Type, auditDto.TableName, auditDto.DateTime,
auditDto.OldValues, auditDto.NewValues, auditDto.AffectedColumns, auditDto.PrimaryKey));
}
}

The change tracker reports on a per-entity basis. If you have a Parent entity and a Child collection beneath it and make a change to the parent and one or more children, the parent only shows up in the change tracker because of the changes to the parent's properties. It does not list the properties of all related / child entities. Any modified children would appear in the ChangeTracker as Child entities.
For example:
var parent = context.Parents.Include(x => x.Children).Single(x => x.Id == parentId);
var oldestChild = parent.Children.OrderByDescending(x => x.Age).FirstOrDefault();
parent.SomeValue = newValue;
if (oldestChild != null)
oldestChild.OtherValue = someOtherValue;
context.SaveChanges();
In this case the change tracker would include Parent if, and only if the newValue differed from the existing SomeValue. The change tracker would also include the Child denoted as the oldestChild if such a child actually exists, and the someOtherValue differed from that child's OtherValue. If the Child's value was the only one that actually changed then the Change tracker would not include that child's Parent, only the Child reference. This means you cannot really capture an easy "complete" before and after snapshot of an entire object graph. (Parent and all of it's children before and after, or including the parent when one of it's children happens to change) You can do it, but it will involve piecing the deltas together based on what's in the change tracker.
When it comes to writing audit records I would suggest using a bounded DbContext solely for this purpose that only manages the audit log entities and keeps the entity structure as simple as absolutely possible. This way main app context can intercept and inspect changes without worrying about audit changes, and the audit DbContext avoids this overhead, just performing the raw writes.

Related

Entity Framework is too slow during mapping data up to 100k

I have min 100 000 data into a Job_Details table and I'm using Entity Framework to map the data.
This is the code:
public GetJobsResponse GetImportJobs()
{
GetJobsResponse getJobResponse = new GetJobsResponse();
List<JobBO> lstJobs = new List<JobBO>();
using (NSEXIM_V2Entities dbContext = new NSEXIM_V2Entities())
{
var lstJob = dbContext.Job_Details.ToList();
foreach (var dbJob in lstJob.Where(ie => ie.IMP_EXP == "I" && ie.Job_No != null))
{
JobBO job = MapBEJobforSearchObj(dbJob);
lstJobs.Add(job);
}
}
getJobResponse.Jobs = lstJobs;
return getJobResponse;
}
I found to this line is taking about 2-3 min to execute
var lstJob = dbContext.Job_Details.ToList();
How can i solve this issue?
To outline the performance issues with your example: (see inline comments)
public GetJobsResponse GetImportJobs()
{
GetJobsResponse getJobResponse = new GetJobsResponse();
List<JobBO> lstJobs = new List<JobBO>();
using (NSEXIM_V2Entities dbContext = new NSEXIM_V2Entities())
{
// Loads *ALL* entities into memory. This effectively takes all fields for all rows across from the database to your app server. (Even though you don't want it all)
var lstJob = dbContext.Job_Details.ToList();
// Filters from the data in memory.
foreach (var dbJob in lstJob.Where(ie => ie.IMP_EXP == "I" && ie.Job_No != null))
{
// Maps the entity to a DTO and adds it to the return collection.
JobBO job = MapBEJobforSearchObj(dbJob);
lstJobs.Add(job);
}
}
// Returns the DTOs.
getJobResponse.Jobs = lstJobs;
return getJobResponse;
}
First: pass your WHERE clause to EF to pass to the DB server rather than loading all entities into memory..
public GetJobsResponse GetImportJobs()
{
GetJobsResponse getJobResponse = new GetJobsResponse();
using (NSEXIM_V2Entities dbContext = new NSEXIM_V2Entities())
{
// Will pass the where expression to be DB server to be executed. Note: No .ToList() yet to leave this as IQueryable.
var jobs = dbContext.Job_Details..Where(ie => ie.IMP_EXP == "I" && ie.Job_No != null));
Next, use SELECT to load your DTOs. Typically these won't contain as much data as the main entity, and so long as you're working with IQueryable you can load related data as needed. Again this will be sent to the DB Server so you cannot use functions like "MapBEJobForSearchObj" here because the DB server does not know this function. You can SELECT a simple DTO object, or an anonymous type to pass to a dynamic mapper.
var dtos = jobs.Select(ie => new JobBO
{
JobId = ie.JobId,
// ... populate remaining DTO fields here.
}).ToList();
getJobResponse.Jobs = dtos;
return getJobResponse;
}
Moving the .ToList() to the end will materialize the data into your JobBO DTOs/ViewModels, pulling just enough data from the server to populate the desired rows and with the desired fields.
In cases where you may have a large amount of data, you should also consider supporting server-side pagination where you pass a page # and page size, then utilize a .Skip() + .Take() to load a single page of entries at a time.

EF datacontext and entity caching

For performance reasons, I cache lots of objects with .NET system.runtime.caching. I'm caching entities coming from a EF datacontext. I'm disposing the datacontext but since the entities coming from it are in the cache, does that mean GC won't be able to collect it?
Example code:
public static Menu GetMenu(string menuName, string languageCode)
{
ObjectCache cache = MemoryCache.Default;
var menu = (Menu)cache[menuName + "cache"];
if (menu == null)
{
using (PageDb db = new PageDb())
{
menu = db.Menus.Where(m => m.Name == menuName && m.LanguageCode == languageCode).FirstOrDefault();
}
CacheItemPolicy policy = new CacheItemPolicy();
policy.AbsoluteExpiration = DateTimeOffset.Now.AddMonths(1);
cache.Set(menuName + "cache", menu, policy);
}
return menu;
}
As you can see I'm storing a menu object in the cache, but that came from the PageDb datacontext. The menu object is supposed to stay in memory for 1 month, but how about the PageDb datacontext?

Entity Framework 4.1 - Child Object collection is duplicating after each save

I'm having a problem with the Entity Framework (EF4.1 - CodeFirst)
It's basically duplicating child objects ...
The object model is a number of Tasks within a Job -> the Task has a TaskType and that TaskType has a number of Statuses ...
At first it was duplicating TaskTypes on a save
However, after I added this it stopped duplicating the TaskTypes ...
context.Entry(task.TaskType).State = EntityState.Unchanged;
So I tried this for the statuses ...
//tried this but it errors
foreach (var status in task.TaskType.Statuses)
{
context.Entry(status).State = EntityState.Unchanged;
}
////
But it Errors with the following ...
"Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. Refresh ObjectStateManager entries."
Heres the code in full ... (with some of the get code trimmed down for readability)
using(var context = new JobContext())
{
var job = //Get with Includes;
job.Tasks.Add(task);
context.Jobs.Attach(job);
context.Entry(task.TaskType).State = EntityState.Unchanged;
//tried this but it errors
foreach (var status in task.TaskType.Statuses)
{
context.Entry(status).State = EntityState.Unchanged;
}
////
context.SaveChanges();
}
Can anyone assist please?
using(var context = new JobContext())
{
var job = //Get with Includes;
context.Jobs.Attach(job);
// can be omitted if "Get with Includes" happens in the same context
context.TaskTypes.Attach(task.TaskType);
// attaches TaskType and all Statuses -> State is Unchanged
job.Tasks.Add(task);
// A new task will be created after SaveChanges. If you don't want that
// replace Attach(task.TaskType) above by Attach(task)
context.SaveChanges();
}
You don't need to load with Include by the way for this. You can load without Include (and then create an empty collection job.Tasks = new List<Task>()).

Is there an Update Object holder on Entity Framework?

I'm currently inserting/updating fields like this (if there's a better way, please say so - we're always learning)
public void UpdateChallengeAnswers(List<ChallengeAnswerInfo> model, Decimal field_id, Decimal loggedUserId)
{
JK_ChallengeAnswers o;
foreach (ChallengeAnswerInfo a in model)
{
o = this.FindChallengeAnswerById(a.ChallengeAnswerId);
if (o == null) o = new JK_ChallengeAnswers();
o.answer = FilterString(a.Answer);
o.correct = a.Correct;
o.link_text = "";
o.link_url = "";
o.position = FilterInt(a.Position);
o.updated_user = loggedUserId;
o.updated_date = DateTime.UtcNow;
if (o.challenge_id == 0)
{
// New record
o.challenge_id = field_id; // FK
o.created_user = loggedUserId;
o.created_date = DateTime.UtcNow;
db.JK_ChallengeAnswers.AddObject(o);
}
else
{
// Update record
this.Save();
}
}
this.Save(); // Commit changes
}
As you can see there is 2 times this.Save() (witch invokes db.SaveChanges();)
when Adding we place the new object into a Place Holder with the AddObject method, in other words, the new object is not committed right away and we can place as many objects we want.
But when it's an update, I need to Save first before moving on to the next object, is there a method that I can use in order to, let's say:
if (o.challenge_id == 0)
{
// New record
o.challenge_id = field_id;
o.created_user = loggedUserId;
o.created_date = DateTime.UtcNow;
db.JK_ChallengeAnswers.AddObject(o);
}
else
{
// Update record
db.JK_ChallengeAnswers.RetainObject(o);
}
}
this.Save(); // Only save once when all objects are ready to commit
}
So if there are 5 updates, I don't need to save into the database 5 times, but only once at the end.
Thank you.
Well if you have an object which is attached to the graph, if you modify values of this object, then the entity is marked as Modified.
If you simply do .AddObject, then the entity is marked as Added.
Nothing has happened yet - only staging of the graph.
Then, when you execute SaveChanges(), EF will translate the entries in the OSM to relevant store queries.
Your code looks a bit strange. Have you debugged through (and ran a SQL trace) to see what is actually getting executed? Because i can't see why you need that first .Save, because inline with my above points, since your modifying the entities in the first few lines of the method, an UPDATE statement will most likely always get executed, regardless of the ID.
I suggest you refactor your code to handle new/modified in seperate method. (ideally via a Repository)
Taken from Employee Info Starter Kit, you can consider the code snippet as below:
public void UpdateEmployee(Employee updatedEmployee)
{
//attaching and making ready for parsistance
if (updatedEmployee.EntityState == EntityState.Detached)
_DatabaseContext.Employees.Attach(updatedEmployee);
_DatabaseContext.ObjectStateManager.ChangeObjectState(updatedEmployee, System.Data.EntityState.Modified);
_DatabaseContext.SaveChanges();
}

.Net Entity Framework SaveChanges is adding without add method

I'm new to the entity framework and I'm really confused about how savechanges works. There's probably a lot of code in my example which could be improved, but here's the problem I'm having.
The user enters a bunch of picks. I make sure the user hasn't already entered those picks.
Then I add the picks to the database.
var db = new myModel()
var predictionArray = ticker.Substring(1).Split(','); // Get rid of the initial comma.
var user = Membership.GetUser();
var userId = Convert.ToInt32(user.ProviderUserKey);
// Get the member with all his predictions for today.
var memberQuery = (from member in db.Members
where member.user_id == userId
select new
{
member,
predictions = from p in member.Predictions
where p.start_date == null
select p
}).First();
// Load all the company ids.
foreach (var prediction in memberQuery.predictions)
{
prediction.CompanyReference.Load();
}
var picks = from prediction in predictionArray
let data = prediction.Split(':')
let companyTicker = data[0]
where !(from i in memberQuery.predictions
select i.Company.ticker).Contains(companyTicker)
select new Prediction
{
Member = memberQuery.member,
Company = db.Companies.Where(c => c.ticker == companyTicker).First(),
is_up = data[1] == "up", // This turns up and down into true and false.
};
// Save the records to the database.
// HERE'S THE PART I DON'T UNDERSTAND.
// This saves the records, even though I don't have db.AddToPredictions(pick)
foreach (var pick in picks)
{
db.SaveChanges();
}
// This does not save records when the db.SaveChanges outside of a loop of picks.
db.SaveChanges();
foreach (var pick in picks)
{
}
// This saves records, but it will insert all the picks exactly once no matter how many picks you have.
//The fact you're skipping a pick makes no difference in what gets inserted.
var counter = 1;
foreach (var pick in picks)
{
if (counter == 2)
{
db.SaveChanges();
}
counter++;
}
I've tested and the SaveChanges doesn't even have to be in the loop.
The below code works, too.
foreach (var pick in picks)
{
break;
}
db.SaveChanges()
There's obviously something going on with the context I don't understand. I'm guessing I've somehow loaded my new picks as pending changes, but even if that's true I don't understand I have to loop over them to save changes.
Can someone explain this to me?
Here's updated working code based on Craig's responses:
1) Remove the Type then loop over the results and populate new objects.
var picks = (from prediction in predictionArray
let data = prediction.Split(':')
let companyTicker = data[0]
where !(from i in memberQuery.predictions
select i.Company.ticker).Contains(companyTicker)
select new //NO TYPE HERE
{
Member = memberQuery.member,
Company = db.Companies.Where(c => c.ticker == companyTicker).First(),
is_up = data[1] == "up", // This turns up and down into true and false.
}).ToList();
foreach (var prediction in picks)
{
if (includePrediction)
{
var p = new Prediction{
Member = prediction.Member,
Company = prediction.Company,
is_up = prediction.is_up
};
db.AddToPredictions(p);
}
}
2) Or if I don't want the predictions to be saved, I can detach the predictions.
foreach (var prediction in picks) {
if (excludePrediction)
{
db.Detach(prediction)
}
}
The reason is here:
select new Prediction
{
Member = memberQuery.member,
These lines will (once the IEnumerable is iterated; LINQ is lazy) :
Instantiate a new Prediction
Associate that Prediction with an existing Member, *which is attached to db.
Associating an instance of an entity with an attached entity automatically adds that entity to the context of the associated, attached entity.
So as soon as you start iterating over predictionArray, the code above executes and you have a new entity in your context.