How to do bulk insert / update (best performance) entity framework - entity-framework

Im trying to do an update / insert of about 100 records:
public static bool SaveTemplates(List<SecurityTemplateItemModel> templates)
{
try
{
using (SecurityDS service = new SecurityDS())
{
foreach (var item in templates)
{
if (item.IsNew)
{
// insert
service.AddToSecurityTemplateExceptions(new SecurityTemplateException()
{
ObjectId = item.ObjectId,
AccessType = item.AccessType,
PlatformType = item.PlatformType,
TemplateUid = item.TemplateUid
});
}
else
{
// update
var exception = service.SecurityTemplateExceptions.Where(e => e.ObjectId == item.ObjectId && e.TemplateUid == item.TemplateUid).FirstOrDefault();
exception.AccessType = item.AccessType;
exception.PlatformType = item.PlatformType;
service.UpdateObject(exception);
}
}
service.SaveChanges();
}
return true;
}
catch
{
return false;
}
}
Im using Data Services, and don't know if the performance issue comes from there. But the function checks if the entity is new then add to the collection if not then get the Entity Framework entity and update some value and then Update the object and finally save the changes.
This process takes about 15 seconds, and it should work in 1 or 2 seconds.
Any clue on how to do this? or if there is a better way?

I just change from using DataServices to just WCF Services exposing POCO Models and it just worked great.

Related

efcore change modified state to update entity is not working with sub data objects

I set up a Generic repository using this code for update
private void AttachIfNot(TEntity entityToActive)
{
if (_dbContext.Entry(entityToActive).State == EntityState.Detached)
{
_dbSet.Attach(entityToActive);
}
}
private void UpdateEntity(TEntity entityToUpdate)
{
AttachIfNot(entityToUpdate);
_dbContext.Entry(entityToUpdate).State = EntityState.Modified;
}
It just attach the entity and set the modified state to save.
But when I use efocre ownsone to map a value object,the update entity function is not working.
I found out that it only works when I set Valueobject to modified too.
_dbContext.Entry(entityToUpdate).State = EntityState.Modified;
_dbContext.Entry(entityToUpdate.Valueobject).State = EntityState.Modified;
But It is hard for me to specify all the value objects in a Generic Repository.
This is code also has problems with one to many or other relations.
The working way is like this:
Classroom classroom = new Classroom
{
Id = 1,
Name = "b",
Students = new List<Student>
{
new Student()
{
Name = "aa",
Id = 2
}
}
};
if (_defaultDbContext.Entry(classroom).State == EntityState.Detached)
{
_defaultDbContext.Classrooms.Attach(classroom);
foreach(var stu in classroom.Students)
{
_defaultDbContext.Students.Attach(stu);
}
}
_defaultDbContext.Entry(classroom).State = EntityState.Modified;
foreach (var stu in classroom.Students)
{
_defaultDbContext.Entry(stu).State = EntityState.Modified;
}
_defaultDbContext.SaveChanges();
I found out one way is get the entity form repo then update it using automapper:
targetEntity = repo.GetById(entityId);
automapper.map(souceEntity,targetEntity);
//or
automapper.map(souceDto,targetEntity);
_dbContext.Save();
The entity comes by query, so the change will be tracked.
But I have to configure the automapper with this entity map when I want to change entity
CreateMap<EntityType, EntityType>();
I think it's not the best solution. Is there a bettere way?
DbContext.Update would be fine to fix this problem.
see:
https://www.learnentityframeworkcore.com/dbcontext/change-tracker

ASP.NET Core Entity Framework SQL Query SELECT

I am one of the many struggling to "upgrade" from ASP.NET to ASP.NET Core.
In the ASP.NET project, I made database calls from my DAL like so:
var result = context.Database.SqlQuery<Object_VM>("EXEC [sp_Object_GetByKey] #Key",
new SqlParameter("#Key", Key))
.FirstOrDefault();
return result;
My viewmodel has additional fields that my object does not, such as aggregates of related tables. It seems unnecessary and counter intuitive to include such fields in a database / table structure. My stored procedure calculates all those things and returns the fields as should be displayed, but not stored.
I see that ASP.NET Core has removed this functionality. I am trying to continue to use stored procedures and load view models (and thus not have the entity in the database). I see options like the following, but as a result I get "2", the number of rows being returned (or another mysterious result?).
using(context)
{
string cmd = "EXEC [sp_Object_getAll]";
var result = context.Database.ExecuteSQLCommand(cmd);
}
But that won't work because context.Database.ExecuteSQLCommand is only for altering the database, not "selecting".
I've also seen the following as a solution, but the code will not compile for me, as "set" is really set<TEntity>, and there isn't a database entity for this viewmodel.
var result = context.Set().FromSql("EXEC [sp_Object_getAll]");
Any assistance much appreciated.
Solution:
(per Tseng's advice)
On the GitHub Entity Framework Issues page, there is a discussion about this problem. One user recommends creating your own class to handle this sort of requests, and another adds an additional method that makes it run smoother. I changed the methods slights to accept slightly different params.
Here is my adaptation (very little difference), for others that are also looking for a solution:
Method in DAL
public JsonResult GetObjectByID(int ID)
{
SqlParameter[] parms = new SqlParameter[] { new SqlParameter("#ID", ID) };
var result = RDFacadeExtensions.GetModelFromQuery<Object_List_VM>(context, "EXEC [sp_Object_GetList] #ID", parms);
return new JsonResult(result.ToList(), setting);
}
Additional Class
public static class RDFacadeExtensions
{
public static RelationalDataReader ExecuteSqlQuery(
this DatabaseFacade databaseFacade,
string sql,
SqlParameter[] parameters)
{
var concurrencyDetector = databaseFacade.GetService<IConcurrencyDetector>();
using (concurrencyDetector.EnterCriticalSection())
{
var rawSqlCommand = databaseFacade
.GetService<IRawSqlCommandBuilder>()
.Build(sql, parameters);
return rawSqlCommand
.RelationalCommand
.ExecuteReader(
databaseFacade.GetService<IRelationalConnection>(),
parameterValues: rawSqlCommand.ParameterValues);
}
}
public static IEnumerable<T> GetModelFromQuery<T>(
DbContext context,
string sql,
SqlParameter[] parameters)
where T : new()
{
DatabaseFacade databaseFacade = new DatabaseFacade(context);
using (DbDataReader dr = databaseFacade.ExecuteSqlQuery(sql, parameters).DbDataReader)
{
List<T> lst = new List<T>();
PropertyInfo[] props = typeof(T).GetProperties();
while (dr.Read())
{
T t = new T();
IEnumerable<string> actualNames = dr.GetColumnSchema().Select(o => o.ColumnName);
for (int i = 0; i < props.Length; ++i)
{
PropertyInfo pi = props[i];
if (!pi.CanWrite) continue;
System.ComponentModel.DataAnnotations.Schema.ColumnAttribute ca = pi.GetCustomAttribute(typeof(System.ComponentModel.DataAnnotations.Schema.ColumnAttribute)) as System.ComponentModel.DataAnnotations.Schema.ColumnAttribute;
string name = ca?.Name ?? pi.Name;
if (pi == null) continue;
if (!actualNames.Contains(name)) { continue; }
object value = dr[name];
Type pt = pi.DeclaringType;
bool nullable = pt.GetTypeInfo().IsGenericType && pt.GetGenericTypeDefinition() == typeof(Nullable<>);
if (value == DBNull.Value) { value = null; }
if (value == null && pt.GetTypeInfo().IsValueType && !nullable)
{ value = Activator.CreateInstance(pt); }
pi.SetValue(t, value);
}//for i
lst.Add(t);
}//while
return lst;
}//using dr
}

Is this really the best way to update detached entities?

I have the following code:
Get contact
public Contact LoadContactFromId(int contactId)
{
using(var ctx = new ContactContext())
{
var contact = ctx.Contacts.AsNoTracking().First(c => c.ContactId == contactId);
return contact;
}
}
Update Contact
public void UpdateExistingContact(Contact updatedContact)
{
using(var ctx = new ContactContext())
{
ctx.Contacts.Attach(updatedContact);
ctx.Entry(updatedContact).State = EntityState.Modified;
foreach (var item in updatedContact.ContactPoints)
{
ctx.Entry(item).State = (item.ContactPointId == 0) ? EntityState.Added : EntityState.Modified;
}
ctx.SaveChanges();
}
}
I am currently working with ASP.NET webforms and because of its stateless model, I can't keep working within the DbContext. Because of this, I end up basically recreating the entire Contact entity from the POST data and then update it.
This seems like a horrible way to do this because I lose conncurrncey resolution since I'm not working with the true original entity and it seems kind of cumbersome to always have to recreate the object.
Is there an easier way to do this? Am I looking at this all wrong?

Persisting changes in EF 4.3 - Change tracking issues

I have a class ReportConfigurationManager which manages the CRUD operations against a UserReport entity. The two operations of interest are "Get" and "SaveUpdate". In both cases I wrap the operation in a using statement so that the DbContext is disposed at the end of the query.
Now eventually these methods will form part of a WCF service, but they may also be called internally within the service. My present difficulties are with getting a set of Unit Tests to work which call the ReportConfigurationManager directly.
I can create a new UserReport and save it (this took me a while to solve as the entity has several nested objects which already exist in the database - I needed to "Attach" each of these in turn to the context before calling Add on the UserReport in order to get it to save correctly.
My issues now are with Updates.
Despite having
context.Configuration.ProxyCreationEnabled = false;
context.Configuration.AutoDetectChangesEnabled = false;
on ALL methods which use the ReportConfigurationManager, when I came to attach a UserReport, it failed with the classic "an object with the same key already exists in the ObjectStateManager" (I thought disabling Change Tracking was meant to handle this?).
So now I have switched to using the following code which I found here
public UserReport SaveUpdateUserReport(UserReport userReport)
{
using (var context = new ReportDataEF())
{
context.Configuration.ProxyCreationEnabled = false;
context.Configuration.AutoDetectChangesEnabled = false;
if (userReport.Id > 0)
{
{
UserReport oldReport = context.UserReports.Where(ur => ur.Id == userReport.Id).FirstOrDefault();
context.Entry(oldReport).CurrentValues.SetValues(userReport);
}
}
else
{
//Need to attach everything to prevent EF trying to create duplicates in the database
context.ReportTopTypes.Attach(userReport.ReportTopType);
context.ReportWindows.Attach(userReport.ReportWindow);
context.ReportSortOptions.Attach(userReport.ReportSortOption);
foreach (var col in userReport.ReportColumnGroups)
{
context.ReportColumnGroups.Attach(col);
}
context.ReportTemplates.Attach(userReport.ReportTemplate);
//just add the new data
context.UserReports.Add(userReport);
}
context.SaveChanges();
}
return userReport;
}
My concern is that my code seems laborious - I need to get a copy of the old object before I can save the updated copy? And I'm not convinced by my Save New logic either.
So is this approach correct, or is there a better way of writing the above?
Further details of other stuff going on:
Because I'll be sending the object graphs over WCF. I've implemented Eager Loading:
public static DbQuery<ReportTemplate> IncludeAll(this DbQuery<ReportTemplate> self)
{
return self
.Include("ReportColumnGroups.ReportColumns.ReportColumnType")
.Include("ReportColumnGroups.ReportColumnType")
.Include("ReportSortOptions.ReportSortColumns.ReportColumn.ReportColumnType")
.Include("ReportSortOptions.ReportSortColumns.ReportSortType");
}
public static DbQuery<UserReport> IncludeAll(this DbQuery<UserReport> self)
{
return self
.Include("ReportTemplate")
.Include("ReportTopType")
.Include("ReportWindow")
.Include("ReportSortOption.ReportSortColumns.ReportColumn.ReportColumnType")
.Include("ReportSortOption.ReportSortColumns.ReportSortType")
.Include("ReportColumnGroups.ReportColumns.ReportColumnType")
.Include("ReportColumnGroups.ReportColumnType");
}
public static DbQuery<ReportSortOption> IncludeAll(this DbQuery<ReportSortOption> self)
{
return self
.Include("ReportSortColumns.ReportColumn.ReportColumnType")
.Include("ReportSortColumns.ReportSortType");
}
public static DbQuery<ReportColumnGroup> IncludeAll(this DbQuery<ReportColumnGroup> self)
{
return self
.Include("ReportColumn.ReportColumnType")
.Include("ReportColumnType");
}
public static DbQuery<ReportColumn> IncludeAll(this DbQuery<ReportColumn> self)
{
return self
.Include("ReportColumnType");
}
public static DbQuery<ReportSortColumn> IncludeAll(this DbQuery<ReportSortColumn> self)
{
return self
.Include("ReportColumn.ReportColumnType")
.Include("ReportSortType");
}
I have a set of static, cached data that I obtain as follows:
using (var context = new ReportDataEF())
{
context.Configuration.ProxyCreationEnabled = false;
context.Configuration.AutoDetectChangesEnabled = false;
reportConfigurationData = new ReportingMetaData()
{
WatchTypes = context.WatchTypes.ToList(),
ReportTemplates = context.ReportTemplates.IncludeAll().ToList(),
ReportTopTypes = context.ReportTopTypes.ToList(),
ReportWindows = context.ReportWindows.ToList(),
ReportSortOptions =
context.ReportSortOptions.IncludeAll().ToList()
};
}
and I retrieve UserReports as follows:
public UserReport GetUserReport(int userReportId)
{
using (var context = new ReportDataEF())
{
context.Configuration.ProxyCreationEnabled = false;
context.Configuration.AutoDetectChangesEnabled = false;
var visibleReports =
context.UserReports.IncludeAll().Where(ur => ur.Id == userReportId).FirstOrDefault();
return visibleReports;
}
}
The test I am concerned with gets an existing UserReport from the DB, Updates its ReportTemplate and ReportColumnGroups properties with objects from the static data class and then attempts to save the updated UserReport.
Using the code from Ladislav's answer, this fails when I try to attach the UserReport, presumably because one of the objects I've attached to it, already exists in the database.
Yes there is another way. First think you should know is that EF doesn't support partially attached object graphs so both Attach and Add have side effects to attach or add all entities in the graph which are not yet tracked by the context. This will simplify your insertion code a lot.
public UserReport SaveUpdateUserReport(UserReport userReport)
{
using (var context = new ReportDataEF())
{
context.Configuration.ProxyCreationEnabled = false;
context.Configuration.AutoDetectChangesEnabled = false;
// Now all entities in the graph are attached in unchanged state
context.ReportTopTypes.Attach(userReport);
if (userReport.Id > 0 &&
context.UserReports.Any(ur => ur.Id == userReport.Id))
{
context.Entry(userReport).State = EntityState.Modified;
}
else
{
context.Entry(userReport).State = EntityState.Added;
}
context.SaveChanges();
}
return userReport;
}
This is equivalent to your original code. You don't load user report again - you just check its existence in DB. This code has a lot of problems - for example if you changed any other related object it will not be persisted to database because currently its state is Unchanged. It can be even more complicated if you need to change relations.

How to log changed values to database on all action?

The controller above has a standard edit ActionResult. I simply find rows in a database by ID and update it. Before db.SaveChanges() there is log.Save() static function that saves all changes in model to separate tables in the database.It simply check old and new values from ChangeTracker.
The problem is, i want use log.Save() after db.SaveChanges(), not before, to be sure that data was really saved.
But after, in the ChangeTracker there aren't any changes so log.Save() doesn't have anything to save.
Controller:
[HttpPost]
public ActionResult edit(int id, MyModel model)
{
var hihi = db.MyModel.First(s => s.ID == model.ID);
hihi.col1 = model.col1;
hihi.col2 = model.col2;
...
log.Save(Log.ChangeType.Edit, db, id);
^ Here i save changes to log.
db.SaveChanges();
return RedirectToAction("Index");
}
Log Class:
public void Save(ChangeType changeType, DBContext parentContext, int id)
{
DBContext db = new DBContext();
foreach (System.Data.Entity.Infrastructure.DbEntityEntry ee in parentContext.ChangeTracker.Entries())
{
foreach (string column in ee.OriginalValues.PropertyNames)
{
string oldValue = ee.OriginalValues[column].ToString();
string newValue = ee.CurrentValues[column].ToString();
if (oldValue != newValue)
{
var model = new LogModel
{
Log_Time = DateTime.Now,
Log_Operator = User.Ope_ID,
Log_Table = ee.Entity.ToString().Replace("xxx.Models.", ""),
Log_Key = id,
Log_Column = column,
Log_Type = (int)changeType,
Log_OldValue = oldValue,
Log_NewValue = newValue
};
var log = db.Log.Add(model);
db.SaveChanges();
}
}
}
}
public enum ChangeType
{
Create = 1,
Delete = 2,
Edit = 3
}
... or maybe someone has another way to save all changes in a database to another table on all controller actions, so after the project release I can see what users do.
PS. I don't what user triggers.
SaveChanges in EF4 is virtual, so you can override it, add custom logging etc.
Why don't you use try{} catch{} within Log Class and change the return parameter from 'void' to 'bool'. This would return true if the db.SaveChanges() succeeds. Then within "ActionResult edit" use bool result = log.Save(Log.ChangeType.Edit, db, id); to retrieve if the log saved the changes, then use a simple if-sentence to validate if you can save all changes to db or not.