I have employee details of more than 4000 employees. While retrieving those rows, I am faced with performance issues due to the looping. So what can I do to improve performance?
This is the looping I mentioned:
List<EmployeesEntityObject> lstEmployee = new List<EmployeesEntityObject>();
foreach (var item in lst)
{
EmployeesEntityObject obj = new EmployeesEntityObject();
obj.EmployeeID = item.EmployeeID;
obj.EmployeeName = item.EmployeeName;
lstEmployee.Add(obj);
}
You could try and see what happens when you completely "linqify" your code:
var lstEmployee = lst.Select(emp => new EmployeesEntityObject
{
EmployeeID = item.EmployeeID,
EmployeeName = item.EmployeeName
}).ToList();
But as marc_s says, there is no apparent cause for any performance issues in your code. Unless, as said, the constructor and/or the setters of the properties conceal time-consuming code. Both of which, by the way, would not be recommendable.
Related
Im finding it really hard to understand maps, I have read lots of examples but they always seem not to relate to what I need to do - I've just about understood how to use oldmap.trigger in a very simple use case.
So anyways I have a trigger on a object Data_ProtectionIA , its parent is Data Agreement, and then its parent is Data request.
I need to update Data agreement and Data request when DPIA gets updated, I have the below so far which I know is a very long winded way to do it, but now I also have to add further updates for when DPIA gets updated to something else and for me to do this like I have done would mean doubling this code again when it is already unnecesarily long.
If someone could please simplify this code into maps then I think I would be able to understand maps properly, I just cant find examples of what I need to do. Many thanks in advance
(So at the moment I am getting the grandchild record and then creating a list of parent records to loop through and then getting a list of all their parent records and then looping through that. - even if you could just show me from child to parent (not even, child parent to grandparent that would be very helpful)
public static void setDRStatus(List<Data_Protection_IA__c> dpia, Map<ID,Data_Protection_IA__c> oldMap ){
List<ID> childDAID = new List<ID>();
for(Data_Protection_IA__c pDA: dpia){
if(pDA.DPIA_Status__c == 'Fully Approved' && oldMap.get(pDA.Id).DPIA_Status__c != 'Fully Approved'){
childDAID.add(pDA.DataProcessingAgreement__c);
}
}
List<Data_Agreement__c> childDA = new List<Data_Agreement__c>();
ChildDA = [Select id, Data_Sharing_Status__c from Data_Agreement__c where id in:childDAID];
List<Data_Agreement__c> listToUpdate = new List <Data_Agreement__c>();
List<ID> dAId = new List <ID>();
system.debug('ChildDA'+ childDA);
for(Data_Agreement__c cda : ChildDA){
cda.Data_Sharing_Status__c = 'DPIA Fully Approved';
listToUpdate.add(cda);
dAId.add(cda.id);
}
update listToUpdate;
List<Data_Request__c> dRList = [SELECT id, Data_Sharing_Status__c,Existing_Data_Agreement__c
FROM Data_Request__c where Existing_Data_Agreement__c in:dAId];
List<Data_Request__c> listToUpdatedr = new List <Data_Request__c>();
system.debug('DRList'+ dRList);
for(Data_Request__c dr :dRList){
dr.Data_Sharing_Status__c = 'DPIA Approved';
listToUpdatedr.add(dr);
}
update listToUpdatedr;
}
----------------------------------------
Here is an example I had previously tried
So I started following an example and got to here
Set<Id> daIds = new Set<Id> {};
for (Data_Protection_IA__c p : dpia)
{
daIds.add(p.DataProcessingAgreement__c );
}
Map<ID, Data_Agreement__c> m = new Map<ID, Data_Agreement__c>(
[select id, from Data_Agreement__c where id in : daIds]);
list<Data_Agreement__c> dAupdates = new List <Data_Agreement__c>();
list<Contact> contactupdates = new List <Contact>();
for (Data_Protection_IA__c p1 :dpia)
{
m.get(p1.Data_Agreement__c) = 'DPIA Fully Approved');
}
but at the last line I think this map is tying to get the field Data_agreement from dpia object, but there is no data agreement field on dpia object. as dpia is the child. there is only a field for dpia on the data agreement object
I am working on .net core entity framework. I have two list of class type. One for update and other for new entry, adding new records all worked fine but which is achieved by context.[Model].Add but update which is done by context.[Model].Update throw exception update i know no record been updated as it is running on local.
$exception {Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: Database operation expected to affect 1 row(s) but actually affected 0 row(s). Data may have been modified or deleted since entities were loaded.
Code
List<AnswerDataModel> surveyResponseListToCreate = new
List<AnswerDataModel>();
List<AnswerDataModel> surveyResponseListToUpdate = new
List<AnswerDataModel>();
if (surveyResponseListToUpdate.Count > 0)
{
foreach (var answerObject in surveyResponseListToUpdate)
{
Context.Answers.Update(answerObject);
if (answerObject.AnswerOptions.Count > 0)
{
foreach (var optItem in answerObject.AnswerOptions)
{
AnswerOptionDataModel answOpt = new AnswerOptionDataModel();
answOpt = optItem;
Context.AnswerOptions.Update(answOpt);
}
}
}
}
var recordsAffected = Context.SaveChanges();
if (!UsingExternalTransaction)
{
FinalizeTransaction(recordsAffected);
}
I can't resist a quote:
"I do not think [your code] means what you think it means."
Assuming that surveyResponseListToUpdate was a list of entities previously loaded and modified:
if (answerObject.AnswerOptions.Count > 0) // Unnecessary...
{
foreach (var optItem in answerObject.AnswerOptions)
{
AnswerOptionDataModel answOpt = new AnswerOptionDataModel(); // does nothing.
answOpt = optItem; // references existing answer option..
Context.AnswerOptions.Update(answOpt);
}
}
The whole block boils down to:
foreach (var optItem in answerObject.AnswerOptions)
Context.AnswerOptions.Update(optItem);
The error you are likely running into is because Update will recurse through navigation properties automatically, so when the parent (Answer) is updated, it's AnswerOptions will be updated as well. So when you go through the extra steps to try and save answer options, they've already been updated when the answer was saved. Provided the Answer was loaded by the same context that you are saving it to, you should be in the clear with:
foreach (var answerObject in surveyResponseListToUpdate)
Context.Answers.Update(answerObject);
var recordsAffected = Context.SaveChanges();
This should update the answer and it's associated answer objects. Even if options were added or removed, the change tracking should do it's job and ensure all of the associated data records are updated.
The extra if checks and such aren't necessary and just add to nesting depth making code harder to read.
However, I suspect that your real code is doing something different to the example given that my tests where I tried to reproduce your error, the code worked fine even updating the child references after updating the parent. If the above still raises issues, please update your example with the code you are running.
I am trying to save hundreds of thousands of records using Entity framework. After saving few hundreds of thousands of records I get following error:
:System.OutOfMemoryException
My code
foreach (BibContent objbibcontents in lstBibContent)
{
db.BibContents.AddObject(objbibcontents);
c = c + 1;
if (c == 1000)
{
db.SaveChanges();
c = 0;
}
}
I noticed after saving 1000 records my db is not overriding another 1000 records. it is adding them into my dbcontext.
I am creating a new instance after 1000 records but my db still has the previous object's data. See my code
foreach (var objbibcontents in lstBibContent)
{
vibrantEntities db1 = new vibrantEntities(szConStr);
lstBibCon.Add(objbibcontents);
// db.BibContents.AddObject(objbibcontents);
c = c + 1;
if (c == 1000)
{
foreach (BibContent bibobject in lstBibCon)
{
db1.BibContents.AddObject(bibobject);
}
lstBibCon.Clear();
db1.SaveChanges();
c = 0;
flag = 1;
}
}
How many objects are you going to save and how big is single object? DbContext holds references to all objects you added with AddObject call. Calling SaveChanges does not purge its internal data structures so if you call your code for 1M objects you will have 1M object in memory and they will be fully alive because their root object for GC will be the context instance which is still in scope of your running code.
If you want to avoid the memory issue you should use a new context instance for every 1000 records (or even every record). The only difference between running SaveChanges for 1000 records and for single record is the automatically involved transaction.
I search on Web and Finally i found good solution for my problem.
Fastest Way of Inserting in Entity Framework
Consider the below program
private static bool CheckFactorPresent(List<FactorReturn> factorReturnCol)
{
bool IsPresent = true;
StringBuilder sb = new StringBuilder();
//Get the exposure names from Exposure list.
//Since this will remain same , so it has been done outside the loop
List<string> lstExposureName = (from item in Exposures
select item.ExposureName).ToList<string>();
foreach (FactorReturn fr in factorReturnCol)
{
//Build the factor names from the ReturnCollection dictionary
List<string> lstFactorNames = fr.ReturnCollection.Keys.ToList<string>();
//Check if all the Factor Names are present in ExposureName list
List<string> result = lstFactorNames.Except(lstExposureName).ToList();
if (result.Count() > 0)
{
result.ForEach(i =>
{
IsPresent = false;
sb.AppendLine("Factor" + i + "is not present for week no: " + fr.WeekNo.ToString());
});
}
}
return IsPresent;
}
Basically I am checking if all the FactorNames[lstFactorNames] are present in
ExposureNames[lstExposureName] list by using lstFactorNames.Except(lstExposureName).
And then by using the Count() function(if count() > 0), I am writing the error
messages to the String Builder(sb)
I am sure that someone can definitely write a better implementation than the one presented.
And I am looking forward for the same to learn something new from that program.
I am using c#3.0 and dotnet framework 3.5
Thanks
Save for some naming convention issues, I'd say that looks fine (for what I can figure out without seeing the rest of the code, or the purpose in the effort. The naming conventions though, need some work. A sporadic mix of ntnHungarian, PascalCase, camelCase, and abbrv is a little disorienting. Try just naming your local variables camelCase exclusively and things will look a lot better. Best of luck to you - things are looking good so far!
- EDIT -
Also, you can clean up the iteration at the end by just running a simple foreach:
...
foreach (var except in result)
{
isPresent = false;
builder.AppendFormat("Factor{0} is not present for week no: {1}\r\n", except, fr.WeekNo);
}
...
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Improve this question
What is the best way to check if an object exists in the database from a performance point of view? I'm using Entity Framework 1.0 (ASP.NET 3.5 SP1).
If you don't want to execute SQL directly, the best way is to use Any(). This is because Any() will return as soon as it finds a match. Another option is Count(), but this might need to check every row before returning.
Here's an example of how to use it:
if (context.MyEntity.Any(o => o.Id == idToMatch))
{
// Match!
}
And in vb.net
If context.MyEntity.Any(function(o) o.Id = idToMatch) Then
' Match!
End If
From a performance point of view, I guess that a direct SQL query using the EXISTS command would be appropriate. See here for how to execute SQL directly in Entity Framework: http://blogs.microsoft.co.il/blogs/gilf/archive/2009/11/25/execute-t-sql-statements-in-entity-framework-4.aspx
I had to manage a scenario where the percentage of duplicates being provided in the new data records was very high, and so many thousands of database calls were being made to check for duplicates (so the CPU sent a lot of time at 100%). In the end I decided to keep the last 100,000 records cached in memory. This way I could check for duplicates against the cached records which was extremely fast when compared to a LINQ query against the SQL database, and then write any genuinely new records to the database (as well as add them to the data cache, which I also sorted and trimmed to keep its length manageable).
Note that the raw data was a CSV file that contained many individual records that had to be parsed. The records in each consecutive file (which came at a rate of about 1 every 5 minutes) overlapped considerably, hence the high percentage of duplicates.
In short, if you have timestamped raw data coming in, pretty much in order, then using a memory cache might help with the record duplication check.
I know this is a very old thread but just incase someone like myself needs this solution but in VB.NET here's what I used base on the answers above.
Private Function ValidateUniquePayroll(PropertyToCheck As String) As Boolean
// Return true if Username is Unique
Dim rtnValue = False
Dim context = New CPMModel.CPMEntities
If (context.Employees.Any()) Then ' Check if there are "any" records in the Employee table
Dim employee = From c In context.Employees Select c.PayrollNumber ' Select just the PayrollNumber column to work with
For Each item As Object In employee ' Loop through each employee in the Employees entity
If (item = PropertyToCheck) Then ' Check if PayrollNumber in current row matches PropertyToCheck
// Found a match, throw exception and return False
rtnValue = False
Exit For
Else
// No matches, return True (Unique)
rtnValue = True
End If
Next
Else
// The is currently no employees in the person entity so return True (Unqiue)
rtnValue = True
End If
Return rtnValue
End Function
I had some trouble with this - my EntityKey consists of three properties (PK with 3 columns) and I didn't want to check each of the columns because that would be ugly.
I thought about a solution that works all time with all entities.
Another reason for this is I don't like to catch UpdateExceptions every time.
A little bit of Reflection is needed to get the values of the key properties.
The code is implemented as an extension to simplify the usage as:
context.EntityExists<MyEntityType>(item);
Have a look:
public static bool EntityExists<T>(this ObjectContext context, T entity)
where T : EntityObject
{
object value;
var entityKeyValues = new List<KeyValuePair<string, object>>();
var objectSet = context.CreateObjectSet<T>().EntitySet;
foreach (var member in objectSet.ElementType.KeyMembers)
{
var info = entity.GetType().GetProperty(member.Name);
var tempValue = info.GetValue(entity, null);
var pair = new KeyValuePair<string, object>(member.Name, tempValue);
entityKeyValues.Add(pair);
}
var key = new EntityKey(objectSet.EntityContainer.Name + "." + objectSet.Name, entityKeyValues);
if (context.TryGetObjectByKey(key, out value))
{
return value != null;
}
return false;
}
I just check if object is null , it works 100% for me
try
{
var ID = Convert.ToInt32(Request.Params["ID"]);
var Cert = (from cert in db.TblCompCertUploads where cert.CertID == ID select cert).FirstOrDefault();
if (Cert != null)
{
db.TblCompCertUploads.DeleteObject(Cert);
db.SaveChanges();
ViewBag.Msg = "Deleted Successfully";
}
else
{
ViewBag.Msg = "Not Found !!";
}
}
catch
{
ViewBag.Msg = "Something Went wrong";
}
Why not do it?
var result= ctx.table.Where(x => x.UserName == "Value").FirstOrDefault();
if(result?.field == value)
{
// Match!
}
Best way to do it
Regardless of what your object is and for what table in the database the only thing you need to have is the primary key in the object.
C# Code
var dbValue = EntityObject.Entry(obj).GetDatabaseValues();
if (dbValue == null)
{
Don't exist
}
VB.NET Code
Dim dbValue = EntityObject.Entry(obj).GetDatabaseValues()
If dbValue Is Nothing Then
Don't exist
End If