Stackoverflow Exception using PredicateBuilder with Entity Framework - entity-framework

I have Predicate builder, which is having predicate and inner predicate and building a dynamic filter based on conditions, let's say I am selecting one department, under that department I am getting list of employees, once I get the list of employees, I need to load the respective records for each and every employee who belongs to the selected department.
Implementation is already done long back and it works fine if department is having not too many employees, once it goes beyond 500 or 1000, the predicate builder is causing a stack overflow. Please see my code snippet for this - I am using .net framework 4.5.2.
Getting stackoverflow exception when assigning to inner predicate at this line with in loop, when record is beyond 1000 or 500, it loops based on the employee records.
Expression<Func<EmployeeTable, bool>> predicate = PredicateBuilder.True<EmployeeTable>();
var innerPredicate = PredicateBuilder.False<EmployeeTable>();
case FilterBy.EmployeeName:
if (!isEmpNameFilterExists)
{
foreach (string empName in item.FieldCollection)
{
innerPredicate = innerPredicate.Or(x => x.Name.Equals(empName,
StringComparison.OrdinalIgnoreCase));
}
predicate = predicate.And(innerPredicate.Expand());
}
break;

This might happen due to the (usually sufficient) but small stack for .NET applications (https://stackoverflow.com/a/823729/2298807). Evaluation on predicates is usually done as part of lamda functions and they use the stack. I do not now the predicate library in detail but I assume a use of recursive functions.
Anyhow: I would suggest to use Contains by building a List<string> containing the names:
Expression<Func<EmployeeTable, bool>> predicate =
PredicateBuilder.True<EmployeeTable>();
var innerPredicate = PredicateBuilder.False<EmployeeTable>();
case FilterBy.EmployeeName:
if (!isEmpNameFilterExists)
{
List<string> namesList = new List<string>();
foreach (string empName in item.FieldCollection)
{
namesList.Add(empName);
}
predicate = predicate.And(x => namesList.Contains(x.Name));
}
break;
Note: Please check the syntax as I do not have a VS environment available at the moment.

I have added my own Expression builder engine, i.e. much better way to generate the Predicate.
PredicateBuilder works well with LINQ to Object, With EntityFramework its having the issue, because it generates the Lambda methods with full namespace of models and keep on adding with multiple search criteria. I felt like its having the limitations with large number of filters in Entity framework. In my case i was passing 728 count to just one field of Model, it was breaking with Stack-overflow exceptions.
728 lambdas method would be adding to the stack with full specific NAMESPACES.
Custom Expression is working totally fine in my case. Please find below Source code for the same.
var entityType = typeof(Emptable);
var parameter = Expression.Parameter(entityType, "a");
var containsMethod = typeof(string).GetMethod("Equals", new[] { typeof(string) });
//Switch Statement for EmployeeName Filter.
case FilterBy.EmployeeName:
if (!isEmpNameFilterExists)
{
var propertyExpression = Expression.Property(parameter, "EmployeeName");
foreach (string empName in item.FieldCollection)
{
var innerExpression = Expression.Call(propertyExpression, containsMethod, Expression.Constant(empName));
body = Expression.OrElse(body, innerExpression);
}
}
break;

Related

How can I create Hierarchical Container in Vaadin when I am having the duplicate ItemIds? If No, What is the alternate?

I want to create a tree of user with n level Hierarchy. I have a POJO object and within that I have id,parent_id.
The problem is user can belong in more than 1 group. So, when I am trying to do,
while (iterator.hasNext()) {
val user_pojo_obj = iterator.next()
val key = user_pojo_obj.id
val parent_key = user_pojo_obj.family_id
var child: Item = container.addItem(key)
child.getItemProperty("caption").asInstanceOf[Property[Any]].setValue(user_pojo_obj.name)
child.getItemProperty("POJOobj").asInstanceOf[Property[Any]].setValue(user_pojo_obj)
container.setParent(key, parent_key)
}
I got NullPointerException at the 2nd line, As per my knowledge it because of the addItem() duplication in container, which returns null.
Please suggest me the alternate if this can not be improve. (Using Scala)
Thanxx..
As far as I know, you can not have multiple parents or duplicate itemIds. The following pseudo-code is an alternate solution which builds the tree-like structure (node is your POJO):
counter = 0;
function process(nodes, parent) {
foreach (node in nodes) {
newId = counter++;
item = container.addItem(newId);
// set item caption etc.
if (parent not null)
container.setParent(newId, parent)
process(getNodesWithParent(node), newId);
}
}
process(getNodesWithParent(null), null);
The method getNodesWithParent needs to be defined by you. I guess you will take the iterator, iterate through your POJOs and return those with family_id equals the parameter's id. The overall performance depends on your implementation of getNodesWithParent, so if you have a large data set you should care to be efficient.

Entity Framework - Linq to Entities - strange issue with Anonymous function

Following is the code, I am trying:
public List<Movie> GetMovies()
{
Func<Movie, Movie> prepareMovieOutput =
(input) =>
{
input.DisplayHtmlContent = String.Empty;
return input;
};
var moviesOutput = from m in db.Movies.ToList()
select prepareMovieOutput(m);
return moviesOutput.ToList();
}
public List<Movie> SearchMovies(string searchTerm)
{
var moviesOutput = db.Movies.Where(m => m.Name.Contains(searchTerm)).ToList();
return moviesOutput.ToList();
}
The GetMovies function is working properly, as it returns List collection after clearing DisplayHtmlContent field, whereas, SearchMovies function is supposed to return Movie collection with DisplayHtmlContent field, but inspite of that it returns that field empty.
If I set DisplayHtmlContent to some fixed value (like, "ABC"),both GetMovies and SearchMovies return the list with all Movie having DisplayHtmlContent field as "ABC" value. I don't understand why the function defined in one method should affect the other one. and also how to fix this issue?
Ideally, I want GetMovies to hold all Movie with that particular field as empty string, and SearchMovies to hold all Movie with that field containing value.
Any help on this much appreciated.
this was due to the use of repository. I have removed it and it started working fine. with having EF 5, I didn't need to use repository

NullReferenceException while trying to including a one-to-many relationship item after saving parent

Framework: I'm using using MVC 3 + EntityFramework 4.1 Code-First.
Concept: One Legislation entity has many Provision entities. The idea is that the user enters a Legislation entity, that gets saved then the function that saves it passes it along to another function to see whether that Legislation has a ShortTitle. If it does, then it formats it into a properly worded string and includes it as the Legislation's first Provision, then saves the changes to db.
Issue: The problem is, I've tried coding it in different ways, I keep getting a NullReferenceException, telling me to create a new object instance with the "new" keyword, and points me to the savedLegislation.Provisions.Add(provision); line in my second function.
Here are the two functions at issue, this first one saves the Legislation proper:
public Legislation Save(NewLegislationView legislation)
{
Legislation newLegislation = new Legislation();
// Simple transfers
newLegislation.ShortTile = legislation.ShortTile;
newLegislation.LongTitle = legislation.LongTitle;
newLegislation.BillType = legislation.BillType;
newLegislation.OriginatingChamber = legislation.OriginatingChamber;
newLegislation.Preamble = legislation.Preamble;
// More complicated properties
newLegislation.Stage = 1;
this.NumberBill(newLegislation); // Provides bill number
newLegislation.Parliament = db.LegislativeSessions.First(p => p.Ending >= DateTime.Today);
newLegislation.Sponsor = db.Members.Single(m => m.Username == HttpContext.Current.User.Identity.Name);
// And save
db.Legislations.Add(newLegislation);
db.SaveChanges();
// Check for Short titles
this.IncludeShortTitle(newLegislation);
// return the saved legislation
return newLegislation;
}
And the second function which is invoked by the first one deals with checking whether ShortTitle is not empty and create a Provision that is related to that Legislation, then save changes.
public void IncludeShortTitle(Legislation legislation)
{
var savedLegislation = db.Legislations.Find(legislation.LegislationID);
if (savedLegislation.ShortTile.Any() && savedLegislation.ShortTile.ToString().Length >= 5)
{
string shortTitle = "This Act may be cited as the <i>" + savedLegislation.ShortTile.ToString() + "</i>.";
var provision = new Provision()
{
Article = Numbers.CountOrNull(savedLegislation.Provisions) + 1,
Proponent = savedLegislation.Sponsor,
Text = shortTitle
};
savedLegislation.Provisions.Add(provision);
db.SaveChanges();
}
}
I've been researching how SaveChanges() works and whether it is properly returning the updated entity, it does (since I get no issue looking it up in the second function). If it works properly, and the legislation is found and the provision is newly created in the second function, I don't see what is the "null" reference it keeps spitting out.
The null reference in this case would be savedLegislation.Provisions. The Provisions collection won't be initialized to a new List<Provision> when EF returns your Legislation instance from the db.Legislations.Find(...) method.
The first thing I'd try is something like this:
var savedLegislation = db.Legislations
.Include("Provisions")
.First(l => l.LegislationID == legislation.LegislationID);
... but I'd also consider just using the legislation instance that was passed into the method rather than fetching it from the database again.

EF builds EntityCollection, but I (think I) want IQueryable

I have an entity A with a simple navigation property B. For any given instance of A, we expect several related thousand instances of B.
There is no case where I call something like:
foreach(var x in A.B) { ... }
Instead, I'm only interested in doing aggregate operations such as
var statY = A.B.Where(o => o.Property == "Y");
var statZ = A.B.Where(o => o.CreateDate > DateTime.Now.AddDays(-1));
As far as I can tell, EF instantiates thousands of references to B and does these operations in memory. This is because navigation properties use EntityCollection. Instead, I'd like it to perform these queries at the SQL level if possible.
My current hunch is that Navigation Properties may not be the right way to go. I'm not attached to EF, so I am open to other approaches. But I'd be very interested to know the right way to do this under EF if possible.
(I'm using EF4.)
CreateSourceQuery seems to do the trick.
So my examples would now be:
var statY = A.B.CreateSourceQuery().Where(o => o.Property == "Y");
var statZ = A.B.CreateSourceQuery().Where(o => o.CreateDate > DateTime.Now.AddDays(-1));
There's one thing you should know. Members that derives from IQueryable<> are executed on the server, not in memory. Members which are derived from IEnumerable<> is executed in memory.
for example
var someEntities = db.SomeEntities; <-- returns an IQueryable<> object. no data fetched. SomeEntities table may contain thousands of rows, but we are not fetching it yet, we are just building a query.
someEntities = someEntities.Where(s => s.Id > 100 && s.Id < 200); <-- creates expression tree with where statement. The query is not executed yet and data is not fetched on the client. We just tell EF to perform a where filter when query will execute. This statement too returns an IQueryable<> object.
var entities = someEntities.AsEnumerable(); <-- here we tell EF to execute query. now entities will be fetched and any additional linq query will be performed in memory.
you can also fetch the data using foreach, calling ToArray() or ToList<>.
Hope you understand what I mean, and sorry for my english :)

ADO.NET Mapping From SQLDataReader to Domain Object?

I have a very simple mapping function called "BuildEntity" that does the usual boring "left/right" coding required to dump my reader data into my domain object. (shown below) My question is this - If I don't bring back every column in this mapping as is, I get the "System.IndexOutOfRangeException" exception and wanted to know if ado.net had anything to correct this so I don't need to bring back every column with each call into SQL ...
What I'm really looking for is something like "IsValidColumn" so I can keep this 1 mapping function throughout my DataAccess class with all the left/right mappings defined - and have it work even when a sproc doesn't return every column listed ...
Using reader As SqlDataReader = cmd.ExecuteReader()
Dim product As Product
While reader.Read()
product = New Product()
product.ID = Convert.ToInt32(reader("ProductID"))
product.SupplierID = Convert.ToInt32(reader("SupplierID"))
product.CategoryID = Convert.ToInt32(reader("CategoryID"))
product.ProductName = Convert.ToString(reader("ProductName"))
product.QuantityPerUnit = Convert.ToString(reader("QuantityPerUnit"))
product.UnitPrice = Convert.ToDouble(reader("UnitPrice"))
product.UnitsInStock = Convert.ToInt32(reader("UnitsInStock"))
product.UnitsOnOrder = Convert.ToInt32(reader("UnitsOnOrder"))
product.ReorderLevel = Convert.ToInt32(reader("ReorderLevel"))
productList.Add(product)
End While
Also check out this extension method I wrote for use on data commands:
public static void Fill<T>(this IDbCommand cmd,
IList<T> list, Func<IDataReader, T> rowConverter)
{
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
list.Add(rowConverter(rdr));
}
}
}
You can use it like this:
cmd.Fill(products, r => r.GetProduct());
Where "products" is the IList<Product> you want to populate, and "GetProduct" contains the logic to create a Product instance from a data reader. It won't help with this specific problem of not having all the fields present, but if you're doing a lot of old-fashioned ADO.NET like this it can be quite handy.
Although connection.GetSchema("Tables") does return meta data about the tables in your database, it won't return everything in your sproc if you define any custom columns.
For example, if you throw in some random ad-hoc column like *SELECT ProductName,'Testing' As ProductTestName FROM dbo.Products" you won't see 'ProductTestName' as a column because it's not in the Schema of the Products table. To solve this, and ask for every column available in the returned data, leverage a method on the SqlDataReader object "GetSchemaTable()"
If I add this to the existing code sample you listed in your original question, you will notice just after the reader is declared I add a data table to capture the meta data from the reader itself. Next I loop through this meta data and add each column to another table that I use in the left-right code to check if each column exists.
Updated Source Code
Using reader As SqlDataReader = cmd.ExecuteReader()
Dim table As DataTable = reader.GetSchemaTable()
Dim colNames As New DataTable()
For Each row As DataRow In table.Rows
colNames.Columns.Add(row.ItemArray(0))
Next
Dim product As Product While reader.Read()
product = New Product()
If Not colNames.Columns("ProductID") Is Nothing Then
product.ID = Convert.ToInt32(reader("ProductID"))
End If
product.SupplierID = Convert.ToInt32(reader("SupplierID"))
product.CategoryID = Convert.ToInt32(reader("CategoryID"))
product.ProductName = Convert.ToString(reader("ProductName"))
product.QuantityPerUnit = Convert.ToString(reader("QuantityPerUnit"))
product.UnitPrice = Convert.ToDouble(reader("UnitPrice"))
product.UnitsInStock = Convert.ToInt32(reader("UnitsInStock"))
product.UnitsOnOrder = Convert.ToInt32(reader("UnitsOnOrder"))
product.ReorderLevel = Convert.ToInt32(reader("ReorderLevel"))
productList.Add(product)
End While
This is a hack to be honest, as you should return every column to hydrate your object correctly. But I thought to include this reader method as it would actually grab all the columns, even if they are not defined in your table schema.
This approach to mapping your relational data into your domain model might cause some issues when you get into a lazy loading scenario.
Why not just have each sproc return complete column set, using null, -1, or acceptable values where you don't have the data. Avoids having to catch IndexOutOfRangeException or re-writing everything in LinqToSql.
Use the GetSchemaTable() method to retrieve the metadata of the DataReader. The DataTable that is returned can be used to check if a specific column is present or not.
Why don't you use LinqToSql - everything you need is done automatically. For the sake of being general you can use any other ORM tool for .NET
If you don't want to use an ORM you can also use reflection for things like this (though in this case because ProductID is not named the same on both sides, you couldn't do it in the simplistic fashion demonstrated here):
List Provider in C#
I would call reader.GetOrdinal for each field name before starting the while loop. Unfortunately GetOrdinal throws an IndexOutOfRangeException if the field doesn't exist, so it won't be very performant.
You could probably store the results in a Dictionary<string, int> and use its ContainsKey method to determine if the field was supplied.
I ended up writing my own, but this mapper is pretty good (and simple): https://code.google.com/p/dapper-dot-net/