I have an entity A with a simple navigation property B. For any given instance of A, we expect several related thousand instances of B.
There is no case where I call something like:
foreach(var x in A.B) { ... }
Instead, I'm only interested in doing aggregate operations such as
var statY = A.B.Where(o => o.Property == "Y");
var statZ = A.B.Where(o => o.CreateDate > DateTime.Now.AddDays(-1));
As far as I can tell, EF instantiates thousands of references to B and does these operations in memory. This is because navigation properties use EntityCollection. Instead, I'd like it to perform these queries at the SQL level if possible.
My current hunch is that Navigation Properties may not be the right way to go. I'm not attached to EF, so I am open to other approaches. But I'd be very interested to know the right way to do this under EF if possible.
(I'm using EF4.)
CreateSourceQuery seems to do the trick.
So my examples would now be:
var statY = A.B.CreateSourceQuery().Where(o => o.Property == "Y");
var statZ = A.B.CreateSourceQuery().Where(o => o.CreateDate > DateTime.Now.AddDays(-1));
There's one thing you should know. Members that derives from IQueryable<> are executed on the server, not in memory. Members which are derived from IEnumerable<> is executed in memory.
for example
var someEntities = db.SomeEntities; <-- returns an IQueryable<> object. no data fetched. SomeEntities table may contain thousands of rows, but we are not fetching it yet, we are just building a query.
someEntities = someEntities.Where(s => s.Id > 100 && s.Id < 200); <-- creates expression tree with where statement. The query is not executed yet and data is not fetched on the client. We just tell EF to perform a where filter when query will execute. This statement too returns an IQueryable<> object.
var entities = someEntities.AsEnumerable(); <-- here we tell EF to execute query. now entities will be fetched and any additional linq query will be performed in memory.
you can also fetch the data using foreach, calling ToArray() or ToList<>.
Hope you understand what I mean, and sorry for my english :)
Related
I have this DbContext:
public class DataContext : DbContext
{
public DbSet<Base> Bases {get;set}
public DbSet<Sub> Subs {get;set}
}
Sub is a subclass of Base.
When I'm querying the list of Base entities like so:
Context.Bases.ToListAsync()
It returns me every entities, either Base or Sub.
How can I configure my model context to get only the entities that are of Base type and not the ones that derives from it.
The best (or least worst) solution I found is to directly use the shadow property:
Context.Bases.Where(b => EF.Property<string>(b, "Discriminator") == "Base")).ToListAsync();
It works but needs to be repeted now and then, each time I need to query Bases. I'd have prefered a solution in the OnModelCreating method.
I'll accept this answer unless someone else find a better solution.
You'd have to use OfType<T>:
var basesOnly = await _context.Bases.OfType<Base>().ToListAsync();
UPDATE
Sorry, then. I could have sworn the above works, but it doesn't. The next best method I can think of is to simply filter out the types you don't want. It's not ideal, because it requires specifying all subtypes in your query, which then means you need to remember to update it if you add more subtypes.
var basesOnly = await _context.Bases.Where(x => !(x is Sub)).ToListAsync();
How can I configure my model context to get only the entities that are
of Base type and not the ones that derives from it.
You cannot. Every Sub is a Base. So querying all Bases includes all Subs. Eg code like the following must succeed:
Base b = db.Bases.Where(i => i.Id == 1).Single();
if (b is Sub)
begin
Sub s = (Sub)b;
. . .
end
else //other Sub
begin
Sub2 s = (Sub2)b;
. . .
end
You can fetch an anonymous type with the just the base class properties.
And asking this question suggests that inheritance might is not the right modeling technique for your scenario.
If what you want it to fetch the Entities of type Base, but not the subtype Sub, then you can do that with a query like:
var q = from b in db.Bases
where !(b is Sub)
select b;
Which translates to :
SELECT [b].[Id], [b].[Discriminator], [b].[Name], [b].[Size]
FROM [Bases] AS [b]
WHERE [b].[Discriminator] IN (N'Sub', N'Base')
AND NOT ([b].[Discriminator] = N'Sub')
But you can't (currently) exclude all subtypes without enumerating them. Eg this query:
var q2 = from b in db.Bases
where b.GetType() == typeof(Base)
select b;
Will not be completely translated to SQL, and will filter out the subtypes on the client.
I have Predicate builder, which is having predicate and inner predicate and building a dynamic filter based on conditions, let's say I am selecting one department, under that department I am getting list of employees, once I get the list of employees, I need to load the respective records for each and every employee who belongs to the selected department.
Implementation is already done long back and it works fine if department is having not too many employees, once it goes beyond 500 or 1000, the predicate builder is causing a stack overflow. Please see my code snippet for this - I am using .net framework 4.5.2.
Getting stackoverflow exception when assigning to inner predicate at this line with in loop, when record is beyond 1000 or 500, it loops based on the employee records.
Expression<Func<EmployeeTable, bool>> predicate = PredicateBuilder.True<EmployeeTable>();
var innerPredicate = PredicateBuilder.False<EmployeeTable>();
case FilterBy.EmployeeName:
if (!isEmpNameFilterExists)
{
foreach (string empName in item.FieldCollection)
{
innerPredicate = innerPredicate.Or(x => x.Name.Equals(empName,
StringComparison.OrdinalIgnoreCase));
}
predicate = predicate.And(innerPredicate.Expand());
}
break;
This might happen due to the (usually sufficient) but small stack for .NET applications (https://stackoverflow.com/a/823729/2298807). Evaluation on predicates is usually done as part of lamda functions and they use the stack. I do not now the predicate library in detail but I assume a use of recursive functions.
Anyhow: I would suggest to use Contains by building a List<string> containing the names:
Expression<Func<EmployeeTable, bool>> predicate =
PredicateBuilder.True<EmployeeTable>();
var innerPredicate = PredicateBuilder.False<EmployeeTable>();
case FilterBy.EmployeeName:
if (!isEmpNameFilterExists)
{
List<string> namesList = new List<string>();
foreach (string empName in item.FieldCollection)
{
namesList.Add(empName);
}
predicate = predicate.And(x => namesList.Contains(x.Name));
}
break;
Note: Please check the syntax as I do not have a VS environment available at the moment.
I have added my own Expression builder engine, i.e. much better way to generate the Predicate.
PredicateBuilder works well with LINQ to Object, With EntityFramework its having the issue, because it generates the Lambda methods with full namespace of models and keep on adding with multiple search criteria. I felt like its having the limitations with large number of filters in Entity framework. In my case i was passing 728 count to just one field of Model, it was breaking with Stack-overflow exceptions.
728 lambdas method would be adding to the stack with full specific NAMESPACES.
Custom Expression is working totally fine in my case. Please find below Source code for the same.
var entityType = typeof(Emptable);
var parameter = Expression.Parameter(entityType, "a");
var containsMethod = typeof(string).GetMethod("Equals", new[] { typeof(string) });
//Switch Statement for EmployeeName Filter.
case FilterBy.EmployeeName:
if (!isEmpNameFilterExists)
{
var propertyExpression = Expression.Property(parameter, "EmployeeName");
foreach (string empName in item.FieldCollection)
{
var innerExpression = Expression.Call(propertyExpression, containsMethod, Expression.Constant(empName));
body = Expression.OrElse(body, innerExpression);
}
}
break;
Given an EntityObject, I'd like an object-oriented way find all related items as part of my data-source query.
The following produces the correct output, but brings all the rows over the wire to do it.
Parent. // EntityObject
Children. // EntityCollection
Where(o => o.Gender == 'm'). // IEnumerable (local!)
OrderBy(o => o.Age). // IOrderedEnumerable
Skip(pages * pageSize).Take(pageSize); // (Inefficient paging!)
I need to support a UI with this (filter using other criteria, sort, and paginate before returning results over the wire). I reordered to leverage Queryable:
Repository. // Repository
Children. // ObjectSet
Where(o => o.Parent == Parent && o.Gender == 'm'). // ObjectQuery, runtime error
OrderBy(o => o.Age). // IOrderedQueryable
Skip(pages * pageSize).Take(pageSize);
but this yields the error:
Unable to create a constant value of type 'DataModel.Parent'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.
Is there a natural, object-oriented way to query on this relation, using Linq to Entities? Or do I necessarily fall-back to SQL for this?
I thought for a moment that CreateSourceQuery was going to be the answer, but it can't be applied to an EntityObject.
I can't do a test for what I'm saying, but I think that you get the error because EF doesn't know how to translate o.Parent == Parent into a SQL statement. Try compare the Ids of the two parents..
o.Parent.Id == Parent.Id
Let's suppose I have the following code:
TEModule teModule = Context.TEModules.Where(module => module.EnumValue.Equals(text.ModuleName)).FirstOrDefault();
if (teModule == null)
{
teModule = new TEModule();
teModule.EnumValue = text.ModuleName;
Context.TEModules.AddObject(teModule);
//Context.SaveChanges();
TEModule aux = Context.TEModules.Where(module => module.EnumValue.Equals(teModule.ModuleName)).FirstOrDefault();
}
My problem is that if I keep the "SaveChanges" commented, then on the next query the aux object is always null, because Context.TEModules is empty, even when I call the "AddObject" method.
However, if I call SaveChanges after AddObject, then on the next query the aux object is not null. The problem is that I don't want to call SaveChanges so often, because this is not the only piece of code in which I add objects, and the performance goes down if I do so.
So question is: Do I have to call SaveChanges after every AddObject call, if later I need to know if the object already exists?
The purpose of linq-to-entities query is to be executed and the execution is performed in the database so if you didn't saved the entity its database representation doesn't exist.
If you need to find locally stored entities (not persisted yet) you must query ObjectStateManager instead.
var entity = Context.ObjectStateManager.GetObjectStateEntries(EntitiState.Added)
.Where(e => !e.IsRelationship)
.Select(e => e.Entity)
.OfType<TEModule>()
.FirstOrDefault(m => m.EnumValue.Equals(teModule.ModuleName));
I have a very simple mapping function called "BuildEntity" that does the usual boring "left/right" coding required to dump my reader data into my domain object. (shown below) My question is this - If I don't bring back every column in this mapping as is, I get the "System.IndexOutOfRangeException" exception and wanted to know if ado.net had anything to correct this so I don't need to bring back every column with each call into SQL ...
What I'm really looking for is something like "IsValidColumn" so I can keep this 1 mapping function throughout my DataAccess class with all the left/right mappings defined - and have it work even when a sproc doesn't return every column listed ...
Using reader As SqlDataReader = cmd.ExecuteReader()
Dim product As Product
While reader.Read()
product = New Product()
product.ID = Convert.ToInt32(reader("ProductID"))
product.SupplierID = Convert.ToInt32(reader("SupplierID"))
product.CategoryID = Convert.ToInt32(reader("CategoryID"))
product.ProductName = Convert.ToString(reader("ProductName"))
product.QuantityPerUnit = Convert.ToString(reader("QuantityPerUnit"))
product.UnitPrice = Convert.ToDouble(reader("UnitPrice"))
product.UnitsInStock = Convert.ToInt32(reader("UnitsInStock"))
product.UnitsOnOrder = Convert.ToInt32(reader("UnitsOnOrder"))
product.ReorderLevel = Convert.ToInt32(reader("ReorderLevel"))
productList.Add(product)
End While
Also check out this extension method I wrote for use on data commands:
public static void Fill<T>(this IDbCommand cmd,
IList<T> list, Func<IDataReader, T> rowConverter)
{
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
list.Add(rowConverter(rdr));
}
}
}
You can use it like this:
cmd.Fill(products, r => r.GetProduct());
Where "products" is the IList<Product> you want to populate, and "GetProduct" contains the logic to create a Product instance from a data reader. It won't help with this specific problem of not having all the fields present, but if you're doing a lot of old-fashioned ADO.NET like this it can be quite handy.
Although connection.GetSchema("Tables") does return meta data about the tables in your database, it won't return everything in your sproc if you define any custom columns.
For example, if you throw in some random ad-hoc column like *SELECT ProductName,'Testing' As ProductTestName FROM dbo.Products" you won't see 'ProductTestName' as a column because it's not in the Schema of the Products table. To solve this, and ask for every column available in the returned data, leverage a method on the SqlDataReader object "GetSchemaTable()"
If I add this to the existing code sample you listed in your original question, you will notice just after the reader is declared I add a data table to capture the meta data from the reader itself. Next I loop through this meta data and add each column to another table that I use in the left-right code to check if each column exists.
Updated Source Code
Using reader As SqlDataReader = cmd.ExecuteReader()
Dim table As DataTable = reader.GetSchemaTable()
Dim colNames As New DataTable()
For Each row As DataRow In table.Rows
colNames.Columns.Add(row.ItemArray(0))
Next
Dim product As Product While reader.Read()
product = New Product()
If Not colNames.Columns("ProductID") Is Nothing Then
product.ID = Convert.ToInt32(reader("ProductID"))
End If
product.SupplierID = Convert.ToInt32(reader("SupplierID"))
product.CategoryID = Convert.ToInt32(reader("CategoryID"))
product.ProductName = Convert.ToString(reader("ProductName"))
product.QuantityPerUnit = Convert.ToString(reader("QuantityPerUnit"))
product.UnitPrice = Convert.ToDouble(reader("UnitPrice"))
product.UnitsInStock = Convert.ToInt32(reader("UnitsInStock"))
product.UnitsOnOrder = Convert.ToInt32(reader("UnitsOnOrder"))
product.ReorderLevel = Convert.ToInt32(reader("ReorderLevel"))
productList.Add(product)
End While
This is a hack to be honest, as you should return every column to hydrate your object correctly. But I thought to include this reader method as it would actually grab all the columns, even if they are not defined in your table schema.
This approach to mapping your relational data into your domain model might cause some issues when you get into a lazy loading scenario.
Why not just have each sproc return complete column set, using null, -1, or acceptable values where you don't have the data. Avoids having to catch IndexOutOfRangeException or re-writing everything in LinqToSql.
Use the GetSchemaTable() method to retrieve the metadata of the DataReader. The DataTable that is returned can be used to check if a specific column is present or not.
Why don't you use LinqToSql - everything you need is done automatically. For the sake of being general you can use any other ORM tool for .NET
If you don't want to use an ORM you can also use reflection for things like this (though in this case because ProductID is not named the same on both sides, you couldn't do it in the simplistic fashion demonstrated here):
List Provider in C#
I would call reader.GetOrdinal for each field name before starting the while loop. Unfortunately GetOrdinal throws an IndexOutOfRangeException if the field doesn't exist, so it won't be very performant.
You could probably store the results in a Dictionary<string, int> and use its ContainsKey method to determine if the field was supplied.
I ended up writing my own, but this mapper is pretty good (and simple): https://code.google.com/p/dapper-dot-net/