Converting ESQL to LINQ to Entities. Sort by related entities - entity-framework

I am using EF + RIA and unfortunately meet some problems with sorting by related entities.
For such purpose there is ESQL query that I implemented (found only this solution):
var queryESQL = string.Format(
#" select VALUE ent from SomeEntities as ent
join Attributes as ea ON ea.EntityId = ent.Id
where ea.AttributeTypeId = #typeId
order by ea.{0} {1}", columnName, descending ? "desc" : "asc");
var query = ObjectContext.CreateQuery<SomeEntity>(queryESQL, new ObjectParameter("typeId", attributeTypeId));
Tables have following structure:
<Attribute>:
int Id;
decimal DecimalColumn;
string StringColumn;
int EntityId;
int AttributeTypeId;
<SomeEntity>:
int Id;
string Name;
Is there any way to rewrite this stuff(sorting), using LINQ to Entities approach?

Here's my attempt, I can't guarantee it will work. I need to think more on how to get a dynamic column name, I'm not sure on that one. EDIT: you can use a string for the order column.
int typeId = 1115;
bool orderAscending = false;
string columnName = "StringColumn";
var query = from ent in SomeEntities
join ea in Attributes on ea.EntityId = ent.Id
where ea.AttributeTypeId = typeId;
if(orderAscending)
{
query = query.OrderBy(ea => columnName).Select(ea => ea.Value);
}
else
{
query = query.OrderByDescending(ea => columnName).Select(ea => ea.Value);
}
var results = query.ToList(); // call toList or enumerate to execute the query, since LINQ has deferred execution.
EDIT: I think that ordering after the select stops is from ordering by. I moved the select statement to after the order by. I also added the "query =", but I'm not sure if that is needed. I don't have a way to test this at the moment.
EDIT 3: I fired up LINQPad today and made a few tweaks to what I had before. I modeled your data in a Code-first approach to using EF and it should be close to what you have.
This approach works better if you're just trying to get a list of Attributes (which you aren't). To get around that I added an Entity property to the MyAttribute class.
This code works in LINQPAD.
void Main()
{
// add test entities as needed. I'm assuming you have an Attibutes collection on your Entity based on your tables.
List<MyEntity> SomeEntities = new List<MyEntity>();
MyEntity e1 = new MyEntity();
MyAttribute a1 = new MyAttribute(){ StringColumn="One", DecimalColumn=25.6M, Id=1, EntityId=1, AttributeTypeId = 1, Entity=e1 };
e1.Attributes.Add(a1);
e1.Id = 1;
e1.Name= "E1";
SomeEntities.Add(e1);
MyEntity e2 = new MyEntity();
MyAttribute a2 = new MyAttribute(){ StringColumn="Two", DecimalColumn=198.7M, Id=2, EntityId=2, AttributeTypeId = 1, Entity=e2 };
e2.Attributes.Add(a2);
e2.Id = 2;
e2.Name = "E2";
SomeEntities.Add(e2);
MyEntity e3 = new MyEntity();
MyAttribute a3 = new MyAttribute(){ StringColumn="Three", DecimalColumn=65.9M, Id=3, EntityId=3, AttributeTypeId = 1, Entity=e3 };
e3.Attributes.Add(a3);
e3.Id = 3;
e3.Name = "E3";
SomeEntities.Add(e3);
List<MyAttribute> attributes = new List<MyAttribute>();
attributes.Add(a1);
attributes.Add(a2);
attributes.Add(a3);
int typeId = 1;
bool orderAscending = true;
string columnName = "StringColumn";
var query = (from ent in SomeEntities
where ent.Attributes.Any(a => a.AttributeTypeId == typeId)
select ent.Attributes).SelectMany(a => a).AsQueryable();
query.Dump("Pre Ordering");
if(orderAscending)
{
// query = is needed
query = query.OrderBy(att => MyEntity.GetPropertyValue(att, columnName));
}
else
{
query = query.OrderByDescending(att => MyEntity.GetPropertyValue(att, columnName));
}
// returns a list of MyAttributes. If you need to get a list of attributes, add a MyEntity property to the MyAttribute class and populate it
var results = query.Select(att => att.Entity).ToList().Dump();
}
// Define other methods and classes here
}
class MyAttribute
{
public int Id { get; set; }
public decimal DecimalColumn { get; set; }
public string StringColumn { get; set; }
public int EntityId { get; set; }
public int AttributeTypeId { get; set; }
// having this property will require an Include in EF to return it then query, which is less effecient than the original ObjectQuery< for the question
public MyEntity Entity { get; set; }
}
class MyEntity
{
public int Id { get; set; }
public string Name { get; set; }
public ICollection<MyAttribute> Attributes { get; set; }
public MyEntity()
{
this.Attributes = new List<MyAttribute>();
}
// this could have been on any class, I stuck it here for ease of use in LINQPad
// caution reflection may be slow
public static object GetPropertyValue(object obj, string property)
{
// from Kjetil Watnedal on http://stackoverflow.com/questions/41244/dynamic-linq-orderby
System.Reflection.PropertyInfo propertyInfo=obj.GetType().GetProperty(property);
return propertyInfo.GetValue(obj, null);
}

Related

Building GroupBy Expression Tree - IEnumerable parameter not defined error

I want to build an expression for IQueryable GroupBy. While at the moment I'm just simplifying the problem to try and get it working, the eventual final implementation will involve the creation of quite complex expression trees so I want to build a complete expression that can then be integrated into other expressions.
I specifically want to build an expression of this overload:
public static System.Linq.IQueryable<TResult> GroupBy<TSource,TKey,TResult> (
this System.Linq.IQueryable<TSource> source,
System.Linq.Expressions.Expression<Func<TSource,TKey>> keySelector,
System.Linq.Expressions.Expression<Func<TKey,System.Collections.Generic.IEnumerable<TSource>,TResult>> resultSelector);
... my problem is in the implementation of the resultSelector and and the IEnumerable<TSource>.
I have a table of Customers (just dummy data for the purposes of working out this problem). This is stored in an SQL DB and I specifically want to use IQueryable to access the data.
public class Customer
{
public int Id { get; set; }
public string? FirstName { get; set; }
public string? LastName { get; set; }
public int Age { get; set; }
}
I also have a GroupResult class used to hold the results of the GroupBy (I have different constructors which I've been using in my testing to work out where my problem is occurring)
internal class GroupResult
{
public string? Name { get; set; }
public int NumRecords { get; set; }
public decimal AverageAge { get; set; }
public int TotalAge { get; set; }
public GroupResult() { }
public GroupResult(string name)
{
Name = name;
}
public GroupResult(IEnumerable<Customer> customers)
{
Name = Guid.NewGuid().ToString();
NumRecords = customers.Count();
}
public GroupResult(string name, IEnumerable<Customer> customers)
{
Name = name;
NumRecords = customers.Count();
}
}
The main static class that displays prompts to select column to group on, creates the relevant expression tree and executes it
internal static class SimpleGroupByCustomer
{
internal static DataContext db;
internal static void Execute()
{
using (db = new DataContext())
{
//get input
Console.WriteLine();
Console.WriteLine("Simple Customer GroupBy");
Console.WriteLine("=======================");
Console.WriteLine("Simple GroupBy on the Customer Table");
Console.WriteLine();
Console.WriteLine("Select the property that you want to group by.");
Console.WriteLine();
var dbSet = db.Set<Customer>();
var query = dbSet.AsQueryable();
//for this example we're just prompting for a column in the customer table
//GetColumnName is a helper function that lists the available columns and allows
//one to be selected
string colName = Wrapper.GetColumnName("Customer");
MethodInfo? method = typeof(SimpleGroupByCustomer).GetMethod("GetGroupBy",
BindingFlags.Static | BindingFlags.NonPublic);
if (method != null)
{
method = method.MakeGenericMethod(new Type[] { typeof(String), query.ElementType });
method.Invoke(null, new object[] { query, colName });
}
}
}
internal static void GetGroupBy<T, TTable>(IQueryable query, string colName)
{
Type TTmp = typeof(TTable);
var param = Expression.Parameter(TTmp, "c");
var prop = Expression.PropertyOrField(param, colName);
LambdaExpression keySelector = Expression.Lambda<Func<TTable, T>>(prop, param);
var param1 = Expression.Parameter(typeof(T), "Key");
var param2 = Expression.Parameter(typeof(IEnumerable<TTable>), "Customers");
var ci = typeof(GroupResult).GetConstructor(new[] { typeof(T), typeof(IEnumerable<TTable>) });
//var ci = typeof(GroupResult).GetConstructor(new[] { typeof(T) });
//var ci = typeof(GroupResult).GetConstructor(new[] { typeof(IEnumerable<TTable>) });
if (ci == null)
return;
var pExp = new ParameterExpression[] { param1, param2 };
var methodExpression = Expression.Lambda<Func<T, IEnumerable<TTable>, GroupResult>>(
Expression.New(ci, new Expression[] { param1, param2 }), //<--- ERROR HERE
pExp
);
Type[] typeArgs = new Type[] { typeof(TTable), typeof(T), typeof(GroupResult) };
Expression[] methodParams = new Expression[] { query.Expression, keySelector, methodExpression };
var resultExpression = Expression.Call(typeof(Queryable), "GroupBy", typeArgs, methodParams);
IQueryable dbQuery = query.Provider.CreateQuery(resultExpression);
if (dbQuery is IQueryable<GroupResult> results)
{
foreach (var result in results)
{
Console.WriteLine("{0,-15}\t{1}", result.Name, result.NumRecords.ToString());
}
}
}
}
When I run this and try and iterate through the results I get the following exception:
System.InvalidOperationException: 'variable 'Customers' of type 'System.Collections.Generic.IEnumerable`1[ExpressionTrees3.Data.Customer]' referenced from scope '', but it is not defined'
which is being caused by the param2 ParameterExpression marked above.
If I use the GroupResult constructor that just takes the key value
var ci = typeof(GroupResult).GetConstructor(new[] { typeof(T) });
and omit the param2 from the Lambda body definition the code works as expected and I get a collection of GroupResult records containing the distinct key values in the Name field (but obviously no summary value).
I've tried everything I can think of and just can't get past this error - it's as though the GroupBy is not actually producing the IEnumerable grouping of Customers for each key.
I suspect I'm missing something really obvious here, but just can't see it. Any help would really very much appreciated.
Please note that I am after answers to this specific issue, I'm not looking for alternative ways of doing a GroupBy (unless there's a fundamental reason why this shouldn't work) - this will be rolled into a much larger solution for building queries and I want to use the same process throughout.
Thanks Svyatoslav - as I thought, it was me being especially dumb!
Your comments, as well as a discussion with a friend who has a lot SQL knowledge pointed me in the right direction.
I had been thinking that the GroupBy expression was going to return an Enumerable for each key value and was trying to pass that into a function ... it always felt wrong, but I just ignored that and kept going.
It's obvious now that I need to tell the GroupBy what to calculate and return (i.e. your comment about aggregation).
So for this easy example, the solution is very simple:
var pExp = new ParameterExpression[] { param1, param2 };
var countTypes = new Type[] { typeof(TTable) };
var countParams = new Expression[] { param2 };
var countExp = Expression.Call(typeof(Enumerable), "Count", countTypes, countParams);
var methodExpression = Expression.Lambda<Func<T, IEnumerable<TTable>, GroupResult>>(
Expression.New(ci, new Expression[] { param1, countExp }),
pExp
);
Just by adding the 'Count' expression into the GroupBy method call it works!
.. and adding a new ctor for GroupResult:
public GroupResult(string name, int count)
{
Name = name;
NumRecords = count;
}
(yep, I feel a bit stupid!)

Project into a (list of) concrete object instead of an (list of) anonymous object

The problem simplified is as follows: in Entity Framework i am doing a join involving 3 tables, and returning the joined result set, which involves (some) fields from the 3 tables.
var query = (
from t1 in dbCtx.TB_Entity1
from t2 in dbCtx.TB_Entity2
.Where(p => p.someCol == t1.someCol && t.IsActive == true)
.DefaultIfEmpty() //LEFT JOIN
from t3 in dbCtx.TB_Entity3
.Where(q => q.someCol == t2.someCol)
where t1.IsLatest == true
&& (t1.istatus == 2
|| t1.istatus == 3
)
select new {
t1.col100,
t1.col101,
t2.col200,
t2.col201,
t3.col300,
t3.col301
}).OrderByDescending(t1 => t1.ID);
var anonObjList = query.ToList();
Thus, at the end of the query I write a projection to select the fields i want.
Finally i run the query with .ToList() and get a list of Anonymous objects.
How do i modify the query to project into a List of MyConcreteClass
i.e. i want to be able to write something similar to
List<MyConcreteClass> myObjList = query.ToList();
You may assume my concrete class looks like
public class MyConcreteClass
{
public string Col100 { get; set; }
public string Col101 { get; set; }
public string Col200 { get; set; }
public string Col201 { get; set; }
public string Col300 { get; set; }
public string Col301 { get; set; }
}
You just use the object initializer syntax:
new MyConcreteClass
{
Col100 = t1.col100,
Col101 = t1.col101,
Col200 = t2.col200,
Col201 = t2.col201,
Col300 = t3.col300,
Col301 = t3.col301
}

EF code first Seed many to many relationship not working when fluent API is used

I have an issue with seeding the many to many tables, when I add the Fluent API.
First I have created the two entities, and the many to many relationship between them like this:
[Table("meta.DataCategory")]
public partial class DataCategory : ResolvableEntityBase
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
[Column("DataCategoryId")]
public int Id { get; set; }
[Required]
[StringLength(50)]
[Column("DataCategory")]
[DisplayName("DataCategory")]
public string Name { get; set; }
public virtual ICollection<DestinationTable> DestinationTables { get; set; }
}
[Table("meta.DestinationTable")]
public partial class DestinationTable
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
[Column("DestinationTableId")]
public int Id { get; set; }
[Required]
[StringLength(200)]
[Column("TableName")]
public string Name { get; set; }
[Required]
[StringLength(200)]
[Column("SchemaName")]
public string Schema { get; set; }
public virtual ICollection<DataCategory> DataCategories { get; set; }
}
Adding the migration, and updating the database create the 3 tables, including the many to many table between the DataCategory and DestinationTable. The seed code showen here also worked as it should, and I was able to populate test data to all 3 tables:
context.DataCategories.AddOrUpdate(s => s.Id,
new DAL.DataCategory() { Id = 0, Name = "Unknown", RegexPattern = #"" },
new DAL.DataCategory() { Id = 1, Name = "ProductsMaster", RegexPattern = #"(?:\b|[_]{1})(product|products|produkter|articles)(?:\b|[_]{1})" },
new DAL.DataCategory() { Id = 2, Name = "CustomersTraffic", RegexPattern = #"(?:\b|[_]{1})(trafik|antal)(?:\b|[_]{1})" },
new DAL.DataCategory() { Id = 3, Name = "ProductSales", RegexPattern = #"(?:\b|[_]{1})(salg|sales|ugedata|uge data|uge_data)(?:\b|[_]{1})" },
new DAL.DataCategory() { Id = 4, Name = "CategorySales", RegexPattern = #"(?:\b|[_]{1})(kategory|kategori|category|kat)(?:\b|[_]{1})" },
new DAL.DataCategory() { Id = 5, Name = "StoresMaster", RegexPattern = #"(?:\b|[_]{1})(site\bmaster|store|stores|butik)(?:\b|[_]{1})" },
new DAL.DataCategory() { Id = 6, Name = "MultipleCategories", RegexPattern = #"" },
new DAL.DataCategory() { Id = 7, Name = "ConsultantsMaster", RegexPattern = #"" }
);
DAL.DestinationTable dt = new DAL.DestinationTable();
dt.Id = 1;
dt.Name = "Product";
dt.Schema = "DW";
dt.Type = "Reference";
dt.DataCategories = new List<DAL.DataCategory>();
dt.DataCategories.Add (context.DataCategories.Where(x => x.Name == "ProductsMaster").First());
context.DestinationTables.AddOrUpdate(x => x.Name, dt);
dt = new DAL.DestinationTable();
dt.Id = 2;
dt.Name = "Store";
dt.Schema = "DW";
dt.Type = "Reference";
dt.DataCategories = new List<DAL.DataCategory>();
dt.DataCategories.Add (context.DataCategories.Where(x => x.Name == "StoresMaster").First());
context.DestinationTables.AddOrUpdate(x => x.Name, dt);
dt = new DAL.DestinationTable();
dt.Id = 3;
dt.Name = "ProductSales";
dt.Schema = "DW";
dt.Type = "Transactions";
dt.DataCategories = new List<DAL.DataCategory>();
dt.DataCategories.Add (context.DataCategories.Where(x => x.Name == "ProductSales").First());
dt.DataCategories.Add (context.DataCategories.Where(x => x.Name == "ProductsMaster").First());
dt.DataCategories.Add (context.DataCategories.Where(x => x.Name == "StoresMaster").First());
context.DestinationTables.AddOrUpdate(x => x.Name, dt);
So far so good, however, the many to many table was created under the dbo schema, and the column names used the _Id suffix, which I do not use in the rest of the model, so I had to change that, so I used the following Fluent API in the OnModelCreating() method:
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<DAL.DestinationTable>()
.HasMany<DAL.DataCategory>(dest => dest.DataCategories)
.WithMany(dcat => dcat.DestinationTables)
.Map(m2m =>
{
m2m.MapLeftKey("DestinationTableId");
m2m.MapRightKey("DataCategoryId");
m2m.ToTable("DestinationTableToDataCategory", "meta");
});
}
This made the expected result in terms of naming the table, and naming the columns the way I want it to be, but now the seeding does not work as before.
The seed code populate the DataCatetory records, and the DestinationTable records, but the many to many table is empty. I tried to reveres the direction, so that the DataCategory was on the left side, but the result was the same, that is the many to many table was not populated.
Note that the seed method run completely without errors.
Why does this happens?
What is the way to seed many to many relationship, when using the Fluent API to bypass EF defaults?
Is there something wrong/missing in the Fluent API part?
OK,
I rolled back the entire database to the initial state, and then I applied all the migrations again, the result was that now the seed is working, and the data is populated in all 3 tables.
Cannot say what was wrong.

IQueryable.Select to a List type sub POCO

I have an Entity Framework model in which there is a "Customers" and a "CustomerPhones" table. A customer can have multiple phone numbers so the "Customer" entity has a collection of "Phone". I can query the model with no problem :
using (CustomerEntities context = new CustomerEntities())
{
Customer customer = context.Customers.FirstOrDefault();
CustomerPhone phone = customer.Phones.FirstOrDefault();
MessageBox.Show(customer.Name + " " + phone.Number);
}
The model is too complex for what I need to do (even though my example is basic) so I'm trying to boil it down to simpler POCOs. Here are the 2 simple classes :
public class SimplePhone
{
public int Id { get; set; }
public string Number { get; set; }
}
public class SimpleCustomer
{
public int Id { get; set; }
public string Name { get; set; }
//Phones is a list because a single Customer can have multiple phone numbers
public List<SimplePhone> Phones { get; set; }
}
I can populate the simple properties of the object using the "Select" method of "IQueryable" :
using (CustomerEntities context = new CustomerEntities())
{
IQueryable<SimpleCustomer> customers = context.Customers.Select(
c => new SimpleCustomer
{
Id = c.Id,
Name = c.Name
}
);
SimpleCustomer customer = customers.FirstOrDefault();
MessageBox.Show(customer.Name);
}
So my question is pretty simple : how can I populate the "Phones" property which is a list?
using (CustomerEntities context = new CustomerEntities())
{
IQueryable<SimpleCustomer> customers = context.Customers.Select(
c => new SimpleCustomer
{
Id = c.Id,
Name = c.Name
Phones = ///????
}
);
SimpleCustomer customer = customers.FirstOrDefault();
SimplePhone phone = customer.Phones.FirstOrDefault();
MessageBox.Show(customer.Name + " " + phone.Number);
}
Let me know if I'm unclear and/or you need more details.
Thanks!
I'm not sure if there isn't something more to your question, but as far as I understand, you can just call ToList and it will be materialized as a list:
IQueryable<SimpleCustomer> customers =
context.Customers.Select(c => new SimpleCustomer
{
Id = c.Id,
Name = c.Name,
Phones = c.Phones.Select(p => new SimplePhone
{
Id = p.Id, // Unless you want the custom Id, i.e. c.Id
Number = p.Number
}).ToList();
});

Parse string to poco class enum in linq query MVC 2.0

I have the following enum and POCO class
public enum Gender
{
Male,
Female,
Unknown
}
public class Person
{
public int PersonId { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
public Gender? Gender { get; set; }
}
I would like to perform a "get all people" query in my repository such that it would look something like this:
return from p in _db.People
select new Model.Person
{
PersonId = p.PersonId,
LastName = p.LastName,
FirstName = p.FirstName,
Gender = p.Gender,
};
Unfortunately I get an error "Cannot implicitly convert type 'string' to 'Model.Gender'"
I would like to convert the string which is being queried from the entity framework to my Gender enum and assign it to my POCO class.
Enums are not supported in Entity Framework. There is a workaround by Alex James, but it's quite involved.
Instead, i prefer to do this:
public enum Gender : byte
{
Male = 1,
Female,
Unknown
}
public class Person
{
public int PersonId { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
public byte Gender { get; set; } // this is the EF model property
public Gender GenderType // this is an additional custom property
{
get { return (Gender) Gender; }
set { Gender = (byte)value; }
}
}
It's basically a hook/wrapper for the actual value. In your database, store Gender as a tinyint (which maps to byte on the conceptual side).
Then you can use a byte enum to map to and from the model property:
return from p in _db.People
select new Model.Person
{
PersonId = p.PersonId,
LastName = p.LastName,
FirstName = p.FirstName,
Gender = p.Gender, // sets byte
};
But then if you access that ViewModel, because your setting the byte field for Gender, you will also have access to the enum property GenderType.
Does that solve your problem?
The Entity Framework that I am familiar with does not provide support for enums. EF uses your query expression to create an SQL statement that it then sends to the server, if it cannot create the SQL equivalent of some operation it will throw a NotSupportedException for that operation. If you are expecting to return a small set of data you can separate from the Entity Framework by creating an object in memory using the ToArray method.
var myEntities = (from entity in _db.Entities
where /* condition */
select entity)
.ToArray();
This will create a sequence of entities in memory. Any further query statements will then be in the realm of LINQ to Objects which allows parsing of strings into enums:
return from myEntity in myEntities
select new MyDataContract
{
ID = myEntity.ID,
Gender g = (Gender)Enum.Parse(typeof(Gender), myEntity.Gender, true)
};
Or you could even break it out into a foreach loop:
List<MyDataContract> myDataContracts = new List<MyDataContract>();
foreach (var myEntity in myEntities)
{
var dataContract = new MyDataContract { ID = myEntity.ID };
if (Enum.IsDefined(typeof(Gender), myEntity.Gender))
dataContract.Gender = (Gender)Enum.Parse(typeof(Gender), myEntity.Gender, true);
myDataContracts.Add(dataContract);
}
return myDataContracts.AsEnumerable();
if (Enum.IsDefined(typeof(Gender), genderstring))
Gender g = (Gender) Enum.Parse(typeof(Gender), genderstring, true);
else
//Deal with invalid string.
try
Gender = p.Gender != null ? (Gender)Enum.Parse(typeof(Gender), p.Gender) : (Gender?)null;
To parse the string as one of the enums
here's a workaround but it means changing your nice and clean POCO
http://blogs.msdn.com/b/alexj/archive/2009/06/05/tip-23-how-to-fake-enums-in-ef-4.aspx