So I partially followed from an SO answer on how to store a property with array datatype in Entity Framework. What I didn't follow on that answer is setting the string InternalData to be private instead of public as I find it a code smell if it is set to public (not enough reputation to comment there yet).
I also managed to map the private property in entity framework from this blog.
When I perform CR (create, read) from that entity, all goes well. However, when my LINQ query has a where clause using that property with array datatype, it says that "System.NotSupportedException: 'The specified type member is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.'".
How to work around on this? Here are the relevant code blocks:
public class ReminderSettings
{
[Key]
public string UserID { get; set; }
[Column("RemindForPaymentStatus")]
private string _remindForPaymentStatusCSV { get; set; }
private Status[] _remindForPaymentStatus;
[NotMapped]
public Status[] RemindForPaymentStatus
{
get
{
return Array.ConvertAll(_remindForPaymentStatusCSV.Split(','), e => (Status)Enum.Parse(typeof(Status), e));
}
set
{
_remindForPaymentStatus = value;
_remindForPaymentStatusCSV = String.Join(",", _remindForPaymentStatus.Select(x => x.ToString()).ToArray());
}
}
public static readonly Expression<Func<ReminderSettings, string>> RemindForPaymentStatusExpression = p => p._remindForPaymentStatusCSV;
}
public enum Status
{
NotPaid = 0,
PartiallyPaid = 1,
FullyPaid = 2,
Overpaid = 3
}
protected override void OnModelCreating(DbModelBuuilder modelBuilder)
{
modelBuilder.Entity<ReminderSettings>().Property(ReminderSettings.RemindForPaymentStatusExpression);
}
//This query will cause the error
public IEnumerable<ReminderSettings> GetReminderSettingsByPaymentStatus(Status[] statusArray)
{
var query = ApplicationDbContext.ReminderSettings.Where(x => x.RemindForPaymentStatus.Intersect(statusArray).Any());
return query.ToList(); //System.NotSupportedException: 'The specified type member 'RemindForPaymentStatus' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.'
}
Entity Framework can not translate LINQ expressions to SQL if they access a property that is annotated as [NotMapped]. (It also can not translate if the property contains custom C# code in its getter/setter).
As a quick (but potentially low performance) workaround, you can execute the part of the query that does not cause problems, then apply additional filtering in-memory.
// execute query on DB server and fetch items into memory
var reminders = dbContext.ReminderSettings.ToList();
// now that we work in-memory, LINQ does not need to translate our custom code to SQL anymore
var filtered = reminders.Where(r => r.RemindForPaymentStatus.Contains(Status.NotPaid));
If this causes performance problems, you have to make the backing field of your NotMapped property public and work directly with it.
var filtered = dbContext.ReminderSettings
.Where(r => r._remindForPaymentStatusCSV.Contains(Status.NotPaid.ToString("D"));
Edit
To handle multiple Status as query parameters, you can attach Where clauses in a loop (which behaves like an AND). This works as long as your Status enum values are distinguishable (i.e. there is no Status "11" if there is also a Status "1").
var query = dbContext.ReminderSettings.Select(r => r);
foreach(var statusParam in queryParams.Status) {
var statusString = statusParam.ToString("D");
query = query.Where(r => r._remindForPaymentStatusCSV.Contains(statusString));
}
var result = query.ToArray();
Related
Background
We have a class library which has a grid (inherits from WPF DataGrid) with refresh functionality. The grid has a IQueryable Query property, which enables the refresh. Each grid's query is defined not in the class library, but in the referencing end-project:
var dg = new RefreshableDataGrid();
dg.Query = () => new ProjectDbContext().Persons;
Each grid also has a textbox for text filtering. When text is entered in the filter, an expression is generated which checks if any string property or string-convertible property (using SqlFunctions.StringConvert) contains the filter string. The expression is then appended to the original query as an argument to Where, and thus only the records containing matching strings are returned.
//within the class library
//pseudo-code -- this is actually done via reflection, because at compile time the
//actual type of the grid is not known, and there is no generic placeholder
this.ItemsSource = this.Query.Where(filterExpression)
In some cases, the filter logic is defined in end-projects, on the entity type. For example:
public interface IFilterable {
public Expression<Func<String, Boolean>> TextSearchExpression();
}
public class Email {
public int ID {get;set;}
public int PersonID {get;set;}
public string Address {get;set;}
}
public class Person : IFilterable
public int ID {get;set;}
public string LastName {get;set;}
public string FirstName {get;set;}
public Expression<Func<String, Boolean>> TextSearchExpression() {
Dim ctx = new ProjectDbContext();
return phrase => LastName.Contains(phrase) || FirstName.Contains(phrase) ||
ctx.Emails.Where(x => x.PersonID = ID && x.Address.Contains(prase).Any();
}
}
This expression tree uses an instance of the project-specific context, which is a different instance from that of the original query. Queries cannot use components from multiple contexts (at least not in Entity Framework). I can rewrite the expression tree to use a specific instance, but I need to extract the original instance from the query.
It seems obvious that the query holds some reference to the context instance, otherwise the query would not be able to return results.
I do not want to pass the context instance to the class library.
Hence:
Given a query, how can I get the strongly-typed DbContext instance used to create the query?
In other words, what goes in the body of this method:
DbContext GetDbContext<TSource>(IQueryable<TSource> qry) {
// ???
}
It seems obvious that the query holds some reference to the context instance, otherwise the query would not be able to return results.
That's true, but it's implementation specific detail and in EF is encapsulated inside internal members/classes/interfaces.
Also taking into account that DbContext is build on top of the ObjectContext, holding a reference to the DbContext is not strongly necessary. Fortunately that's not the case :)
The following uses reflection and implementation details of the latest EF6.1.3 (tested and working if you don't use some 3rd party extensions like LinqKit and similar that replace the query provider):
public static DbContext GetDbContext<TSource>(this IQueryable<TSource> query)
{
const BindingFlags flags = BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public;
var provider = query.Provider;
var internalContextProperty = provider.GetType().GetProperty("InternalContext", flags);
if (internalContextProperty == null) return null;
var internalContext = internalContextProperty.GetValue(provider, null);
if (internalContext == null) return null;
var ownerProperty = internalContext.GetType().GetProperty("Owner", flags);
if (ownerProperty == null) return null;
var dbContext = (DbContext)ownerProperty.GetValue(internalContext, null);
return dbContext;
}
I would recommend passing an instance of MyDataContext into your query function
public class DACPerson
{
public static IQueryable<Person> GetAllAsQueryable(MyDataContext db)
{
return db.People.AsQueryable();
}
}
This allows you to do the following in the calling function:
public List<Person> UpdateListofPeople(string term)
{
using(DataContext db = new DataContext())
{
var people = DACPerson.GetAllAsQueryable(db);
var result = people.Where(x=>x.Username.Contains(term)).
//call other Data Access Component FUnctions here using same DB....
}
}
i.e. you can bring the filtering to the layer above the data access class.
Some people may not like to do this. You may get the advice that its best to keep all entityframeowrk functionality within the same layer and just return the DTO. I like the above approach though. It depends on who will have to maintain each part of your application in the future.
Hope this helps at least
Supposing I have three EF entity classes:
public class Person {
...
public ICollection Vehicles { get; set; }
}
public class Vehicle {
...
public Person Owner { get; set; }
public CarModel ModelInfo { get; set; }
}
public class CarModel {
...
// properties for make, model, color, etc
}
The Person.Vehicles property is lazy-loaded.
Supposing I have a Person instance already loaded and I want to load its Vehicle collection property such that it also includes the related ModelInfo property.
So I have this:
void LoadVehiclesAndRelated(MyDbContext dbContext, Person person)
{
dbContext.Entry( person )
.Collection( p => p.Vehicles )
.Query()
.Include( v => v.ModelInfo )
.Load();
}
Used like so:
using( MyDbContext dbContext = ... ) {
Person p = GetPerson( 123 );
LoadVehiclesAndRelated( dbContext, p );
}
foreach(Vehicle v in p.Vehicles) {
Console.WriteLine( v.ModelInfo );
}
However when I do this, I get an exception at runtime when it first evaluates the p.Vehicles expression because the property is actually empty (so it wants to load it) but the DbContext is now disposed.
When the .Load() call was made (inside LoadVehiclesAndRelated() I saw the SQL being executed against the server (in SQL Server Profiler) but the collection property remains empty.
How can I then load the property and with the Included sub-properties?
Annoyingly, this scenario is not mentioned in the MSDN guide for explicit-loading: https://msdn.microsoft.com/en-us/data/jj574232.aspx
Looks like calling .Query().Load() is not the same as calling DbCollectionEntry.Load directly, and the important difference is that the former does not set the IsLoaded property, which then is causing triggering lazy load later. Most likely because as explained in the link, the former is intended to be used for filtered (partial) collection load scenarios.
Shortly, to fix the issue, just set IsLoaded to true after loading the collection:
var entry = dbContext.Entry(person);
var vehicleCollection = entry.Collection(p => p.Vehicles);
vehicleCollection.Query()
.Include( v => v.ModelInfo )
.Load();
vehicleCollection.IsLoaded = true;
P.S. For the sake of correctness, this behavior is sort of mentioned at the end of the Applying filters when explicitly loading related entities section:
When using the Query method it is usually best to turn off lazy loading for the navigation property. This is because otherwise the entire collection may get loaded automatically by the lazy loading mechanism either before or after the filtered query has been executed.
What is the feasibility of modifying the mapping code to convert a short of value zero or non-zero to false or true, if the boolean destination property is marked with an attribute in the POCO model?
I mean, this is supposed to be one of the advantages of EF being open sourced, and would be for in house use only.
Any tips on where in the code I would look would be appreciated, but this question is really more general and I'd like to hear anything anyone has to say on this.
With regard to the General comments please.
I dont know to make the EF change, but dealing with similar issues is not an uncommon issue in EF.
Not all standard types are supported by EF.
You can have a helper field in your POCO class.
So one field is the actual DB field, but no used outside of POCO.
The help field is NOTMAPPED or ignored in fluent API.
You access the DB via you helper and execute any required casting.
A simple example. Or the reverse if I got helper and DB field types back to front.
[NotMapped]
public virtual bool IsVisible { set; get; } // Helper Field NOT on DB
public int Test { get { return IsVisible ? 1 : 0; } // on DB, but set and get via helper only.
set { IsVisible = (value != 0); } }
Edit: Power Fluent API
Here is a snippet that outlines how you have code that runs for every mapped poco in a consistent way.
public class MyDbContext : DbContext
// model building, set breakpoint so you know when this is triggered
// it is important this ISNT called everytime, only on model cache.
// in my case that is app pool recycle.
protected override void OnModelCreating(DbModelBuilder modelBuilder)
// use the CONFIG add feature to better organize and allow use of inheritance when mapping
// I will use snippets and statics to keep it simple.
modelBuilder.Configurations.Add(XYZMap.Map()); // POCO map
modelBuilder.Configurations.Add(ABCMAP.Map()); // poco map
modelBuilder.Configurations.Add(XXXMap.MAP()); // poco map
// etc for your POCO set
// Note, no need to declare DBset<xyz> XYZs {get;set;} !!!!
public static class XYZMap {
public static BaseEntityIntConfiguration<PocoXYZ> Map() {
//see return object !
var entity = new BaseEntityLongConfiguration<PocoXYZ>();
//entity.Property()... // map away as usual POCO specifc
///entity.HasRequired()...// property and relationships as required
// do nothing for default
return entity;
}
}
}
// all tables with int key use this base config. do it once never again
public class BaseEntityIntConfiguration<T> : BaseEntityConfiguration<T> where T : BaseObjectInt {
public BaseEntityIntConfiguration(DatabaseGeneratedOption DGO = DatabaseGeneratedOption.Identity) {
// Primary Key
this.HasKey(t => t.Id);
// Properties
//Id is an int allocated by DB
this.Property(t => t.Id).HasDatabaseGeneratedOption(DGO); // default to db generated
// optimistic lock is also added here, Specific to out poco design
this.Property(t => t.RowVersion)
.IsRequired()
.IsFixedLength()
.HasMaxLength(8)
.IsRowVersion();
// any other common mappings/ rules ??
}
}
public class BaseEntityConfiguration<T> : EntityTypeConfiguration<T> where T : BaseObject {
public BaseEntityConfiguration() {
this.ApplyAttributeRules(); // <<<<< Here is where I apply SYSTEM WIDE rules
}
}
public static void ApplyAttributeRules<T>(this EntityTypeConfiguration<T> entity) where T : BaseObject {
// so this will be called for each mapped type
foreach (var propertyInfo in typeof (T).GetProperties()) {
// I use reflection to look for properties that meet certain criteria.
// eg string. I want as NVARCHAR 4000 not NVCAHR max so i can index it.
if (propertyInfo.UnderLyingType().FullName == "System.String") {
SetStringLength(BosTypeTool.StringLengthIndexable, propertyInfo.Name, entity);
continue;
}
SetStringLength(4000, propertyInfo.Name, entity);
}
}
private static void SetStringLength<TModelPoco>(int length, string propertyName,
EntityTypeConfiguration<TModelPoco> entity) where TModelPoco : BaseObject {
var propLambda = DynamicExpression.ParseLambda<TModelPoco, String>(propertyName);
entity.Property(propLambda).HasMaxLength(length);
// dynamic library from Microsoft.... http://msdn.microsoft.com/en-US/vstudio/bb894665.aspx
}
// get underlying type incase it is nullable
public static Type UnderLyingType(this PropertyInfo propertyInfo) {
return Nullable.GetUnderlyingType(propertyInfo.PropertyType) ?? propertyInfo.PropertyType;
}
I have the following method automatically generated from the scaffold template with repository:-
public Group Find(int id)
{
return context.Groups.Find(id);
}
But since the Groups object has two navigation properties which I need , so I wanted to include the .Include, so I replace the .find with .where :-
public Group Find(int id)
{
return context.Groups.Where(c=>c.GroupID==id)
.Include(a => a.UserGroups)
.Include(a2 => a2.SecurityRoles)
.SingleOrDefault();
}
But my question is how can I apply the .Include with the .find() instead of using .Where()?
I was just thinking about what find actually does. #lazyberezovsky is right include and find cant be used in conjunction with each other. I think this is quite deliberate and here's why:
The Find method on DbSet uses the primary key value to attempt to find
an entity tracked by the context. If the entity is not found in the
context then a query will be sent to the database to find the entity
there. Null is returned if the entity is not found in the context or
in the database.
Find is different from using a query in two significant ways:
A round-trip to the database will only be made if the entity with the given key is not found in the context.
Find will return entities that are in the Added state. That is, Find will return entities that have been added to the context but have
not yet been saved to the database.
(from http://msdn.microsoft.com/en-us/data/jj573936.aspx)
Because find is an optimised method it can avoid needing a trip to the server. This is great if you have the entity already tracked, as EF can return it faster.
However if its not just this entity which we are after (eg we want to include some extra data) there is no way of knowing if this data has already been loaded from the server. While EF could probably make this optimisation in conjunction with a join it would be prone to errors as it is making assumptions about the database state.
I imagine that include and find not being able to be used together is a very deliberate decision to ensure data integrity and unnecessary complexity. It is far cleaner and simpler
when you are wanting to do a join to always go to the database to perform that join.
You can't. Find method defined on DbSet<T> type and it returns entity. You can't call Include on entity, so the only possible option is calling Find after Include. You need DbSet<T> type for that, but Include("UserGroups") will return DbQuery<T>, and Include(g => g.UserGroups) will also return DbQuery<T>:
public static IQueryable<T> Include<T>(this IQueryable<T> source, string path)
where T: class
{
RuntimeFailureMethods.Requires(source != null, null, "source != null");
DbQuery<T> query = source as DbQuery<T>;
if (query != null)
return query.Include(path); // your case
// ...
}
DbQuery<T> is not a child of DbSet<T> thus method Find is not available. Also keep in mind, that Find first looks for entity in local objects. How would it include some referenced entities, if they don't loaded yet?
You can try to do this:
public static class DbContextExtention
{
public static TEntity FirstOfDefaultIdEquals<TEntity, TKey>(
this IQueryable<TEntity> source, TKey otherKeyValue)
where TEntity : class
{
var parameter = Expression.Parameter(typeof(TEntity), "x");
var property = Expression.Property(parameter, "ID");
var equal = Expression.Equal(property, Expression.Constant(otherKeyValue));
var lambda = Expression.Lambda<Func<TEntity, bool>>(equal, parameter);
return source.FirstOrDefault(lambda);
}
public static TEntity FirstOfDefaultIdEquals<TEntity>(
this ObservableCollection<TEntity> source, TEntity enity)
where TEntity : class
{
var value = (int)enity.GetType().GetProperty("ID").GetValue(enity, null);
var parameter = Expression.Parameter(typeof(TEntity), "x");
var property = Expression.Property(parameter, "ID");
var equal = Expression.Equal(property, Expression.Constant(value));
var lambda = Expression.Lambda<Func<TEntity, bool>>(equal, parameter);
var queryableList = new List<TEntity>(source).AsQueryable();
return queryableList.FirstOrDefault(lambda);
}
}
GetById:
public virtual TEntity GetByIdInclude(TId id, params Expression<Func<TEntity, object>>[] includes)
{
var entry = Include(includes).FirstOfDefaultIdEquals(id);
return entry;
}
Method include EntityFramework Core (look here(EF6 and EF Core)):
protected IQueryable<TEntity> Include(params Expression<Func<TEntity, object>>[] includes)
{
IIncludableQueryable<TEntity, object> query = null;
if (includes.Length > 0)
{
query = DbSet.Include(includes[0]);
}
for (int queryIndex = 1; queryIndex < includes.Length; ++queryIndex)
{
query = query.Include(includes[queryIndex]);
}
return query == null ? DbSet : (IQueryable<TEntity>)query;
}
Background
I am creating a projection from a parent/child relationship that includes a Name property of the parent and a list of the children's Ids.
Code
private class ParentChildInfo
{
public string Name { get; set; }
public List<int> ChildIds { get; set; }
}
var infos = ctx.Masters.Include(m => m.Children).Select(
m => new ParentChildInfo()
{
Name = m.Name,
ChildIds = m.Children.Where(c => c.SomeProp.StartsWith("SpecialValue"))
.Select(c => c.Id).ToList()
}).ToList();
Unfortunately that produced the error
LINQ to Entities does not recognize the method 'System.Collections.Generic.List`1[System.Int32] ToList[Int32]
That lead me to this post, which suggested (in the comments) making the following changes:
private class ParentChildInfo
{
public string Name { get; set; }
public IEnumerable<int> ChildIds { get; set; } // No longer List<int>
}
var infos = ctx.Masters.Include(m => m.Children).Select(
m => new ParentChildInfo()
{
Name = m.Name,
ChildIds = m.Children.Where(c => c.SomeProp.StartsWith("SpecialValue"))
.Select(c => c.Id) // Dropped the .ToList()
}).ToList();
I originally wanted to get lists rather than enumerables because the code that uses the result runs for several minutes, and I did not want to tie up the DbContext that long.
I use the code like this:
using (MyContext ctx = new MyContext())
{
// code from above that populates infoes
}
foreach (var info in infoes)
{
// use info.ChildIds
}
I planned to move the foreach into the using so that I can enumerate the ChildIds, but hit F5 instead and was surprised to see that the code works.
Question
Given that the DbContext is disposed at that point and ChildIds is an IEnumerable<int> rather than List<int>, why exactly can I enumerate ChildIds?
It is because the ToList() of the infos query actually executes the query. So the collection ctx.Masters is enumerated and the projections are populated. Even without the Include it would notice that Master.Children is addressed and emit the SQL join. The implementing type of IEnumerable<int> ChildIds is probably List<int>.
You did .ToList() on the query so the query was executed and all the results are materialized and the connection to the database should be closed. I assume it would not work if you did not have .ToList() since (at least in EF5) the results are being processed in streaming fashion and entities are materialized when requested (i.e. on each iteration in the loop).