f#: Only parameterless constructors and initializers are supported in LINQ to Entities - entity-framework

I am trying to get Envelope's back from a query. Envelope is defined as follows.
[<CLIMutable>]
type Envelope<'T> = {
Id : Guid
StreamId: Guid
Created : DateTimeOffset
Item : 'T }
MyLibAAS.DataStore.MyLibAASDbContext is a EF DbContext written in c#. When I extend it in f# as follows, I get the error: Only parameterless constructors and initializers are supported in LINQ to Entities.
type MyLibAAS.DataStore.MyLibAASDbContext with
member this.GetEvents streamId =
query {
for event in this.Events do
where (event.StreamId = streamId)
select {
Id = event.Id;
StreamId = streamId;
Created = event.Timestamp;
Item = (JsonConvert.DeserializeObject<QuestionnaireEvent> event.Payload)
}
}
If I return the event and map it to Envelope after the fact, it works fine.
type MyLibAAS.DataStore.MyLibAASDbContext with
member this.GetEvents streamId =
query {
for event in this.Events do
where (event.StreamId = streamId)
select event
} |> Seq.map (fun event ->
{
Id = event.Id
StreamId = streamId
Created = event.Timestamp
Item = (JsonConvert.DeserializeObject<QuestionnaireEvent> event.Payload)
}
)
Why does this make a difference? The Envelope type is not even a EF type.

How F# records work
F# records get compiled into .NET classes with read-only properties and a constructor that takes values of all fields as parameters (plus a few interfaces).
For example, your record would be expressed in C# roughly as follows:
public class Envelope<T> : IComparable<Envelope<T>>, IEquatable<Envelope<T>>, ...
{
public Guid Id { get; private set; }
public Guid StreamId { get; private set; }
public DateTimeOffset Created { get; private set; }
public T Item { get; private set; }
public Envelope( Guid id, Guid streamId, DateTimeOffset created, T item ) {
this.Id = id;
this.StreamId = streamId;
this.Created = created;
this.Item = item;
}
// Plus implementations of IComparable, IEquatable, etc.
}
When you want to create an F# record, the F# compiler emits a call to this constructor, supplying values for all fields.
For example, the select part of your query would look in C# like this:
select new Envelope<QuestionnaireEvent>(
event.Id, streamId, event.Timestamp,
JsonConvert.DeserializeObject<QuestionnaireEvent>(event.Payload) )
Entity Framework limitations
It so happens that Entity Framework does not allow calling non-default constructors in queries. There is a good reason: if it did allow it, you could, in principle, construct a query like this:
from e in ...
let env = new Envelope<E>( e.Id, ... )
where env.Id > 0
select env
Entity Framework wouldn't know how to compile this query, because it doesn't know that the value of e.Id passed to the constructor becomes the value of the property env.Id. This is always true for F# records, but not for other .NET classes.
Entity Framework could, in principle, recognize that Envelope is an F# record and apply the knowledge of the connection between constructor arguments and record properties. But it doesn't. Unfortunately, the designers of Entity Framework did not think of F# as a valid use case.
(fun fact: C# anonymous types work the same way, and EF does make an exception for them)
How to fix this
In order to make this work, you need to declare Envelope as a type with default constructor. The only way to do this is to make it a class, not a record:
type Envelope<'T>() =
member val Id : Guid = Guid.Empty with get, set
member val StreamId : Guid = Guid.Empty with get, set
member val Created : DateTimeOffset = DateTimeOffset.MinValue with get, set
member val Item : 'T = Unchecked.defaultof<'T> with get, set
And then create it using property initialization syntax:
select Envelope<_>( Id = event.Id, StreamId = streamId, ... )
Why does moving the select to a Seq.map work
The Seq.map call is not part of the query expression. It does not end up as part of the IQueryable, so it does not end up compiled to SQL by Entity Framework. Instead, EF compiles just what's inside query and returns you the resulting sequence, after fetching it from SQL Server. And only after that you apply Seq.map to that sequence.
The code inside Seq.map is executed on CLR, not compiled to SQL, so it can call anything it wants, including non-default constructors.
This "fix" comes with a cost though: instead of just the fields you need, the whole Event entity gets fetched from DB and materialized. If this entity is heavy, this may have a performance impact.
Another thing to watch out for
Even if you fix the problem by making Envelope a type with default constructor (as suggested above), you'll still hit the next problem: the method JsonConvert.DeserializeObject can't be compiled to SQL, so Entity Framework will complain about it, too. The way you should do it is fetch all fields to the CLR side, then apply whatever non-SQL-compilable transformations you need.

Using LINQ to Entities, everything that happens in the query computational expression is actually executed within the database engine, which may reside on a remote server. Everything outside of it is executed in the running application on the client.
So, in your first snippet, Entity Framework refuses to execute Envelope<'T>'s constructor because, in order to do so, it would need to translate that into SQL commands for the server. This is plainly not something it can guarantee, because the constructor could potentially contain any sort of side effects and .NET framework code - it could request user input, read files from the client's hard disk, whatever.
What EF can do, in your second snippet, is sending its own POCO event objects back to the client, which is then tasked with Seq.mapping them to your fancy Envelopes, which it can do because it's running on your client machine with access to the full .NET framework.
Addendum: So why are parameterless constructors ok? What if I were to call MsgBox() in a parameterless constructor? I think that parameterless constructors work by having the client construct the objects (without knowing the query results), sending them to the server in serialised form, and having the server just fill the object's properties with the query results.
I haven't actually tested that. But F# record types have no parameterless constructors anyway, so the point is moot in your case.

Related

How to mock the limitations of EntityFramework's implementation of IQueryable

I am currently writing unit tests for my repository implementation in an MVC4 application. In order to mock the data context, I started by adopting some ideas from this post, but I have now discovered some limitations that make me question whether it is even possible to properly mock IQueryable.
In particular, I have seen some situations where the tests pass but the code fails in production and I have not been able to find any way to mock the behavior that causes this failure.
For example, the following snippet is used to select Post entities that fall within a predefined list of categories:
var posts = repository.GetEntities<Post>(); // Returns IQueryable<Post>
var categories = GetCategoriesInGroup("Post"); // Returns a fixed list of type Category
var filtered = posts.Where(p => categories.Any(c => c.Name == p.Category)).ToList();
In my test environment, I have tried mocking posts using the fake DbSet implementation mentioned above, and also by creating a List of Post instances and converting it to IQueryable using the AsQueryable() extension method. Both of these approaches work under test conditions, but the code actually fails in production, with the following exception:
System.NotSupportedException : Unable to create a constant value of type 'Category'. Only primitive types or enumeration types are supported in this context.
Although LINQ issues like this are easy enough to fix, the real challenge is finding them, given that they do not reveal themselves in the test environment.
Am I being unrealistic in expecting that I can mock the behavior of Entity Framework's implementation of IQueryable?
Thanks for your ideas,
Tim.
I think it is very very hard, if impossible, to mock Entity Framework behaviour. First and foremost because it would require profound knowledge of all peculiarities and edge cases where linq-to-entites differs from linq-to-objects. As you say: the real challenge is finding them. Let me point out three main areas without claiming to be even nearly exhaustive:
Cases where Linq-to-Objects succeeds and Linq-to-Entities fails:
.Select(x => x.Property1.ToString(). LINQ to Entities does not recognize the method 'System.String ToString()' method... This applies to nearly all methods in native .Net classes and of course to own methods. Only a few .Net methods will be translated into SQL. See CLR Method to Canonical Function Mapping. As of EF 6.1, ToString is supported by the way. But only the parameterless overload.
Skip() without preceding OrderBy.
Except and Intersect: can produce monstrous queries that throw Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries.
Select(x => x.Date1 - x.Date2): DbArithmeticExpression arguments must have a numeric common type.
(your case) .Where(p => p.Category == category): Only primitive types or enumeration types are supported in this context.
Nodes.Where(n => n.ParentNodes.First().Id == 1): The method 'First' can only be used as a final query operation.
context.Nodes.Last(): LINQ to Entities does not recognize the method '...Last...'. This applies to many other IQueryable extension methods. See Supported and Unsupported LINQ Methods.
(See Slauma's comment below): .Select(x => new A { Property1 = (x.BoolProperty ? new B { BProp1 = x.Prop1, BProp2 = x.Prop2 } : new B { BProp1 = x.Prop1 }) }): The type 'B' appears in two structurally incompatible initializations within a single LINQ to Entities query... from here.
context.Entities.Cast<IEntity>(): Unable to cast the type 'Entity' to type 'IEntity'. LINQ to Entities only supports casting EDM primitive or enumeration types.
.Select(p => p.Category?.Name). Using null propagation in an expression throws CS8072 An expression tree lambda may not contain a null propagating operator. This may get fixed one day.
This question: Why does this combination of Select, Where and GroupBy cause an exception? made me aware of the fact that there are even entire query constructions that are not supported by EF, while L2O wouldn't have any trouble with them.
Cases where Linq-to-Objects fails and Linq-to-Entities succeeds:
.Select(p => p.Category.Name): when p.Category is null L2E returns null, but L2O throws Object reference not set to an instance of an object. This can't be fixed by using null propagation (see above).
Nodes.Max(n => n.ParentId.Value) with some null values for n.ParentId. L2E returns a max value, L2O throws Nullable object must have a value.
Using EntityFunctions (DbFunctions as of EF 6) or SqlFunctions.
Cases where both succeed/fail but behave differently:
Nodes.Include("ParentNodes"): L2O has no implementation of include. It will run and return nodes (if Nodes is IQueryable), but without parent nodes.
Nodes.Select(n => n.ParentNodes.Max(p => p.Id)) with some empty ParentNodes collections: both fail but with different exceptions.
Nodes.Where(n => n.Name.Contains("par")): L2O is case sensitive, L2E depends on the database collation (often not case sensitive).
node.ParentNode = parentNode: with a bidirectional relationship, in L2E this will also add the node to the nodes collection of the parent (relationship fixup). Not in L2O. (See Unit testing a two way EF relationship).
Work-around for failing null propagation: .Select(p => p.Category == null ? string.Empty : p.Category.Name): the result is the same, but the generated SQL query also contains the null check and may be harder to optimize.
Nodes.AsNoTracking().Select(n => n.ParentNode. This one is very tricky!. With AsNoTracking EF creates new ParentNode objects for each Node, so there can be duplicates. Without AsNoTracking EF reuses existing ParentNodes, because now the entity state manager and entity keys are involved. AsNoTracking() can be called in L2O, but it doesn't do anything, so there will never be a difference with or without it.
And what about mocking lazy/eager loading and the effect of context life cycle on lazy loading exceptions? Or the effect of some query constructs on performance (like constructs that trigger N+1 SQL queries). Or exceptions due to duplicate or missing entity keys? Or relationship fixup?
My opinion: nobody is going to fake that. The most alarming area is where L2O succeeds and L2E fails. Now what's the value of green unit tests? It has been said before that EF can only reliably be tested in integration tests (e.g. here) and I tend to agree.
However, that does not mean that we should forget about unit tests in projects with EF as data layer. There are ways to do it, but, I think, not without integration tests.
I have written a few Unit Tests with Entity Framework 6.1.3 using Moq and used it to override IQueryable. Note that all DbSet that should be tested needs to be marked as virtual. Example from Microsoft themselves:
Query:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
namespace TestingDemo
{
[TestClass]
public class QueryTests
{
[TestMethod]
public void GetAllBlogs_orders_by_name()
{
var data = new List<Blog>
{
new Blog { Name = "BBB" },
new Blog { Name = "ZZZ" },
new Blog { Name = "AAA" },
}.AsQueryable();
var mockSet = new Mock<DbSet<Blog>>();
mockSet.As<IQueryable<Blog>>().Setup(m => m.Provider).Returns(data.Provider);
mockSet.As<IQueryable<Blog>>().Setup(m => m.Expression).Returns(data.Expression);
mockSet.As<IQueryable<Blog>>().Setup(m => m.ElementType).Returns(data.ElementType);
mockSet.As<IQueryable<Blog>>().Setup(m => m.GetEnumerator()).Returns(0 => data.GetEnumerator());
var mockContext = new Mock<BloggingContext>();
mockContext.Setup(c => c.Blogs).Returns(mockSet.Object);
var service = new BlogService(mockContext.Object);
var blogs = service.GetAllBlogs();
Assert.AreEqual(3, blogs.Count);
Assert.AreEqual("AAA", blogs[0].Name);
Assert.AreEqual("BBB", blogs[1].Name);
Assert.AreEqual("ZZZ", blogs[2].Name);
}
}
}
Insert:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using System.Data.Entity;
namespace TestingDemo
{
[TestClass]
public class NonQueryTests
{
[TestMethod]
public void CreateBlog_saves_a_blog_via_context()
{
var mockSet = new Mock<DbSet<Blog>>();
var mockContext = new Mock<BloggingContext>();
mockContext.Setup(m => m.Blogs).Returns(mockSet.Object);
var service = new BlogService(mockContext.Object);
service.AddBlog("ADO.NET Blog", "http://blogs.msdn.com/adonet");
mockSet.Verify(m => m.Add(It.IsAny<Blog>()), Times.Once());
mockContext.Verify(m => m.SaveChanges(), Times.Once());
}
}
}
Example service:
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using System.Threading.Tasks;
namespace TestingDemo
{
public class BlogService
{
private BloggingContext _context;
public BlogService(BloggingContext context)
{
_context = context;
}
public Blog AddBlog(string name, string url)
{
var blog = _context.Blogs.Add(new Blog { Name = name, Url = url });
_context.SaveChanges();
return blog;
}
public List<Blog> GetAllBlogs()
{
var query = from b in _context.Blogs
orderby b.Name
select b;
return query.ToList();
}
public async Task<List<Blog>> GetAllBlogsAsync()
{
var query = from b in _context.Blogs
orderby b.Name
select b;
return await query.ToListAsync();
}
}
}
Source: https://learn.microsoft.com/en-us/ef/ef6/fundamentals/testing/mocking

Entity Framework doesn't query derived classes - Error in DbOfTypeExpression

I have a base class and two derived classes.
Each of the derived classes implements the same type as a property - the only difference is the property name.
Sadly I don't have much influence on the class design -> they have been generated from a wsdl file.
I then have a property on the BaseType to encapsulate the common property. The plan was to use this property in my web views etc.
I have used the famous "Fruit-Example" to demonstrate the problem:
public class FruitBase
{
public virtual int ID { get; set; }
//
// The plan is to use this property in mvc view
//
[NotMapped]
public virtual FruitnessFactor Fruitness
{
get
{
if (this.GetType().BaseType == typeof(Apple))
return ((Apple)this).AppleFruitness;
else if (this.GetType().BaseType == typeof(Orange))
return ((Orange)this).OrangeFruitness;
else
return null;
}
}
}
public class FruitnessFactor { }
In my MVC controller, the following query works absolutely fine:
return View(context.FruitEntities
.OfType<Apple>().Include(a =>a.AppleFruitness)
.ToList());
But this one doesn't:
return View(context.FruitEntities
.OfType<Apple>().Include(a =>a.AppleFruitness)
.OfType<Orange>().Include(o => o.OrangeFruitness)
.ToList());
The error message I get is:
DbOfTypeExpression requires an expression argument with a polymorphic result type that is compatible with the type argument.
I am using EF 5.0 RC and the Code First approach.
Any help is much appreciated!
As far as I can tell you can't apply Include on multiple subtypes in a single database query. You can query one type (OfType<Apple>().Include(a => a.AppelFruitness)) and the same for another subtype. The problem is that you can't concat the results in the same query because the result collections have different generic types (apples and oranges).
One option would be to run two queries and copy the result collection into a new collection of the base type - as you already indicated in the comment section under your question.
The other option (which would only need a single query) is a projection. You would have to define a projection type (you could also project into an anonymous type)...
public class FruitViewModel
{
public FruitBase Fruit { get; set; }
public FruitnessFactor Factor { get; set; }
}
...and then can use the query:
List<FruitViewModel> fruitViewModels = context.FruitEntities
.OfType<Apple>()
.Select(a => new FruitViewModel
{
Fruit = a,
Factor = a.AppleFruitness
})
.Concat(context.FruitEntities
.OfType<Orange>()
.Select(o => new FruitViewModel
{
Fruit = o,
Factor = o.OrangeFruitness
}))
.ToList();
If you don't disable change tracking (by using AsNoTracking) the navigation properties get populated automatically when the entities get attached to the context ("Relationship fixup") which means that you can extract the fruits from the viewModel collection...
IEnumerable<FruitBase> fruits = fruitViewModels.Select(fv => fv.Fruit);
...and you'll get the fruits including the FruitnessFactor properties.
This code is pretty awkward but a direct approach without using a projection has been asked for several times without success:
bottleneck using entity framework inheritance
Entity Framework - Eager loading of subclass related objects
How do I deeply eager load an entity with a reference to an instance of a persistent base type (Entity Framework 4)

Can I write a generic repository with the db first scenario to dynamically get me an entity by id?

I'd like to know how to create a method that will allow me to generically do this...
public class Repo<T> : IGenericRepo<T> where T : class
{
protected PteDotNetEntities db;
public T Get(int id)
{
//how do I dynamically get to the correct entity object and select it by
//id?
}
}
Yes you can. If you know that all your entities will have simple primary key property of type int and name Id you can do simply this:
public interface IEntity
{
int Id { get; }
}
All your entities must implement this interface. Next you can simply do:
public class Repo<T> : IGenericRepo<T> where T : class, IEntity
{
protected PteDotNetEntities db;
public T Get(int id)
{
return db.CreateObjectSet<T>().FirstOrDefault(e => e.Id == id);
}
}
This is the simplest possible solution. There are better solutions using GetObjectByKey but they are more complex. The difference between FirstOrDefault and GetObjectByKey is repeatable execution. FirstOrDefault always executes DB query whereas GetObjectByKey first checks if the entity with the same key was already loaded / attached to the context and returns it without querying the database. As reference for version using GetObjectByKey you can check similar questions:
Entity Framework Simple Generic GetByID but has differents PK Name
Generic GetById for complex PK
You can simplify those examples if you know the name of the key property upfront (forced by the interface).
In case of using code first / DbContext API you can also check this question:
Generic repository EF4 CTP5 getById

Limiting EF result set permanently by overriding ObjectQuery ESQL

Does anyone have any idea how to limit result set of EntityFramework permanently? I'm speaking about something like this Conditional Mapping. This is exactly what I want to achieve with one exception: I want to do this programmatically. That's because condition value will be passed to EF only on context creation. Beside I don't want this column to disappear from mapping.
I know how to achieve this with EF2.0 and reflection. I was using CreateQuery() method to generate my own ObjectQuery. CreateQuery() allows to inject my own ESQL query with additional condition e.g. WHERE TABLE.ClientID == value.
Problem with EF40 is that there is no more ObjectQuery but only ObjectSet and CreateQuery() is not used. I have no idea how to inject my own ESQL query.
The reason why I want to limit result sets is that I want to separate clients data from each other. This separation should be done automatically inside context so that programmers will not have to add condition .Where(x => x.ClientID == 5) to each individual query.
Maybe my approach is completely bad — but I don't know any alternative.
You don't need reflection for this. You can simply use class inherited from ObjectContext or create custom implementation of UnitOfWork and Repositories which will wrap this functionality in better way (upper layer has access only to UnitOfWork and Repositories which do not expose EF context).
Simple example of object context:
public class CustomContext : ObjectContext
{
private ObjectSet<MyObject> _myObjectsSet;
private int _clientId;
public CustomContext(string connectionString, int clientId)
: base(connectionString)
{
_myObjectSet = CreateObjectSet<MyObject>();
_clientId = clientId;
}
public IQueryable<MyObject> MyObjectQuery
{
get
{
return _myObjectsSet.Where(o => o.ClientId == _clientId);
}
}
}

Exposing enum typed properties for entity framework entities fields in ADO.NET Data Service client context

I have an entity with fields that are typed int
and i want to exposed those fields as properties which get and recieve enum type values for working strongly typed.
so i've created a new partial class for the entity and added the two properties.
when i try to create a new instance of the TestEntity and add it to the context ,
and call save changes i get the following exception:
An error occurred while processing this request.
at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.HandleBatchResponse()
at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.EndRequest()
at System.Data.Services.Client.DataServiceContext.SaveChanges(SaveChangesOptions options)
at System.Data.Services.Client.DataServiceContext.SaveChanges()
the inner exception is:
System.InvalidOperationException: The type 'enum1' has no settable properties.
at System.Data.Services.Client.ClientType..ctor(Type type, String typeName, Boolean skipSettableCheck)
at System.Data.Services.Client.ClientType.Create(Type type, Boolean expectModelType)
at System.Data.Services.Client.DataServiceContext.WriteContentProperties(XmlWriter writer, ClientType type, Object resource)
at System.Data.Services.Client.DataServiceContext.CreateRequestData(ResourceBox box, Boolean newline)
at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.CreateChangeData(Int32 index, Boolean newline)
at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.BeginNextChange(Boolean replaceOnUpdate)
so i figured it tries to reflect the enum properties as classes properties.
how can i make the context ignore those properties when it tries to reflect on them.
i am using VS 2008 team suite sp1 , SQL Server 2008 , .Net 3.5 Sp1.
Help.
the partial class code:
public partial class TestEntity
{
public enum1 Field1
{
get
{
return (enum1)field1;
}
set
{
field1 = (Int16)value;
}
}
public enum2 Field2
{
get
{
return (enum2)field2;
}
set
{
field2 = (Int16)value;
}
}
}
I don't think that you can. In ADO.Net Data Services, you can't have enums on your proxy objects that get sent up to the server. Try changing your object around to use an int (or short) instead.