I am currently writing unit tests for my repository implementation in an MVC4 application. In order to mock the data context, I started by adopting some ideas from this post, but I have now discovered some limitations that make me question whether it is even possible to properly mock IQueryable.
In particular, I have seen some situations where the tests pass but the code fails in production and I have not been able to find any way to mock the behavior that causes this failure.
For example, the following snippet is used to select Post entities that fall within a predefined list of categories:
var posts = repository.GetEntities<Post>(); // Returns IQueryable<Post>
var categories = GetCategoriesInGroup("Post"); // Returns a fixed list of type Category
var filtered = posts.Where(p => categories.Any(c => c.Name == p.Category)).ToList();
In my test environment, I have tried mocking posts using the fake DbSet implementation mentioned above, and also by creating a List of Post instances and converting it to IQueryable using the AsQueryable() extension method. Both of these approaches work under test conditions, but the code actually fails in production, with the following exception:
System.NotSupportedException : Unable to create a constant value of type 'Category'. Only primitive types or enumeration types are supported in this context.
Although LINQ issues like this are easy enough to fix, the real challenge is finding them, given that they do not reveal themselves in the test environment.
Am I being unrealistic in expecting that I can mock the behavior of Entity Framework's implementation of IQueryable?
Thanks for your ideas,
Tim.
I think it is very very hard, if impossible, to mock Entity Framework behaviour. First and foremost because it would require profound knowledge of all peculiarities and edge cases where linq-to-entites differs from linq-to-objects. As you say: the real challenge is finding them. Let me point out three main areas without claiming to be even nearly exhaustive:
Cases where Linq-to-Objects succeeds and Linq-to-Entities fails:
.Select(x => x.Property1.ToString(). LINQ to Entities does not recognize the method 'System.String ToString()' method... This applies to nearly all methods in native .Net classes and of course to own methods. Only a few .Net methods will be translated into SQL. See CLR Method to Canonical Function Mapping. As of EF 6.1, ToString is supported by the way. But only the parameterless overload.
Skip() without preceding OrderBy.
Except and Intersect: can produce monstrous queries that throw Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries.
Select(x => x.Date1 - x.Date2): DbArithmeticExpression arguments must have a numeric common type.
(your case) .Where(p => p.Category == category): Only primitive types or enumeration types are supported in this context.
Nodes.Where(n => n.ParentNodes.First().Id == 1): The method 'First' can only be used as a final query operation.
context.Nodes.Last(): LINQ to Entities does not recognize the method '...Last...'. This applies to many other IQueryable extension methods. See Supported and Unsupported LINQ Methods.
(See Slauma's comment below): .Select(x => new A { Property1 = (x.BoolProperty ? new B { BProp1 = x.Prop1, BProp2 = x.Prop2 } : new B { BProp1 = x.Prop1 }) }): The type 'B' appears in two structurally incompatible initializations within a single LINQ to Entities query... from here.
context.Entities.Cast<IEntity>(): Unable to cast the type 'Entity' to type 'IEntity'. LINQ to Entities only supports casting EDM primitive or enumeration types.
.Select(p => p.Category?.Name). Using null propagation in an expression throws CS8072 An expression tree lambda may not contain a null propagating operator. This may get fixed one day.
This question: Why does this combination of Select, Where and GroupBy cause an exception? made me aware of the fact that there are even entire query constructions that are not supported by EF, while L2O wouldn't have any trouble with them.
Cases where Linq-to-Objects fails and Linq-to-Entities succeeds:
.Select(p => p.Category.Name): when p.Category is null L2E returns null, but L2O throws Object reference not set to an instance of an object. This can't be fixed by using null propagation (see above).
Nodes.Max(n => n.ParentId.Value) with some null values for n.ParentId. L2E returns a max value, L2O throws Nullable object must have a value.
Using EntityFunctions (DbFunctions as of EF 6) or SqlFunctions.
Cases where both succeed/fail but behave differently:
Nodes.Include("ParentNodes"): L2O has no implementation of include. It will run and return nodes (if Nodes is IQueryable), but without parent nodes.
Nodes.Select(n => n.ParentNodes.Max(p => p.Id)) with some empty ParentNodes collections: both fail but with different exceptions.
Nodes.Where(n => n.Name.Contains("par")): L2O is case sensitive, L2E depends on the database collation (often not case sensitive).
node.ParentNode = parentNode: with a bidirectional relationship, in L2E this will also add the node to the nodes collection of the parent (relationship fixup). Not in L2O. (See Unit testing a two way EF relationship).
Work-around for failing null propagation: .Select(p => p.Category == null ? string.Empty : p.Category.Name): the result is the same, but the generated SQL query also contains the null check and may be harder to optimize.
Nodes.AsNoTracking().Select(n => n.ParentNode. This one is very tricky!. With AsNoTracking EF creates new ParentNode objects for each Node, so there can be duplicates. Without AsNoTracking EF reuses existing ParentNodes, because now the entity state manager and entity keys are involved. AsNoTracking() can be called in L2O, but it doesn't do anything, so there will never be a difference with or without it.
And what about mocking lazy/eager loading and the effect of context life cycle on lazy loading exceptions? Or the effect of some query constructs on performance (like constructs that trigger N+1 SQL queries). Or exceptions due to duplicate or missing entity keys? Or relationship fixup?
My opinion: nobody is going to fake that. The most alarming area is where L2O succeeds and L2E fails. Now what's the value of green unit tests? It has been said before that EF can only reliably be tested in integration tests (e.g. here) and I tend to agree.
However, that does not mean that we should forget about unit tests in projects with EF as data layer. There are ways to do it, but, I think, not without integration tests.
I have written a few Unit Tests with Entity Framework 6.1.3 using Moq and used it to override IQueryable. Note that all DbSet that should be tested needs to be marked as virtual. Example from Microsoft themselves:
Query:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
namespace TestingDemo
{
[TestClass]
public class QueryTests
{
[TestMethod]
public void GetAllBlogs_orders_by_name()
{
var data = new List<Blog>
{
new Blog { Name = "BBB" },
new Blog { Name = "ZZZ" },
new Blog { Name = "AAA" },
}.AsQueryable();
var mockSet = new Mock<DbSet<Blog>>();
mockSet.As<IQueryable<Blog>>().Setup(m => m.Provider).Returns(data.Provider);
mockSet.As<IQueryable<Blog>>().Setup(m => m.Expression).Returns(data.Expression);
mockSet.As<IQueryable<Blog>>().Setup(m => m.ElementType).Returns(data.ElementType);
mockSet.As<IQueryable<Blog>>().Setup(m => m.GetEnumerator()).Returns(0 => data.GetEnumerator());
var mockContext = new Mock<BloggingContext>();
mockContext.Setup(c => c.Blogs).Returns(mockSet.Object);
var service = new BlogService(mockContext.Object);
var blogs = service.GetAllBlogs();
Assert.AreEqual(3, blogs.Count);
Assert.AreEqual("AAA", blogs[0].Name);
Assert.AreEqual("BBB", blogs[1].Name);
Assert.AreEqual("ZZZ", blogs[2].Name);
}
}
}
Insert:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using System.Data.Entity;
namespace TestingDemo
{
[TestClass]
public class NonQueryTests
{
[TestMethod]
public void CreateBlog_saves_a_blog_via_context()
{
var mockSet = new Mock<DbSet<Blog>>();
var mockContext = new Mock<BloggingContext>();
mockContext.Setup(m => m.Blogs).Returns(mockSet.Object);
var service = new BlogService(mockContext.Object);
service.AddBlog("ADO.NET Blog", "http://blogs.msdn.com/adonet");
mockSet.Verify(m => m.Add(It.IsAny<Blog>()), Times.Once());
mockContext.Verify(m => m.SaveChanges(), Times.Once());
}
}
}
Example service:
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using System.Threading.Tasks;
namespace TestingDemo
{
public class BlogService
{
private BloggingContext _context;
public BlogService(BloggingContext context)
{
_context = context;
}
public Blog AddBlog(string name, string url)
{
var blog = _context.Blogs.Add(new Blog { Name = name, Url = url });
_context.SaveChanges();
return blog;
}
public List<Blog> GetAllBlogs()
{
var query = from b in _context.Blogs
orderby b.Name
select b;
return query.ToList();
}
public async Task<List<Blog>> GetAllBlogsAsync()
{
var query = from b in _context.Blogs
orderby b.Name
select b;
return await query.ToListAsync();
}
}
}
Source: https://learn.microsoft.com/en-us/ef/ef6/fundamentals/testing/mocking
I have a base class and two derived classes.
Each of the derived classes implements the same type as a property - the only difference is the property name.
Sadly I don't have much influence on the class design -> they have been generated from a wsdl file.
I then have a property on the BaseType to encapsulate the common property. The plan was to use this property in my web views etc.
I have used the famous "Fruit-Example" to demonstrate the problem:
public class FruitBase
{
public virtual int ID { get; set; }
//
// The plan is to use this property in mvc view
//
[NotMapped]
public virtual FruitnessFactor Fruitness
{
get
{
if (this.GetType().BaseType == typeof(Apple))
return ((Apple)this).AppleFruitness;
else if (this.GetType().BaseType == typeof(Orange))
return ((Orange)this).OrangeFruitness;
else
return null;
}
}
}
public class FruitnessFactor { }
In my MVC controller, the following query works absolutely fine:
return View(context.FruitEntities
.OfType<Apple>().Include(a =>a.AppleFruitness)
.ToList());
But this one doesn't:
return View(context.FruitEntities
.OfType<Apple>().Include(a =>a.AppleFruitness)
.OfType<Orange>().Include(o => o.OrangeFruitness)
.ToList());
The error message I get is:
DbOfTypeExpression requires an expression argument with a polymorphic result type that is compatible with the type argument.
I am using EF 5.0 RC and the Code First approach.
Any help is much appreciated!
As far as I can tell you can't apply Include on multiple subtypes in a single database query. You can query one type (OfType<Apple>().Include(a => a.AppelFruitness)) and the same for another subtype. The problem is that you can't concat the results in the same query because the result collections have different generic types (apples and oranges).
One option would be to run two queries and copy the result collection into a new collection of the base type - as you already indicated in the comment section under your question.
The other option (which would only need a single query) is a projection. You would have to define a projection type (you could also project into an anonymous type)...
public class FruitViewModel
{
public FruitBase Fruit { get; set; }
public FruitnessFactor Factor { get; set; }
}
...and then can use the query:
List<FruitViewModel> fruitViewModels = context.FruitEntities
.OfType<Apple>()
.Select(a => new FruitViewModel
{
Fruit = a,
Factor = a.AppleFruitness
})
.Concat(context.FruitEntities
.OfType<Orange>()
.Select(o => new FruitViewModel
{
Fruit = o,
Factor = o.OrangeFruitness
}))
.ToList();
If you don't disable change tracking (by using AsNoTracking) the navigation properties get populated automatically when the entities get attached to the context ("Relationship fixup") which means that you can extract the fruits from the viewModel collection...
IEnumerable<FruitBase> fruits = fruitViewModels.Select(fv => fv.Fruit);
...and you'll get the fruits including the FruitnessFactor properties.
This code is pretty awkward but a direct approach without using a projection has been asked for several times without success:
bottleneck using entity framework inheritance
Entity Framework - Eager loading of subclass related objects
How do I deeply eager load an entity with a reference to an instance of a persistent base type (Entity Framework 4)
I'd like to know how to create a method that will allow me to generically do this...
public class Repo<T> : IGenericRepo<T> where T : class
{
protected PteDotNetEntities db;
public T Get(int id)
{
//how do I dynamically get to the correct entity object and select it by
//id?
}
}
Yes you can. If you know that all your entities will have simple primary key property of type int and name Id you can do simply this:
public interface IEntity
{
int Id { get; }
}
All your entities must implement this interface. Next you can simply do:
public class Repo<T> : IGenericRepo<T> where T : class, IEntity
{
protected PteDotNetEntities db;
public T Get(int id)
{
return db.CreateObjectSet<T>().FirstOrDefault(e => e.Id == id);
}
}
This is the simplest possible solution. There are better solutions using GetObjectByKey but they are more complex. The difference between FirstOrDefault and GetObjectByKey is repeatable execution. FirstOrDefault always executes DB query whereas GetObjectByKey first checks if the entity with the same key was already loaded / attached to the context and returns it without querying the database. As reference for version using GetObjectByKey you can check similar questions:
Entity Framework Simple Generic GetByID but has differents PK Name
Generic GetById for complex PK
You can simplify those examples if you know the name of the key property upfront (forced by the interface).
In case of using code first / DbContext API you can also check this question:
Generic repository EF4 CTP5 getById
Does anyone have any idea how to limit result set of EntityFramework permanently? I'm speaking about something like this Conditional Mapping. This is exactly what I want to achieve with one exception: I want to do this programmatically. That's because condition value will be passed to EF only on context creation. Beside I don't want this column to disappear from mapping.
I know how to achieve this with EF2.0 and reflection. I was using CreateQuery() method to generate my own ObjectQuery. CreateQuery() allows to inject my own ESQL query with additional condition e.g. WHERE TABLE.ClientID == value.
Problem with EF40 is that there is no more ObjectQuery but only ObjectSet and CreateQuery() is not used. I have no idea how to inject my own ESQL query.
The reason why I want to limit result sets is that I want to separate clients data from each other. This separation should be done automatically inside context so that programmers will not have to add condition .Where(x => x.ClientID == 5) to each individual query.
Maybe my approach is completely bad — but I don't know any alternative.
You don't need reflection for this. You can simply use class inherited from ObjectContext or create custom implementation of UnitOfWork and Repositories which will wrap this functionality in better way (upper layer has access only to UnitOfWork and Repositories which do not expose EF context).
Simple example of object context:
public class CustomContext : ObjectContext
{
private ObjectSet<MyObject> _myObjectsSet;
private int _clientId;
public CustomContext(string connectionString, int clientId)
: base(connectionString)
{
_myObjectSet = CreateObjectSet<MyObject>();
_clientId = clientId;
}
public IQueryable<MyObject> MyObjectQuery
{
get
{
return _myObjectsSet.Where(o => o.ClientId == _clientId);
}
}
}
I have an entity with fields that are typed int
and i want to exposed those fields as properties which get and recieve enum type values for working strongly typed.
so i've created a new partial class for the entity and added the two properties.
when i try to create a new instance of the TestEntity and add it to the context ,
and call save changes i get the following exception:
An error occurred while processing this request.
at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.HandleBatchResponse()
at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.EndRequest()
at System.Data.Services.Client.DataServiceContext.SaveChanges(SaveChangesOptions options)
at System.Data.Services.Client.DataServiceContext.SaveChanges()
the inner exception is:
System.InvalidOperationException: The type 'enum1' has no settable properties.
at System.Data.Services.Client.ClientType..ctor(Type type, String typeName, Boolean skipSettableCheck)
at System.Data.Services.Client.ClientType.Create(Type type, Boolean expectModelType)
at System.Data.Services.Client.DataServiceContext.WriteContentProperties(XmlWriter writer, ClientType type, Object resource)
at System.Data.Services.Client.DataServiceContext.CreateRequestData(ResourceBox box, Boolean newline)
at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.CreateChangeData(Int32 index, Boolean newline)
at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.BeginNextChange(Boolean replaceOnUpdate)
so i figured it tries to reflect the enum properties as classes properties.
how can i make the context ignore those properties when it tries to reflect on them.
i am using VS 2008 team suite sp1 , SQL Server 2008 , .Net 3.5 Sp1.
Help.
the partial class code:
public partial class TestEntity
{
public enum1 Field1
{
get
{
return (enum1)field1;
}
set
{
field1 = (Int16)value;
}
}
public enum2 Field2
{
get
{
return (enum2)field2;
}
set
{
field2 = (Int16)value;
}
}
}
I don't think that you can. In ADO.Net Data Services, you can't have enums on your proxy objects that get sent up to the server. Try changing your object around to use an int (or short) instead.