EF deferred execution using SQLquery as input - entity-framework

I am trying to get an entity with EF by having an initial sql as input.
I tried the context.Entities.SQLQuery method but this returns a DBSet when I require an IQueriable.
I learned that I cannot transform DBSet to IQueryable because the first is already a result of data while the second is the container for the results of a "query" (executed yet or not). Correct me if i'm wrong :)
So I thought that when I write the following lambda I get the resulting query:
db.MyTable.Where(x => x.id == "123")
Becomes:
SELECT * FROM myTable WHERE id = '123'
With this I thought if I can set directly my query without needing to set my lambda...
Is that an option?
Or an alternative?
Thanks!

It's a bit unclear what you mean:
I am trying to get an entity with EF having an initial sql as input.
I'd interpret this, as that you have an SQL statement as input of something and you want to get an entity framework entity that would have this statement (whatever having a statement means)? Not understandable!
I learned that I cannot transform DBSet to IQueryable because the
first (the DbSet) is already a result of data while the second (the IQueryable) is the container for the results of a "query"
NOT!
Every DbSet<T> implements IQueryable<T>, meaning that if you have an object of class DbSet<t>, this object implements all functionality of IQueryable<T>. Just using this IQueryable does not execute the query. The query will only be executed once the first element of the sequence if requested.
using (var dbContext = new MyDbcontext())
{
var result = dbContext.MyItems
.Where(item => ...)
.Select(item => new
{
X = item.Property1,
Y = item.Property2,
...
};
Until here, the first element of the sequence is not asked, the query is not performed yet. No communication with the database was needed (except to create the dbContext object)
Only if you use execution functions like ToList(),Count(), First(), Max(), etc, the query is performed.
You can check this, because you get exceptoin if you do these kind of functions after the using block:
Wrong
IQueryable largeItems;
using (var dbContext = new MyDbcontext())
{
largeItems = dbContext.MyItems
.Where(item => item.Size > 1000);
// query not executed yet
}
int nrOfLargeItems = largeItems.Count();
// exception, query executed after dbContext is disposed
correct
int nrOfLargeItems;
using (var dbContext = new MyDbcontext())
{
var largeItems = dbContext.MyItems
.Where(item => item.Size > 1000);
// query not executed yet
nrOfLargeItems = largeItems.Count();
// the query is performed
}
Conclusion: users of a DbSet<T> inside a dbContext can use the DbSet<T> as if it was an IQueryable<T>, the query will not be executed until you perform any function that needs the first element of the query.
This includes complex functions like Join, GroupBy, OrderBy, etc. You can recognize these functions because MSDN add the following to the remarks section
This method is implemented by using deferred execution. The immediate return value is an object that stores all the information that is required to perform the action. The query represented by this method is not executed until the object is enumerated either by calling its GetEnumerator method directly or by using foreach.

Related

Entity Framework Core: User Defined SQL Functions

Is it possible to invoke a user-defined SQL function from the query interface in EF Core? For example, the generated SQL would look like
select * from X where dbo.fnCheckThis(X.a, X.B) = 1
In my case, this clause is in addition to other Query() method calls so FromSQL() is not an option.
I just managed this with help from this article (H/T #IvanStoev for his comment on the question).
In your DbContext class:
[DbFunction("my_user_function_name")]
public static bool SatisfiesMyUserFunction(int i)
{
throw new Exception(); // this code doesn't get executed; the call is passed through to the database function
}
Note that the function must be in the DbContext class, even though it is static.
Then create a database migration and define the user function in the script.
Usage:
var query = db.Foos.Where(f => MyDbContext.SatisfiesMyUserFunction(f.FieldValue));

How to support OData query syntax but return non-Edm models

Exposing my EF models to an API always seemed wrong. I'd like my API to return a custom entity model to the caller but use EF on the back.
So I may have PersonRestEntity and a controller for CRUD ops against that and a Person EF code-first entity behind in and map values.
When I do this, I can no longer use the following to allow ~/people?$top=10 etc. in the URL
[EnableQuery]
public IQueryable<Person> Get(ODataQueryOptions<Person> query) { ... }
Because that exposes Person which is private DB implementation.
How can I have my cake and eat it?
I found a way. The trick is not to just return the IQueryable from the controller, because you need to materialise the query first. This doesn't mean materialising the whole set into RAM, the query is still run at the database, but by explicitly applying the query and materialising the results you can return mapped entities thereafter.
Define this action, specifying the DbSet entity type:
public async Task<HttpResponseMessage> Get(ODataQueryOptions<Person> oDataQuery)
And then apply the query manually to the DbSet<Person> like so:
var queryable = oDataQuery.ApplyTo(queryableDbSet);
Then use the following to run the query and turn the results into the collection of entities you publicly expose:
var list = await queryable.ToListAsync(cancellationToken);
return list
.OfType<Person>()
.Select(p => MyEntityMapper.MapToRestEntity(p));
Then you can return the list in an HttpResponseMessage as normal.
That's it, though obviously where the property names between the entities don't match or are absent on either class, there's going to be some issues, so its probably best to ensure the properties you want to include in query options are named the same in both entities.
Else, I guess you could choose to not support filters and just allow $top and $skip and impose a default order yourself. This can be achieved like so, making sure to order the queryable first, then skip, then top. Something like:
IQueryable queryable = people
.GetQueryable(operationContext)
.OrderBy(r => r.Name);
if (oDataQuery.Skip != null)
queryable = oDataQuery.Skip.ApplyTo(queryable, new System.Web.OData.Query.ODataQuerySettings());
if (oDataQuery.Top != null)
queryable = oDataQuery.Top.ApplyTo(queryable, new System.Web.OData.Query.ODataQuerySettings());
var list = await queryable.ToListAsync(operationContext.CreateToken());
return list
.OfType<Person>()
.Select(i => this.BuildPersonEntity(i));
More information:
If you simply use the non-generic ODataQueryOptions you get
Cannot create an EDM model as the action 'Get' on controller 'People'
has a return type 'System.Net.Http.HttpResponseMessage' that does not
implement IEnumerable
And other errors occur under different circumstances.

How to mock the limitations of EntityFramework's implementation of IQueryable

I am currently writing unit tests for my repository implementation in an MVC4 application. In order to mock the data context, I started by adopting some ideas from this post, but I have now discovered some limitations that make me question whether it is even possible to properly mock IQueryable.
In particular, I have seen some situations where the tests pass but the code fails in production and I have not been able to find any way to mock the behavior that causes this failure.
For example, the following snippet is used to select Post entities that fall within a predefined list of categories:
var posts = repository.GetEntities<Post>(); // Returns IQueryable<Post>
var categories = GetCategoriesInGroup("Post"); // Returns a fixed list of type Category
var filtered = posts.Where(p => categories.Any(c => c.Name == p.Category)).ToList();
In my test environment, I have tried mocking posts using the fake DbSet implementation mentioned above, and also by creating a List of Post instances and converting it to IQueryable using the AsQueryable() extension method. Both of these approaches work under test conditions, but the code actually fails in production, with the following exception:
System.NotSupportedException : Unable to create a constant value of type 'Category'. Only primitive types or enumeration types are supported in this context.
Although LINQ issues like this are easy enough to fix, the real challenge is finding them, given that they do not reveal themselves in the test environment.
Am I being unrealistic in expecting that I can mock the behavior of Entity Framework's implementation of IQueryable?
Thanks for your ideas,
Tim.
I think it is very very hard, if impossible, to mock Entity Framework behaviour. First and foremost because it would require profound knowledge of all peculiarities and edge cases where linq-to-entites differs from linq-to-objects. As you say: the real challenge is finding them. Let me point out three main areas without claiming to be even nearly exhaustive:
Cases where Linq-to-Objects succeeds and Linq-to-Entities fails:
.Select(x => x.Property1.ToString(). LINQ to Entities does not recognize the method 'System.String ToString()' method... This applies to nearly all methods in native .Net classes and of course to own methods. Only a few .Net methods will be translated into SQL. See CLR Method to Canonical Function Mapping. As of EF 6.1, ToString is supported by the way. But only the parameterless overload.
Skip() without preceding OrderBy.
Except and Intersect: can produce monstrous queries that throw Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries.
Select(x => x.Date1 - x.Date2): DbArithmeticExpression arguments must have a numeric common type.
(your case) .Where(p => p.Category == category): Only primitive types or enumeration types are supported in this context.
Nodes.Where(n => n.ParentNodes.First().Id == 1): The method 'First' can only be used as a final query operation.
context.Nodes.Last(): LINQ to Entities does not recognize the method '...Last...'. This applies to many other IQueryable extension methods. See Supported and Unsupported LINQ Methods.
(See Slauma's comment below): .Select(x => new A { Property1 = (x.BoolProperty ? new B { BProp1 = x.Prop1, BProp2 = x.Prop2 } : new B { BProp1 = x.Prop1 }) }): The type 'B' appears in two structurally incompatible initializations within a single LINQ to Entities query... from here.
context.Entities.Cast<IEntity>(): Unable to cast the type 'Entity' to type 'IEntity'. LINQ to Entities only supports casting EDM primitive or enumeration types.
.Select(p => p.Category?.Name). Using null propagation in an expression throws CS8072 An expression tree lambda may not contain a null propagating operator. This may get fixed one day.
This question: Why does this combination of Select, Where and GroupBy cause an exception? made me aware of the fact that there are even entire query constructions that are not supported by EF, while L2O wouldn't have any trouble with them.
Cases where Linq-to-Objects fails and Linq-to-Entities succeeds:
.Select(p => p.Category.Name): when p.Category is null L2E returns null, but L2O throws Object reference not set to an instance of an object. This can't be fixed by using null propagation (see above).
Nodes.Max(n => n.ParentId.Value) with some null values for n.ParentId. L2E returns a max value, L2O throws Nullable object must have a value.
Using EntityFunctions (DbFunctions as of EF 6) or SqlFunctions.
Cases where both succeed/fail but behave differently:
Nodes.Include("ParentNodes"): L2O has no implementation of include. It will run and return nodes (if Nodes is IQueryable), but without parent nodes.
Nodes.Select(n => n.ParentNodes.Max(p => p.Id)) with some empty ParentNodes collections: both fail but with different exceptions.
Nodes.Where(n => n.Name.Contains("par")): L2O is case sensitive, L2E depends on the database collation (often not case sensitive).
node.ParentNode = parentNode: with a bidirectional relationship, in L2E this will also add the node to the nodes collection of the parent (relationship fixup). Not in L2O. (See Unit testing a two way EF relationship).
Work-around for failing null propagation: .Select(p => p.Category == null ? string.Empty : p.Category.Name): the result is the same, but the generated SQL query also contains the null check and may be harder to optimize.
Nodes.AsNoTracking().Select(n => n.ParentNode. This one is very tricky!. With AsNoTracking EF creates new ParentNode objects for each Node, so there can be duplicates. Without AsNoTracking EF reuses existing ParentNodes, because now the entity state manager and entity keys are involved. AsNoTracking() can be called in L2O, but it doesn't do anything, so there will never be a difference with or without it.
And what about mocking lazy/eager loading and the effect of context life cycle on lazy loading exceptions? Or the effect of some query constructs on performance (like constructs that trigger N+1 SQL queries). Or exceptions due to duplicate or missing entity keys? Or relationship fixup?
My opinion: nobody is going to fake that. The most alarming area is where L2O succeeds and L2E fails. Now what's the value of green unit tests? It has been said before that EF can only reliably be tested in integration tests (e.g. here) and I tend to agree.
However, that does not mean that we should forget about unit tests in projects with EF as data layer. There are ways to do it, but, I think, not without integration tests.
I have written a few Unit Tests with Entity Framework 6.1.3 using Moq and used it to override IQueryable. Note that all DbSet that should be tested needs to be marked as virtual. Example from Microsoft themselves:
Query:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
namespace TestingDemo
{
[TestClass]
public class QueryTests
{
[TestMethod]
public void GetAllBlogs_orders_by_name()
{
var data = new List<Blog>
{
new Blog { Name = "BBB" },
new Blog { Name = "ZZZ" },
new Blog { Name = "AAA" },
}.AsQueryable();
var mockSet = new Mock<DbSet<Blog>>();
mockSet.As<IQueryable<Blog>>().Setup(m => m.Provider).Returns(data.Provider);
mockSet.As<IQueryable<Blog>>().Setup(m => m.Expression).Returns(data.Expression);
mockSet.As<IQueryable<Blog>>().Setup(m => m.ElementType).Returns(data.ElementType);
mockSet.As<IQueryable<Blog>>().Setup(m => m.GetEnumerator()).Returns(0 => data.GetEnumerator());
var mockContext = new Mock<BloggingContext>();
mockContext.Setup(c => c.Blogs).Returns(mockSet.Object);
var service = new BlogService(mockContext.Object);
var blogs = service.GetAllBlogs();
Assert.AreEqual(3, blogs.Count);
Assert.AreEqual("AAA", blogs[0].Name);
Assert.AreEqual("BBB", blogs[1].Name);
Assert.AreEqual("ZZZ", blogs[2].Name);
}
}
}
Insert:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using System.Data.Entity;
namespace TestingDemo
{
[TestClass]
public class NonQueryTests
{
[TestMethod]
public void CreateBlog_saves_a_blog_via_context()
{
var mockSet = new Mock<DbSet<Blog>>();
var mockContext = new Mock<BloggingContext>();
mockContext.Setup(m => m.Blogs).Returns(mockSet.Object);
var service = new BlogService(mockContext.Object);
service.AddBlog("ADO.NET Blog", "http://blogs.msdn.com/adonet");
mockSet.Verify(m => m.Add(It.IsAny<Blog>()), Times.Once());
mockContext.Verify(m => m.SaveChanges(), Times.Once());
}
}
}
Example service:
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using System.Threading.Tasks;
namespace TestingDemo
{
public class BlogService
{
private BloggingContext _context;
public BlogService(BloggingContext context)
{
_context = context;
}
public Blog AddBlog(string name, string url)
{
var blog = _context.Blogs.Add(new Blog { Name = name, Url = url });
_context.SaveChanges();
return blog;
}
public List<Blog> GetAllBlogs()
{
var query = from b in _context.Blogs
orderby b.Name
select b;
return query.ToList();
}
public async Task<List<Blog>> GetAllBlogsAsync()
{
var query = from b in _context.Blogs
orderby b.Name
select b;
return await query.ToListAsync();
}
}
}
Source: https://learn.microsoft.com/en-us/ef/ef6/fundamentals/testing/mocking

Add index with entity framework code first (CTP5)

Is there a way to get EF CTP5 to create an index when it creates a schema?
Update: See here for how EF 6.1 handles this (as pointed out by juFo below).
You can take advantage of the new CTP5’s ExecuteSqlCommand method on Database class which allows raw SQL commands to be executed against the database.
The best place to invoke SqlCommand method for this purpose is inside a Seed method that has been overridden in a custom Initializer class. For example:
protected override void Seed(EntityMappingContext context)
{
context.Database.ExecuteSqlCommand("CREATE INDEX IX_NAME ON ...");
}
As some mentioned in the comments to Mortezas answer there is a CreateIndex/DropIndex method if you use migrations.
But if you are in "debug"/development mode and is changing the schema all the time and are recreating the database every time you can use the example mentioned in Morteza answer.
To make it a little easier, I have written a very simple extension method to make it strongly typed, as inspiration that I want to share with anyone who reads this question and maybe would like this approach aswell. Just change it to fit your needs and way of naming indexes.
You use it like this: context.Database.CreateUniqueIndex<User>(x => x.Name);
.
public static void CreateUniqueIndex<TModel>(this Database database, Expression<Func<TModel, object>> expression)
{
if (database == null)
throw new ArgumentNullException("database");
// Assumes singular table name matching the name of the Model type
var tableName = typeof(TModel).Name;
var columnName = GetLambdaExpressionName(expression.Body);
var indexName = string.Format("IX_{0}_{1}", tableName, columnName);
var createIndexSql = string.Format("CREATE UNIQUE INDEX {0} ON {1} ({2})", indexName, tableName, columnName);
database.ExecuteSqlCommand(createIndexSql);
}
public static string GetLambdaExpressionName(Expression expression)
{
MemberExpression memberExp = expression as MemberExpression;
if (memberExp == null)
{
// Check if it is an UnaryExpression and unwrap it
var unaryExp = expression as UnaryExpression;
if (unaryExp != null)
memberExp = unaryExp.Operand as MemberExpression;
}
if (memberExp == null)
throw new ArgumentException("Cannot get name from expression", "expression");
return memberExp.Member.Name;
}
Update: From version 6.1 and onwards there is an [Index] attribute available.
For more info, see http://msdn.microsoft.com/en-US/data/jj591583#Index
This feature should be available in the near-future via data annotations and the Fluent API. Microsoft have added it into their public backlog:
http://entityframework.codeplex.com/workitem/list/basic?keywords=DevDiv [Id=87553]
Until then, you'll need to use a seed method on a custom Initializer class to execute the SQL to create the unique index, and if you're using code-first migrations, create a new migration for adding the unique index, and use the CreateIndex and DropIndex methods in your Up and Down methods for the migration to create and drop the index.
Check my answer here Entity Framework Code First Fluent Api: Adding Indexes to columns this allows you to define multi column indexes by using attributes on properties.

Entity Framework IQueryable

I'm having problems querying the entity model to get additional information.
My db has a Program table with a one to many relation with an Events table. The Entity model generates the relationships just fine, but I'm unable to figure out how to query the model to get the progam object with its events.
I can do this:
var foo = from program in entities.ProgramSet
where program.StartDate > DateTime.now
orderby program.StartDate
select program;
No problems there. From what I've found on Microsofts Page (Shaping queries with Entity framework): msdn.microsoft.com/en-us/library/bb896272.aspx, if I wanted to get the child objects, I just do the following:
// Define a LINQ query with a path that returns
// orders and items for a contact.
var contacts = (from contact in context.Contact
.Include("SalesOrderHeader.SalesOrderDetail")
select contact).FirstOrDefault();
However, there is no .Include or Include that I can find on the query.
Any suggestion? I know that I can do a foreach across the results, then run a .Events.Load() on it, but doesn't that force the IQueriable result to execute the sql, instead of optomize it to run only when a .ToList() etc is called on it?
Here is some sample code from my project:
public class ProgramRepository : CT.Models.IProgramRepository
{
CTEntities db = new CTEntities();
public IQueryable<Program> FindAllPrograms()
{
return db.ProgramSet;
}
public IQueryable<Program> FindUpcomingPrograms()
{
var programs = from program in FindAllPrograms()
where program.StartDate > DateTime.Now
orderby program.StartDate
select program;
return programs;
}
With the FindUpComingPrograms I would like to have it also include the Events Data. There is a relationship between the Program and Events model. Program has a List<Events> property, that I would like to fill and return with the IQueryable method.
Thanks again!
The Include Function is part of the ObjectQuery object...
I think you are going to need to re-write your query to look something like this:
var contacts = context.Contact.Include("SalesOrderHeader.SalesOrderDetail").FirstOrDefault();
//Not sure on your dot path you might have to debug that a bit
Here is an Article that has some examples...