I have never yet used .AsEnumerable() in an EntityFramework query.
See the below example and tell me why they use .AsEnumerable() before Select ?
Could they not just use Select directly?
Please tell me the reason for the usage of .AsEnumerable() here in below query.
Why did they use .ToArray() instead of .Tolist() ?
private IEnumerable<AutoCompleteData> GetAutoCompleteData(string searchTerm)
{
using (var context = new AdventureWorksEntities())
{
var results = context.Products
.Include("ProductSubcategory")
.Where(p => p.Name.Contains(searchTerm)
&& p.DiscontinuedDate == null)
.AsEnumerable()
.Select(p => new AutoCompleteData
{
Id = p.ProductID,
Text = BuildAutoCompleteText(p)
})
.ToArray();
return results;
}
}
The difference between AsEnumerable and AsQueryable is that the enumerable contains all information to create an enumerator. Once you've got the enumerator you can ask for the first element, and if there is one, you can get the next one.
The Queryable does not hold the information to create the enumerator. It holds an Expression and a Provider. The Provider knows which process must execute the Expression and which language this process uses. Quite often the other process is a database management system, and the language is SQL.
The result of a Queryable.Select(...) is still an IQueryable, meaning that the query is not performed yet. The Select function only changed the Expression.
Only if you ask for the Enumerator, either explicitly by calling GetEnumerator(), or implicitly by calling foreach, or one of the non-deferred execution functions like ToList(), ToDictionary(), FirstOrDefault(), Sum(), the Provider will translate the expression into the format that the execution process understands and execute the query. Once the data is transported to the local process the enumerator is created.
Alas, sometimes you want to call your own functions in your query. SQL does not know these functions, and thus the Provider can't translate such Expressions into SQL. In fact, the provider of DbContext does not even know all Linq functions. See supported and unsupported Linq methods
That is the moment when you use AsEnumerable(). If you ask for the Enumerator (in your foreach for example), the Provider will translate the Expression until AsEnumerable; send it to the execution process and transport all data to local process. After that, the query will be AsEnumerable: the rest of the LINQ will be performed in local memory, and thus your local functions can be called.
You could of course use ToList() to fetch all data to local memory and continue your linq after that. But that would be a waste if you'd only want the first element, or every other one.
This brings me to the final remark: the transport of the data from the DBMS to your local memory is one of the slower parts. Try to limit this transport to only the data you'll actually use.
For example: if you have a one-to-many relation between a Teacher and his Students, don't fetch the Teacher and his Students, because you'll transport Student.TeacherId many times, and they will all have the same value as Teacher.Id. Instead, only select the data you really want to use
Not all Select projections, Where predicates, and Aggregations can be translated from C# Expressions into native database queries - in your case, the full LINQ expression attempts to construct a AutoCompleteData class with using a custom function BuildAutoCompleteText to set one of its properties - this cannot be trivially converted into native database code like SQL.
In your case, AsEnumerable serves to terminate the work which will be done in SQL before this will be executed in SQL.
i.e.
.Include("ProductSubcategory")
.Where(p => p.Name.Contains(searchTerm)
&& p.DiscontinuedDate == null)
will be executed in SQL, roughly as a JOIN to ProductSubcategory, and a WHERE predicate translated from your Products such as:
Product.Name LIKE '%' + #SearchTerm + '%' AND Product.DiscontinuedDate IS NULL
All work subsequent to the AsEnumerable (i.e. the projection of the results to AutoCompleteData objects) will be done in-memory with LINQ to objects.
ToArray and ToList will both execute (materialize) the result, but into different data structures. In your example, neither materialisation is required - since the return type is IEnumerable<AutoCompleteData> - the caller of the function might execute .Any() or First() which would render full materialisation wasteful - I would recommend you remove .ToArray() altogether - since the using statement controls the SQL lifespan is protected by the AsEnumerable() materialization, there is no issue with connection lifespans here.
tell me the intention of usage of .AsEnumerable() here in below query?
In this particular example AsEnumerable() was used to bring the data back to the client, because EF has no idea how to map BuildAutoCompleteText() to SQL query.
they could use select directly.....is not it?
No, unless you define custom function BuildAutoCompleteText on SQL Server and make EF aware of that function.
why they use .ToArray(); instead of Tolist() ?
In this case it does not matter both implement IEnumerable<T>
Related
I have the most bizarre issue with EF Core 3.1. In EF Core 2.2 I used to be able to execute stored procedures. I see there is a breaking change in the documentation but, I am following the documentation exactly and it is not working. I have no nulls anywhere in the returned data. The NoticeOfInspection object matches the returned data exactly. What on Earth did they change that this is not working?
var data = _dbContext.NoticeOfInspections.FromSqlRaw("EXEC dbo.NewReportApp_NoticeOfInspection {0}", FacilityId).Single();
The error message is not helpful at all. First with the above line, it says, "InvalidOperationException: FromSqlRaw or FromSqlInterpolated was called with non-composable SQL and with a query composing over it. Consider calling AsEnumerable after the FromSqlRaw or FromSqlInterpolated method to perform the composition on the client side."
What?
So, I add AsEnumerable and then it throws, "InvalidCastException: Unable to cast object of type 'System.Int32' to type 'System.String'."
What on Earth have they done. This is not intuitive at all.
FromSqlRaw or FromSqlInterpolated was called with non-composable SQL
The non-composable SQL is the one which cannot be converted to subquery select * from (your_sql). Calling SP (EXEC …) is one of the non-composable constructs.
and with a query composing over it
Non query returning LINQ operators like Single, First, Count, Max, Sum etc. require composing over the provided SQL query, for instance select count * from (your_query).
You can read more about it in Raw SQL queries - Composing with LINQ documentation topic, which also contains the "calling SP" and other limitations/restrictions:
Composing with LINQ requires your raw SQL query to be composable since EF Core will treat the supplied SQL as a subquery. SQL queries that can be composed on begin with the SELECT keyword. Further, SQL passed shouldn't contain any characters or options that aren't valid on a subquery, such as:
A trailing semicolon
On SQL Server, a trailing query-level hint (for example, OPTION (HASH JOIN))
On SQL Server, an ORDER BY clause that isn't used with OFFSET 0 OR TOP 100 PERCENT in the SELECT clause
SQL Server doesn't allow composing over stored procedure calls, so any attempt to apply additional query operators to such a call will result in invalid SQL. Use AsEnumerable or AsAsyncEnumerable method right after FromSqlRaw or FromSqlInterpolated methods to make sure that EF Core doesn't try to compose over a stored procedure.
With that being said, inserting AsEnumerable() before Single() should really work.
The new exception you are getting is either EF Core bug or data type mapping issue (either you are passing int to string parameter, or SP is returning int for string class property). You need to examine the exception stack trace and/or compare your SP parameter and column types to FacilityId argument type and NoticeOfInspection class property types/mappings.
Let's say I have an Entity Framwork query
var query = db.Entities
.FancyQueryStuff()
.Where(GetFilter()) // *
.OrderBy(GetSort()) // *
.Take(GetNumberOfRows()) // *
;
and figure that this query is very slow. Testing reveals that the following rewrite is much faster:
var ids = db.Entities
.FancyQueryStuff()
.Where(GetFilter()) // *
.OrderBy(GetSort()) // *
.Take(GetNumberOfRows()) // *
.Select(x => x.Id)
.ToArray()
;
var query = db.Entries
.FancyQueryStuff()
.OrderBy(GetSort()) // *
.Where(x => ids.Contains(x.Id));
Whether that is quicker depends on a lot of things, including the sql database used, but I have a scenario in which this is the case with SQL Server and a particular query doing heavy joining.
Now the problem I have is that I want to use libraries that take IQueryables and apply Where, OrderBy, Take and Skip internally according to UI information the get from somewhere else (DevExpress/Telerik grids with paging, where the user clicks on captions to sort, etc.).
That means I have to write the query in a form where all the rows marked with an asterisk can be applied by a third-party framework.
With Devextreme, for example, you have a method that takes the query plus a data structure representing the filter/sorting/paging in a custom format and returns the query results you are supposed to pass to a client in an html application:
var result = DataSourceLoader.Load(query, loadOptions);
DataSourceLoader.Load applies everything of the kind I marked with an asterisk to the end of the query, executes it and returns the result.
I guess it's possible to do what I want with some heavy guns of linq magic (dynamic linq?), but before I try myself I thought maybe someone already has a snippet ready for this probably not too uncommon use case.
I'm getting errors when I try and do something like this:
from s in db.SomeDbSet where IsValid(s) select s
It errors telling me that it can't process IsValid.
Basically what I'm trying to do is filter based on another dbSet inside the Where that is linked and does an any, but it won't let me.
I've tried a million different ways of doing a Expression but I can't find the right way and building my own Extension method like Where doesn't seem to work either.
Thanks!
Can you paste your IsValid function?
In this case it's EF job to take LINQ syntax and turn it into SQL syntax.
EF can't turn your function into SQL. it only supports a set number of functions that have a clear SQL equivalent commend.
you have two options:
1) Rewrite the function as a series of supported commends. This will be turned into a SQL sub-query, Meaning a single trip to the database, For example:
// will only return records that have at least one related entity marked as full.
query.Where(m => m.ReletedEntities.Any(re => re.IsFull == true));
2) Get all the data from the database and then using Linq and your function work with the data. this will be done in memory using your actual function that will be called once for every item in the collection. You will also have to load the related entity collection. or it will still be an "entity framework translated to SQL query", And will fail if you use your function.
I have ADO.NET EF expression like:
db.Table1.Select(
x => new { ..., count = db.Table2.Count(y => y.ForeignKey.ID == x.ID) })
Does I understand correctly it's translated into several SQL client-server requests and may be refactored for better performance?
Thank you in advance!
Yes - the expression will get translated (in the best way it can) to a SQL query.
And just like any T-SQL query, an EF (or L2SQL) query expression can be refactored for performance.
Why not run SQL profiler in the background to see what it is getting executed, and try and optimize the raw T-SQL first - which will help optimize the expression.
Or if you have LinqPad, just optimize the T-SQL query and get LinqPad to write your query for you.
Also, im not really sure why you have specified the delegate for the Count() expression.
You can simply do this:
var query= from c in db.Table1
select new { c.CustomerID, OrderCount = c.Table2s.Count() };
The answer is NO - this query will be translated into one client-to-RDBMS request.
RPM1984 advised to use LinqPad. LinqPad showed that the query will be translated into very straightforward SQL expression. Approach with grouping will be translated into another SQL expression but still will be executed in one request.
I have the following LINQ to Entities query...
var results = from c in context.Contacts
select c;
which works fine in returning a collection of contacts. But I have seen sample code that does this instead...
ObjectResult<Contact> results = (from c in context.Contacts
select c).Execute();
What is the difference? The ObjectResult also has a collection of returned contacts. Is it just syntactic or is there a real fundamental difference?
ObjectResult<> is simply the type returned by EF when you start enumerating the IQueryable<> (i.e. context.Contacts).
So if you immediately enumerate either of your two queries, semantically it is the same.
The only difference is that in the first example if compose more query operations they will get appended to the query sent to the database when you enumerate, whereas in the second example they will get applied in memory by LINQ to Objects.
Also Execute(..) provides somewhat easier access to MergeOptions (like, should the database copy overwrite copies already in memory or visa versa). You can do this using the MergeOptions property on the ObjectQuery<> too, but that is a little more cumbersome.
Hope this helps
Alex