I have ADO.NET EF expression like:
db.Table1.Select(
x => new { ..., count = db.Table2.Count(y => y.ForeignKey.ID == x.ID) })
Does I understand correctly it's translated into several SQL client-server requests and may be refactored for better performance?
Thank you in advance!
Yes - the expression will get translated (in the best way it can) to a SQL query.
And just like any T-SQL query, an EF (or L2SQL) query expression can be refactored for performance.
Why not run SQL profiler in the background to see what it is getting executed, and try and optimize the raw T-SQL first - which will help optimize the expression.
Or if you have LinqPad, just optimize the T-SQL query and get LinqPad to write your query for you.
Also, im not really sure why you have specified the delegate for the Count() expression.
You can simply do this:
var query= from c in db.Table1
select new { c.CustomerID, OrderCount = c.Table2s.Count() };
The answer is NO - this query will be translated into one client-to-RDBMS request.
RPM1984 advised to use LinqPad. LinqPad showed that the query will be translated into very straightforward SQL expression. Approach with grouping will be translated into another SQL expression but still will be executed in one request.
Related
I have the most bizarre issue with EF Core 3.1. In EF Core 2.2 I used to be able to execute stored procedures. I see there is a breaking change in the documentation but, I am following the documentation exactly and it is not working. I have no nulls anywhere in the returned data. The NoticeOfInspection object matches the returned data exactly. What on Earth did they change that this is not working?
var data = _dbContext.NoticeOfInspections.FromSqlRaw("EXEC dbo.NewReportApp_NoticeOfInspection {0}", FacilityId).Single();
The error message is not helpful at all. First with the above line, it says, "InvalidOperationException: FromSqlRaw or FromSqlInterpolated was called with non-composable SQL and with a query composing over it. Consider calling AsEnumerable after the FromSqlRaw or FromSqlInterpolated method to perform the composition on the client side."
What?
So, I add AsEnumerable and then it throws, "InvalidCastException: Unable to cast object of type 'System.Int32' to type 'System.String'."
What on Earth have they done. This is not intuitive at all.
FromSqlRaw or FromSqlInterpolated was called with non-composable SQL
The non-composable SQL is the one which cannot be converted to subquery select * from (your_sql). Calling SP (EXEC …) is one of the non-composable constructs.
and with a query composing over it
Non query returning LINQ operators like Single, First, Count, Max, Sum etc. require composing over the provided SQL query, for instance select count * from (your_query).
You can read more about it in Raw SQL queries - Composing with LINQ documentation topic, which also contains the "calling SP" and other limitations/restrictions:
Composing with LINQ requires your raw SQL query to be composable since EF Core will treat the supplied SQL as a subquery. SQL queries that can be composed on begin with the SELECT keyword. Further, SQL passed shouldn't contain any characters or options that aren't valid on a subquery, such as:
A trailing semicolon
On SQL Server, a trailing query-level hint (for example, OPTION (HASH JOIN))
On SQL Server, an ORDER BY clause that isn't used with OFFSET 0 OR TOP 100 PERCENT in the SELECT clause
SQL Server doesn't allow composing over stored procedure calls, so any attempt to apply additional query operators to such a call will result in invalid SQL. Use AsEnumerable or AsAsyncEnumerable method right after FromSqlRaw or FromSqlInterpolated methods to make sure that EF Core doesn't try to compose over a stored procedure.
With that being said, inserting AsEnumerable() before Single() should really work.
The new exception you are getting is either EF Core bug or data type mapping issue (either you are passing int to string parameter, or SP is returning int for string class property). You need to examine the exception stack trace and/or compare your SP parameter and column types to FacilityId argument type and NoticeOfInspection class property types/mappings.
I am using Linq2DB for PostgreSQL. I need to group the data by week in Linq. Below is my code:
Func<DateTime,int> weekProjector = d => CultureInfo.CurrentCulture.Calendar.GetWeekOfYear(d,
CalendarWeekRule.FirstFourDayWeek,
DayOfWeek.Sunday);
var ddquery = from t in db.VLogData
group t by weekProjector(t.NewTime);
foreach (var f in ddquery)
{
f.Key.ToString();
}
But it show the error or foreach there
(xxx.Controllers.DashboardController+<>c__DisplayClass1_0).weekProjector,
selectParam.NewTime)' cannot be converted to SQL.'
May I know what is the issue there?
weekProjector is a client-side function, but you pass server-side data to it as input. To make it work you can:
perform grouping on client by pasting AsEnumerable() call between Select and GroupBy calls (worst option ever)
tell linq2db which aggregation logic to use by applying Sql.ExpressionAttribute or Sql.ExtensionAttribute on weekProjector with SQL expression and IsAggregate=true set for attribute
same as previous, but by using ExpressionMethodAttribute to define aggregation logic using C# expressions
I have never yet used .AsEnumerable() in an EntityFramework query.
See the below example and tell me why they use .AsEnumerable() before Select ?
Could they not just use Select directly?
Please tell me the reason for the usage of .AsEnumerable() here in below query.
Why did they use .ToArray() instead of .Tolist() ?
private IEnumerable<AutoCompleteData> GetAutoCompleteData(string searchTerm)
{
using (var context = new AdventureWorksEntities())
{
var results = context.Products
.Include("ProductSubcategory")
.Where(p => p.Name.Contains(searchTerm)
&& p.DiscontinuedDate == null)
.AsEnumerable()
.Select(p => new AutoCompleteData
{
Id = p.ProductID,
Text = BuildAutoCompleteText(p)
})
.ToArray();
return results;
}
}
The difference between AsEnumerable and AsQueryable is that the enumerable contains all information to create an enumerator. Once you've got the enumerator you can ask for the first element, and if there is one, you can get the next one.
The Queryable does not hold the information to create the enumerator. It holds an Expression and a Provider. The Provider knows which process must execute the Expression and which language this process uses. Quite often the other process is a database management system, and the language is SQL.
The result of a Queryable.Select(...) is still an IQueryable, meaning that the query is not performed yet. The Select function only changed the Expression.
Only if you ask for the Enumerator, either explicitly by calling GetEnumerator(), or implicitly by calling foreach, or one of the non-deferred execution functions like ToList(), ToDictionary(), FirstOrDefault(), Sum(), the Provider will translate the expression into the format that the execution process understands and execute the query. Once the data is transported to the local process the enumerator is created.
Alas, sometimes you want to call your own functions in your query. SQL does not know these functions, and thus the Provider can't translate such Expressions into SQL. In fact, the provider of DbContext does not even know all Linq functions. See supported and unsupported Linq methods
That is the moment when you use AsEnumerable(). If you ask for the Enumerator (in your foreach for example), the Provider will translate the Expression until AsEnumerable; send it to the execution process and transport all data to local process. After that, the query will be AsEnumerable: the rest of the LINQ will be performed in local memory, and thus your local functions can be called.
You could of course use ToList() to fetch all data to local memory and continue your linq after that. But that would be a waste if you'd only want the first element, or every other one.
This brings me to the final remark: the transport of the data from the DBMS to your local memory is one of the slower parts. Try to limit this transport to only the data you'll actually use.
For example: if you have a one-to-many relation between a Teacher and his Students, don't fetch the Teacher and his Students, because you'll transport Student.TeacherId many times, and they will all have the same value as Teacher.Id. Instead, only select the data you really want to use
Not all Select projections, Where predicates, and Aggregations can be translated from C# Expressions into native database queries - in your case, the full LINQ expression attempts to construct a AutoCompleteData class with using a custom function BuildAutoCompleteText to set one of its properties - this cannot be trivially converted into native database code like SQL.
In your case, AsEnumerable serves to terminate the work which will be done in SQL before this will be executed in SQL.
i.e.
.Include("ProductSubcategory")
.Where(p => p.Name.Contains(searchTerm)
&& p.DiscontinuedDate == null)
will be executed in SQL, roughly as a JOIN to ProductSubcategory, and a WHERE predicate translated from your Products such as:
Product.Name LIKE '%' + #SearchTerm + '%' AND Product.DiscontinuedDate IS NULL
All work subsequent to the AsEnumerable (i.e. the projection of the results to AutoCompleteData objects) will be done in-memory with LINQ to objects.
ToArray and ToList will both execute (materialize) the result, but into different data structures. In your example, neither materialisation is required - since the return type is IEnumerable<AutoCompleteData> - the caller of the function might execute .Any() or First() which would render full materialisation wasteful - I would recommend you remove .ToArray() altogether - since the using statement controls the SQL lifespan is protected by the AsEnumerable() materialization, there is no issue with connection lifespans here.
tell me the intention of usage of .AsEnumerable() here in below query?
In this particular example AsEnumerable() was used to bring the data back to the client, because EF has no idea how to map BuildAutoCompleteText() to SQL query.
they could use select directly.....is not it?
No, unless you define custom function BuildAutoCompleteText on SQL Server and make EF aware of that function.
why they use .ToArray(); instead of Tolist() ?
In this case it does not matter both implement IEnumerable<T>
I'm getting errors when I try and do something like this:
from s in db.SomeDbSet where IsValid(s) select s
It errors telling me that it can't process IsValid.
Basically what I'm trying to do is filter based on another dbSet inside the Where that is linked and does an any, but it won't let me.
I've tried a million different ways of doing a Expression but I can't find the right way and building my own Extension method like Where doesn't seem to work either.
Thanks!
Can you paste your IsValid function?
In this case it's EF job to take LINQ syntax and turn it into SQL syntax.
EF can't turn your function into SQL. it only supports a set number of functions that have a clear SQL equivalent commend.
you have two options:
1) Rewrite the function as a series of supported commends. This will be turned into a SQL sub-query, Meaning a single trip to the database, For example:
// will only return records that have at least one related entity marked as full.
query.Where(m => m.ReletedEntities.Any(re => re.IsFull == true));
2) Get all the data from the database and then using Linq and your function work with the data. this will be done in memory using your actual function that will be called once for every item in the collection. You will also have to load the related entity collection. or it will still be an "entity framework translated to SQL query", And will fail if you use your function.
I know how to use "DISTINCT" in Doctrine 2, but I really need to use "DISTINCT ON (field)" and I don't know how to do this with the QueryBuilder.
My SQL query looks like:
SELECT DISTINCT ON (currency) currency, amount FROM payments ORDER BY currency
And this query works perfect, but I can't use it with the QueryBuilder. Maybe I could write this query on some other way?
I would suggest that the SELECT DISTINCT ON (..) construct that PostgreSQL supports is outside the Object Relational Model (ORM) that is central to Doctrine. Or, perhaps put another way, because SELECT DISTINCT ON (..) is rare in SQL implementations Doctrine haven't coded for it.
Regardless of the actual logic for it not working, I would suggest you try Doctrine's "Native SQL". You need to map the results of your query to the ORM.
With NativeQuery you can execute native SELECT SQL statements and map
the results to Doctrine entities or any other result format supported
by Doctrine.
In order to make this mapping possible, you need to describe to
Doctrine what columns in the result map to which entity property. This
description is represented by a ResultSetMapping object.
With this feature you can map arbitrary SQL code to objects, such as
highly vendor-optimized SQL or stored-procedures.
SELECT DISTINCT ON (..) falls into vendor-optimized SQL I think, so using NativeQuery should allow you to access it.
Doctrine QueryBuilder has some limitations. Even if I didn't check if it's was possible with query builder, I do not hesitate to use DQL when I do not know how to write the query with query builder.
Check theses examples at
http://doctrine-orm.readthedocs.org/en/latest/reference/dql-doctrine-query-language.html#dql-select-examples
Hope this help.
INDEX BY can be used in DQL, allowing first result rows indexed by the defined string/int field to be overwritten by following ones with the same index:
SELECT
p.currency,
p.amount
FROM Namespace\To\Payments p INDEX BY p.currency
ORDER BY p.currency ASC
DQL - EBNF - INDEX BY