I have the most bizarre issue with EF Core 3.1. In EF Core 2.2 I used to be able to execute stored procedures. I see there is a breaking change in the documentation but, I am following the documentation exactly and it is not working. I have no nulls anywhere in the returned data. The NoticeOfInspection object matches the returned data exactly. What on Earth did they change that this is not working?
var data = _dbContext.NoticeOfInspections.FromSqlRaw("EXEC dbo.NewReportApp_NoticeOfInspection {0}", FacilityId).Single();
The error message is not helpful at all. First with the above line, it says, "InvalidOperationException: FromSqlRaw or FromSqlInterpolated was called with non-composable SQL and with a query composing over it. Consider calling AsEnumerable after the FromSqlRaw or FromSqlInterpolated method to perform the composition on the client side."
What?
So, I add AsEnumerable and then it throws, "InvalidCastException: Unable to cast object of type 'System.Int32' to type 'System.String'."
What on Earth have they done. This is not intuitive at all.
FromSqlRaw or FromSqlInterpolated was called with non-composable SQL
The non-composable SQL is the one which cannot be converted to subquery select * from (your_sql). Calling SP (EXEC …) is one of the non-composable constructs.
and with a query composing over it
Non query returning LINQ operators like Single, First, Count, Max, Sum etc. require composing over the provided SQL query, for instance select count * from (your_query).
You can read more about it in Raw SQL queries - Composing with LINQ documentation topic, which also contains the "calling SP" and other limitations/restrictions:
Composing with LINQ requires your raw SQL query to be composable since EF Core will treat the supplied SQL as a subquery. SQL queries that can be composed on begin with the SELECT keyword. Further, SQL passed shouldn't contain any characters or options that aren't valid on a subquery, such as:
A trailing semicolon
On SQL Server, a trailing query-level hint (for example, OPTION (HASH JOIN))
On SQL Server, an ORDER BY clause that isn't used with OFFSET 0 OR TOP 100 PERCENT in the SELECT clause
SQL Server doesn't allow composing over stored procedure calls, so any attempt to apply additional query operators to such a call will result in invalid SQL. Use AsEnumerable or AsAsyncEnumerable method right after FromSqlRaw or FromSqlInterpolated methods to make sure that EF Core doesn't try to compose over a stored procedure.
With that being said, inserting AsEnumerable() before Single() should really work.
The new exception you are getting is either EF Core bug or data type mapping issue (either you are passing int to string parameter, or SP is returning int for string class property). You need to examine the exception stack trace and/or compare your SP parameter and column types to FacilityId argument type and NoticeOfInspection class property types/mappings.
Related
I have never yet used .AsEnumerable() in an EntityFramework query.
See the below example and tell me why they use .AsEnumerable() before Select ?
Could they not just use Select directly?
Please tell me the reason for the usage of .AsEnumerable() here in below query.
Why did they use .ToArray() instead of .Tolist() ?
private IEnumerable<AutoCompleteData> GetAutoCompleteData(string searchTerm)
{
using (var context = new AdventureWorksEntities())
{
var results = context.Products
.Include("ProductSubcategory")
.Where(p => p.Name.Contains(searchTerm)
&& p.DiscontinuedDate == null)
.AsEnumerable()
.Select(p => new AutoCompleteData
{
Id = p.ProductID,
Text = BuildAutoCompleteText(p)
})
.ToArray();
return results;
}
}
The difference between AsEnumerable and AsQueryable is that the enumerable contains all information to create an enumerator. Once you've got the enumerator you can ask for the first element, and if there is one, you can get the next one.
The Queryable does not hold the information to create the enumerator. It holds an Expression and a Provider. The Provider knows which process must execute the Expression and which language this process uses. Quite often the other process is a database management system, and the language is SQL.
The result of a Queryable.Select(...) is still an IQueryable, meaning that the query is not performed yet. The Select function only changed the Expression.
Only if you ask for the Enumerator, either explicitly by calling GetEnumerator(), or implicitly by calling foreach, or one of the non-deferred execution functions like ToList(), ToDictionary(), FirstOrDefault(), Sum(), the Provider will translate the expression into the format that the execution process understands and execute the query. Once the data is transported to the local process the enumerator is created.
Alas, sometimes you want to call your own functions in your query. SQL does not know these functions, and thus the Provider can't translate such Expressions into SQL. In fact, the provider of DbContext does not even know all Linq functions. See supported and unsupported Linq methods
That is the moment when you use AsEnumerable(). If you ask for the Enumerator (in your foreach for example), the Provider will translate the Expression until AsEnumerable; send it to the execution process and transport all data to local process. After that, the query will be AsEnumerable: the rest of the LINQ will be performed in local memory, and thus your local functions can be called.
You could of course use ToList() to fetch all data to local memory and continue your linq after that. But that would be a waste if you'd only want the first element, or every other one.
This brings me to the final remark: the transport of the data from the DBMS to your local memory is one of the slower parts. Try to limit this transport to only the data you'll actually use.
For example: if you have a one-to-many relation between a Teacher and his Students, don't fetch the Teacher and his Students, because you'll transport Student.TeacherId many times, and they will all have the same value as Teacher.Id. Instead, only select the data you really want to use
Not all Select projections, Where predicates, and Aggregations can be translated from C# Expressions into native database queries - in your case, the full LINQ expression attempts to construct a AutoCompleteData class with using a custom function BuildAutoCompleteText to set one of its properties - this cannot be trivially converted into native database code like SQL.
In your case, AsEnumerable serves to terminate the work which will be done in SQL before this will be executed in SQL.
i.e.
.Include("ProductSubcategory")
.Where(p => p.Name.Contains(searchTerm)
&& p.DiscontinuedDate == null)
will be executed in SQL, roughly as a JOIN to ProductSubcategory, and a WHERE predicate translated from your Products such as:
Product.Name LIKE '%' + #SearchTerm + '%' AND Product.DiscontinuedDate IS NULL
All work subsequent to the AsEnumerable (i.e. the projection of the results to AutoCompleteData objects) will be done in-memory with LINQ to objects.
ToArray and ToList will both execute (materialize) the result, but into different data structures. In your example, neither materialisation is required - since the return type is IEnumerable<AutoCompleteData> - the caller of the function might execute .Any() or First() which would render full materialisation wasteful - I would recommend you remove .ToArray() altogether - since the using statement controls the SQL lifespan is protected by the AsEnumerable() materialization, there is no issue with connection lifespans here.
tell me the intention of usage of .AsEnumerable() here in below query?
In this particular example AsEnumerable() was used to bring the data back to the client, because EF has no idea how to map BuildAutoCompleteText() to SQL query.
they could use select directly.....is not it?
No, unless you define custom function BuildAutoCompleteText on SQL Server and make EF aware of that function.
why they use .ToArray(); instead of Tolist() ?
In this case it does not matter both implement IEnumerable<T>
I'm getting errors when I try and do something like this:
from s in db.SomeDbSet where IsValid(s) select s
It errors telling me that it can't process IsValid.
Basically what I'm trying to do is filter based on another dbSet inside the Where that is linked and does an any, but it won't let me.
I've tried a million different ways of doing a Expression but I can't find the right way and building my own Extension method like Where doesn't seem to work either.
Thanks!
Can you paste your IsValid function?
In this case it's EF job to take LINQ syntax and turn it into SQL syntax.
EF can't turn your function into SQL. it only supports a set number of functions that have a clear SQL equivalent commend.
you have two options:
1) Rewrite the function as a series of supported commends. This will be turned into a SQL sub-query, Meaning a single trip to the database, For example:
// will only return records that have at least one related entity marked as full.
query.Where(m => m.ReletedEntities.Any(re => re.IsFull == true));
2) Get all the data from the database and then using Linq and your function work with the data. this will be done in memory using your actual function that will be called once for every item in the collection. You will also have to load the related entity collection. or it will still be an "entity framework translated to SQL query", And will fail if you use your function.
I primarily use CFQUERYPARAM to prevent SQL injection. Since Query-of-Queries (QoQ) does not touch the database, is there any logical reason to use CFQUERYPARAM in them? I know that values that do not match the cfsqltype and maxlength will throw an exception, but, these values should already be validated before that and display friendly messages (from a UX viewpoint).
Since Query-of-Queries (QoQ) does not touch the database, is there any logical reason to use CFQUERYPARAM in them? Actually, it does touch the database, the database that you currently have stored in memory. The data in that database could still theoretically be tampered with via some sort of injection from the user. Does that affect your physical database - no. Does that affect the use of the data within your application - yes.
You did not give any specific details but I would err on the side of caution. If ANY of the data you are using to build your query comes from the client then use cfqueryparam in them. If you can guarantee that none of the elements in your query comes from the client then I think it would be okay to not use the cfqueryparam.
As an aside, using cfqueryparam also helps optimize the query for the database although I'm not sure if that is true for query of queries. It also escapes characters for you like apostrophes.
Here is a situation where it's simpler, in my opinion.
<cfquery name="NoVisit" dbtype="query">
select chart_no, patient_name, treatment_date, pr, BillingCompareField
from BillingData
where BillingCompareField not in
(<cfqueryparam cfsqltype="cf_sql_varchar"
value="#ValueList(FinalData.FinalCompareField)#" list="yes">)
</cfquery>
The alternative would be to use QuotedValueList. However, if anything in that value list contained an apostrophe, cfqueryparam will escape it. Otherwise I would have to.
Edit starts here
Here is another example where not using query parameters causes an error.
QueryAddRow(x,2);
QuerySetCell(x,"dt",CreateDate(2001,1,1),1);
QuerySetCell(x,"dt",CreateDate(2001,1,11),2);
</cfscript>
<cfquery name="y" dbtype="query">
select * from x
<!---
where dt in (<cfqueryparam cfsqltype="cf_sql_date" value="#ValueList(x.dt)#" list="yes">)
--->
where dt in (#ValueList(x.dt)#)
</cfquery>
The code as written throws this error:
Query Of Queries runtime error.
Comparison exception while executing IN.
Unsupported Type Comparison Exception:
The IN operator does not support comparison between the following types:
Left hand side expression type = "DATE".
Right hand side expression type = "LONG".
With the query parameter, commented out above, the code executes successfully.
I have a stored procedure in my project under sql/my_prod.sql
there I have my function delete_entity
In my entity
#NamedNativeQuery(name = "delete_entity_prod",
query = "{call /sql/delete_entity(:lineId)}",
and I call it
Query query = entityManager.createNamedQuery("delete_entity_prod")
setParameter("lineId",lineId);
I followed this example: http://objectopia.com/2009/06/26/calling-stored-procedures-in-jpa/
but it does not execute the delete and it does not send any error.
I haven't found clear information about this, am I missing something? Maybe I need to load the my_prod.sql first? But how?
JPA 2.1 standardized stored procedure support if you are able to use it, with examples here http://en.wikibooks.org/wiki/Java_Persistence/Advanced_Topics#Stored_Procedures
This is actually they way you create a query.
Query query = entityManager.createNamedQuery("delete_entity_prod")
setParameter("lineId",lineId);
To call it you must execute:
query.executeUpdate();
Of course, the DB must already contain the procedure. So if you have it defined in your SQL file, have a look at Executing SQL Statements from a Text File(this is for MySQL but other database systems use a similar approach to execute scripts)
There is no error shown because query is not executed at any point - just instance of Query is created. Query can be executed by calling executeUpdate:
query.executeUpdate();
Then next problem will arise: Writing some stored procedures to file is not enough - procedures live in database, not in files. So next thing to do is to check that there is correct script to create stored procedure in hands (maybe that is currently content of sql/my_prod.sql) and then use that to create procedure via database client.
All JPA implementations do not support calling stored procedures, but I assume Hibernate is used under the hood, because that is also used in linked tutorial.
It can be the case that current
{call /sql/delete_entity(:lineId)}
is right syntax for calling stored procedure in your database. It looks rather suspicious because of /sql/. If it turns out that this is incorrect syntax, then:
Consult manual for correct syntax
Test via client
Use that as a value of query attribute in NamedNativeQuery annotation.
All that with combination MySQL+Hibernate is explained for example here.
I am using JPA native query to call a stored procedure and map the return result to a class
createNativeQuery(String sqlString, Class resultClass);
Here sqlString is a stored procedure in this format:
{call storeProcedureName parameter1, paramter2, parameter3}
I noticed that when one of the parameter contains a forward slash (/), the result result will be wrong. Has any body encountered this problem before and how to solve it? thanks
I am using EclipseLink and glassfish server.
Are you in-lining the parameters into your SQL, or using parameters on your query? You should use parameters on your query, in-lining parameters into SQL is very bad (can lead to SQL injection attacks).
See,
http://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Querying/Native#Parameters