I have been doing queries in EF and everything working great but now i have in the db 2 fields that are actually CHAR.. They hold a date but in the form of a number, in SQL Management Studio i can do date1 >= date2 for example and i can also check to see if a number i have falls in between these 2 dates.
Its nothing unusual, but basically a field that represents a date (the number grows as the date does)...
Now in EF when i try to do >= it states you can't do this on a string, ok understand its c# so i tried doing Convert.ToDecimal(date1) but it gives me an error saying that its not supported.
I have no option of changing the db fields, they are set in stone :-(
the way i got it to work was request of details and do a .ToList and then use the .ToDecimal and it works but of course this is doing it in memory! and this defeats the object of EF i.e. for example adding to the query using iqueryable.
Another way i got it to work was to pass the SQL query to SqlQuery of the dbcontext but again i lose a lot of ef functionality.
Can anyone help?
I am really stuck
As you say that you tried >= I assume that it would work for you if you could do that in plain SQL. And that is possible by doing
String.Compare(date1, date2) >= 0
EF is smart enough to translate that into a >= operator.
The advantage is that you do not need to compare converted values, so indexes can be used in execution plans.
First of all, you can at least enable deferred execution of the query by using AsEnumerable() instead of ToList(). This won't change the fact that the database would need to return all the records when you do in fact execute the query, however.
To let the database perform the filtering, you need your query to be compatible with SQL. Since you can't do ToDecimal() in SQL, you need to work with strings directly by converting your myvar to a string that is in the same format as dateStart and dateEnd, then form your query.
Related
I am having an issue with a date literal not returning any results when I run an SOQL query. The query is as follows:
Select * From dbo.Case WHERE CreatedDate = YESTERDAY
With the query, I would like to obtain case data from the previous day. I know there is data available that was created the previous day. The results of the query when I preview it, though, are an empty set with no error message.
A different wrinkle that makes this not quite a strict SOQL issue is that I am trying to use this query as an SQL command on an ADO.NET connection using the CData ADO.NET driver to connect to a SalesForce.com instance. My goal is to be able to build SSDT packages that will allow me to stage the data from SalesForce into our SQL Server for processing there.
I have similar issues using the LAST_N_DAYS date literal as well.
I believe I should be using SOQL to query in the SQL command text field for the ADO.NET source connection but I am not 100% sure about that. I know for certain that I cannot use T-SQL because it does not recognize the GETDATE().
Any guidance on how to pull the records from Case for the previous day or where the query I am using might be wrong would be greatly appreciated.
Found an answer. The following SQL command will pull the data from the previous day:
Select * From dbo.Case Where CreatedDate = 'YESTERDAY'
The single quotes evaluate the yesterday date literal as expected.
Likewise, the following SQL statement will get the last 30 days of data.
Select * From dbo.Case Where CreatedDate = 'LAST_N_DAYS:30'
Thanks to anyone who researched and attempted the question! :)
i have a DB2 data source and an Oracle 12c target.
The Oracle has a DB link to the DB2 defined which is working in general.
Now i have a huge table in the DB2 which has a timestamp column (lets call it ROW_CHANGED) for row changes. I want to retrieve rows which have changed after a particular time.
Running
SELECT * FROM lib.tbl WHERE ROW_CHANGED >'2016-08-01 10:00:00'
on the DB2 returns exactly 1 row after ca. 90 secs which is fine.
Now i try the same query from the Oracle via the db link:
SELECT * FROM lib.tbl#dblink_name WHERE ROW_CHANGED >TO_TIMESTAMP('2016-08-01 10:00:00')
This runs for hours and ends up in a timeout.
I read some Oracle docs and found distributed query optimization tips but most of them refer to joining a local to a remote table which is not my case.
In my desperation, i have tried the DRIVING_SITE hint, without effect.
Now i wonder when the WHERE part of the query will be evaluated. Since i have to use Oracle syntax and not DB2 syntax for the query, is it possible the Oracle will try to first copy the full table and apply the where clause afterwards? I did some research but did not find anything which would help me in this direction.
The ROW_CHANGED is a hidden column in the DB2, if that matters.
Thx for any hint in advance.
Update
Thanks#all for help. I'll share what did the trick for me.
First of all i have used TO_TIMESTAMP since the DB2 column is also Timestamp (not date) and i had expected to circumvent implicit conversions by this.
Without the explicit conversion i ran into ORA-28534: Heterogeneous Services preprocessing error and i have no hope of touching the DB config within reasonable time.
The explain plan btw did not bring much. It showed a FULL hint and no conversion on the predicates. Indeed it showed the ROW_CHANGED column as Date, i wonder why.
I have tried Justins suggestion to use a bind variable, however i got ORA-28534 again. Next thing i did was to wrap it into a pl/sql block (will run in a SP anyway later).
declare
v_tmstmp TIMESTAMP := 01.08.16 10:00:00;
begin
INSERT INTO ORAUSER.TMP_TBL (SRC_PK,ROW_CHANGED)
SELECT SRC_PK,ROW_CHANGED
FROM lib.tbl#dblink_name
WHERE ROW_CHANGED > v_tmstmp;
end;
This was executing in the same time as in DB2 itself. The date format is DD.MM.YY here since it is the default unfortunately.
When changing the variable assignment to
v_tmstmp TIMESTAMP := TO_TIMESTAMP('01.08.16 10:00:00','DD.MM.YY HH24:MI:SS');
I got the same problem as before.
Meanwhile the DB2 operators have created an index in the ROW_CHANGED column which i requested earlier that day. This has solved the problem in general it seems. Even my original query finishes in no time now.
If you are actually using an Oracle-specific conversion function like to_timestamp, that forces the predicate to be evaluated on the Oracle side. Oracle isn't going to know how to convert a built-in function like to_timestamp into an exactly equivalent function call in DB2.
If you used a bind variable, that would be more likely to get evaluated on the DB2 side. But that may be complicated by the data type mapping between different databases-- there may not be a perfect mapping between one engine's date and another engine's timestamp data type. If this was a numeric column, a bind variable would be almost certain to get pushed. In this case, it probably involves playing around a bit to figure out exactly what data type to use for your variable that works for your framework, Oracle, and DB2.
If using a bind variable doesn't work, you can force the predicate to be evaluated on the remote server using the dbms_hs_passthrough package. That lets you send a query verbatim to the remote server which allows you to do things like use functions defined in your DB2 database. That's a bit of overkill in this situation, hopefully, but it's nice to have the hammer as your backup if the simpler solution doesn't work quickly enough.
I'm getting errors when I try and do something like this:
from s in db.SomeDbSet where IsValid(s) select s
It errors telling me that it can't process IsValid.
Basically what I'm trying to do is filter based on another dbSet inside the Where that is linked and does an any, but it won't let me.
I've tried a million different ways of doing a Expression but I can't find the right way and building my own Extension method like Where doesn't seem to work either.
Thanks!
Can you paste your IsValid function?
In this case it's EF job to take LINQ syntax and turn it into SQL syntax.
EF can't turn your function into SQL. it only supports a set number of functions that have a clear SQL equivalent commend.
you have two options:
1) Rewrite the function as a series of supported commends. This will be turned into a SQL sub-query, Meaning a single trip to the database, For example:
// will only return records that have at least one related entity marked as full.
query.Where(m => m.ReletedEntities.Any(re => re.IsFull == true));
2) Get all the data from the database and then using Linq and your function work with the data. this will be done in memory using your actual function that will be called once for every item in the collection. You will also have to load the related entity collection. or it will still be an "entity framework translated to SQL query", And will fail if you use your function.
I primarily use CFQUERYPARAM to prevent SQL injection. Since Query-of-Queries (QoQ) does not touch the database, is there any logical reason to use CFQUERYPARAM in them? I know that values that do not match the cfsqltype and maxlength will throw an exception, but, these values should already be validated before that and display friendly messages (from a UX viewpoint).
Since Query-of-Queries (QoQ) does not touch the database, is there any logical reason to use CFQUERYPARAM in them? Actually, it does touch the database, the database that you currently have stored in memory. The data in that database could still theoretically be tampered with via some sort of injection from the user. Does that affect your physical database - no. Does that affect the use of the data within your application - yes.
You did not give any specific details but I would err on the side of caution. If ANY of the data you are using to build your query comes from the client then use cfqueryparam in them. If you can guarantee that none of the elements in your query comes from the client then I think it would be okay to not use the cfqueryparam.
As an aside, using cfqueryparam also helps optimize the query for the database although I'm not sure if that is true for query of queries. It also escapes characters for you like apostrophes.
Here is a situation where it's simpler, in my opinion.
<cfquery name="NoVisit" dbtype="query">
select chart_no, patient_name, treatment_date, pr, BillingCompareField
from BillingData
where BillingCompareField not in
(<cfqueryparam cfsqltype="cf_sql_varchar"
value="#ValueList(FinalData.FinalCompareField)#" list="yes">)
</cfquery>
The alternative would be to use QuotedValueList. However, if anything in that value list contained an apostrophe, cfqueryparam will escape it. Otherwise I would have to.
Edit starts here
Here is another example where not using query parameters causes an error.
QueryAddRow(x,2);
QuerySetCell(x,"dt",CreateDate(2001,1,1),1);
QuerySetCell(x,"dt",CreateDate(2001,1,11),2);
</cfscript>
<cfquery name="y" dbtype="query">
select * from x
<!---
where dt in (<cfqueryparam cfsqltype="cf_sql_date" value="#ValueList(x.dt)#" list="yes">)
--->
where dt in (#ValueList(x.dt)#)
</cfquery>
The code as written throws this error:
Query Of Queries runtime error.
Comparison exception while executing IN.
Unsupported Type Comparison Exception:
The IN operator does not support comparison between the following types:
Left hand side expression type = "DATE".
Right hand side expression type = "LONG".
With the query parameter, commented out above, the code executes successfully.
I have a generic code that is used to retrieve DDL information from a Firebird database (FB2.1). It generates SQL code like
SELECT * FROM MyTable where 'c' <> 'c'
I cannot change this code. Actually, if that matters, it is inside Report Builder 10.
The fact is that some tables from my database are becoming a litle too populated (>1M records) and that query is starting to take too long to execute.
If I try to execute
SELECT * FROM MyTable where SomeIndexedField = SomeImpossibleValue
it will obviously use that index and run very quickly.
Well, it wouldn´t be that hard to the database find out that that is an impossible matcher and make some sort of optimization and avoid testing it against each row.
Is there any way to make my firebird database to optimize that search?
As the filter condition is a negative proposition (and also doesn't refer a column to search, but only a value to compare to another value), Firebird need to do a full table scan (without use any index) to confirm that aren't any record that meet your criteria.
If you can't change you need to wait for the upcoming 3.0 version, that will implement the Boolean data type, and therefore should start to evaluate "constant" fake comparisons in advance (maybe the client library will do this evaluation before send the statement to the server?).