I want to run this issue by those who know more about PL/SQL and T-SQL than I do. Below is some code that I inherited and it's returning dates that are 1 hour off. When I remove the "to_char(i.UPDATE_DT, ''YYYY-MM-DD HH24:MI:SS.FF3'')" in the select statement of the remote query the hour offset goes away. I'm worried about other impacts this change could have, so my question is why would someone convert an Oracle time to a string via an OpenQuery and then use T-SQL convert in the select statement? What am I missing here and why would the to_char cause an offset (if it is the reason)?
SELECT
CSE_ID AS ID,
OTHR_CSE_ID AS Case_Num,
convert(datetime,UPDATE_DT,121) AS Export_Time
FROM
OpenQuery( OracleTbl, '
SELECT c.CSE_ID, r.OTHR_CSE_ID, to_char(i.UPDATE_DT,''YYYY-MM-DD HH24:MI:SS.FF3'') UPDATE_DT
FROM AE_CSES c
INNER JOIN CSES i ON i.CASE_ID = c.CSE_ID AND i.ACTION_CD = ''INS''
INNER JOIN OTH_CSE_REFS r ON r.CSE_ID = c.CSE_ID AND r.OTH_CS_REF_SEQ_NBR = 1 AND r.OTHER_SOURCE_TYPE = ''SIE''
') AS i
Simplistic answer but; Daylight savings issue between one server and the other?
Related
I've just started working with PostgreSQL, I've used to work with SQL Server and I'm currently migrating some of the existing processes.
The current issue which I'm facing is the performance for an Update statement.
I'm trying to update all records from one table (e.g. MyTable_History) and set new values for some columns.
In Sql Server I've used the following syntax:
declare #NewEndDate datetime = (select dateadd(minute, -1, getdate()))
update MyTable_History
set isLastestVersion=0, ValidTo=#NewEndDate , ModifiedBy='TestSCriptSql',ModifiedTime=GETDATE()
The code which i could come up with (since I don't know how to simply use variables, therefore used a temp tbl) for PostgreSQL is:
CREATE TEMP TABLE dates AS VALUES (current_timestamp + (-1 ||' minutes')::interval);
with d as (
select th.validto as validto, th.islatestversion as islatestversion,
th.modifiedby as modifiedby, th.modifiedtime as modifiedtime, d.column1 as newvalidto
from MyTable_History th, dates d
)
update MyTable_History
set validto = d.newvalidto, islatestversion=false, modifiedby='test_update_script', modifiedtime=current_timestamp
from d
The Sql Server runs localy on my laptop (not a super config) and the PosgreSQL server runs on AWS as RDS (i don't know the exact specs).
My question is am I doing something wrong in the PostgreSql update statement? Because on a 5000+ dataset sample on Sql Server the statement is instantly performed, while on PostgreSql it takes around 50 secs to successfully finish.
Also, from my point of view it seems I've over engineered, since on Sql Server I was having 3 lines of code, while on postgreSql i'm using a CTE.
Regrards,
I don't see why you would need a variable to begin with. current_timestamp returns the same value throughout a transaction as documented in the manual and thus will have the same value for all updated rows.
update mytable_history
set islastestversion = 0,
validto = current_timestamp - interval '1 minute',
modifiedby = 'test_update_script',
modifiedtime = current_timestamp;
But your usage of FROM in the UPDATE statement is wrong. The semantics of using FROM in an UPDATE statement are very different between Postgres and SQL Server
The way you use it, creates a cross join between the CTE and mytable_history. (so essentially a cross join of the table with itself).
You need to have a join condition in the WHERE clause on the primary key:
with d as (...)
update MyTable_History
set validto = d.newvalidto, islatestversion=false,
modifiedby='test_update_script', modifiedtime=current_timestamp
from d
where d.pk_column = MyTable_History.pk_column;
But if you really want to simulate something like variables, you don't need the CTE:
update mytable_history
set islastestversion = 0,
validto = t.newvalidto
modifiedby = 'test_update_script',
modifiedtime = current_timestamp
from (
values (current_timestamp - interval '1 minute')
) t (newvalidto);
The above still creates a "cross join" but as the joined table (from (values ...)) only contains a single row, it's not really a cross join.
Dear StackOverflow Community,
as a New to DB2 ,i have a a query
may be its a very basic question for you, please share your knowledge.
i have a start date and End Date.
I need a list of each and every date in between.
Its ok with me ,if it creates a temp table no issue.
Thanks in Advance
You can generate the dates between start and end dates by using Recursive CTE expression. Try below code
with cte(your_columns,startdate,enddate)
as (select your_columns,startdate,enddate,startdate
as derDate
from yourTable
union all
select your_columns,startdate,enddate,derDate+1
from cte where
derDate<=endDate)
select * from cte
I have a calendar table called CalendarInformation that gives me a list of dates from 2015 to 2025. This table has a column called BusinessDay that shows what dates are weekends or holidays. I have another table called OpenProblemtimeDiffTable with a column called number for my problem number and a date for when the problem was opened called ProblemNew and another date for the current column called Now. What I want to do is for each problem number grab its date ranges and find the dates between and then sum them up to give me the number of business days. Then I want to insert these values in another table with the problem number associated with the business day.
Thanks in advance and I hope I was clear.
TRUNCATE TABLE ProblemsMoreThan7BusinessDays
DECLARE #date AS date
DECLARE #businessday AS INT
DECLARE #Startdate as DATE, #EndDate as DATE
DECLARE CONTACT_CURSOR CURSOR FOR
SELECT date, businessday
FROM CalendarInformation
OPEN contact_cursor
FETCH NEXT FROM Contact_cursor INTO #date, #businessday
WHILE (##FETCH_STATUS=0)
BEGIN
SELECT #enddate= now FROM OpenProblemtimeDiffTable
SELECT #Startdate= problemnew FROM OpenProblemtimeDiffTable
SET #Date=#Startdate
PRINT #enddate
PRINT #startdate
SELECT #businessday= SUM (businessday) FROM CalendarInformation WHERE date > #startdate AND date <= #Enddate
INSERT INTO ProblemsMoreThan7BusinessDays (businessdays, number)
SELECT #businessday, number
FROM OpenProblemtimeDiffTable
FETCH NEXT FROM CONTACT_CURSOR INTO #date, #businessday
END
CLOSE CONTACT_CURSOR
DEALLOCATE CONTACT_CURSOR
I tried this code using a cursor and I'm close, but I cannot get the date ranges to change for each row.
So if I have a problemnumber with date ranges between 02-07-2018 and 05-20-2019, I would want in my new table the sum of business days from the calendar along with the problem number. So my output would be column number PROB0421 businessdays (with the correct sum). Then the next problem PRB0422 with date ranges of 11-6-18 to 5-20-19. So my output would be PROB0422 with the correct sum of business days.
Rather than doing this in with a cursor, you should approach this in a set based manner. That you already have a calendar table makes this a lot easier. The basic approach is to select from your data table and join into your calendar table to return all the rows in the calendar table that sit within your date range. From here you can then aggregate as you require.
This would look something like the below, though apply it to your situation and adjust as required:
select p.ProblemNow
,p.Now
,sum(c.BusinessDay) as BusinessDays
from dbo.Problems as p
join dbo.calendar as c
on c.CalendarDate between p.ProblemNow and p.Now
and c.BusinessDay = 1
group by p.ProblemNow
,p.Now
I think you can do this without a cursor. Should only require a single insert..select statement.
I assume your "businessday" column is just a bit or flag-type field that is 1 if the date is a business day and 0 if not? If so, this should work (or something close to it if I'm not understanding your environment properly).:
insert ProblemsMoreThan7BusinessDays
(
businessdays
, number
)
select
number
, sum( businessday ) -- or count(*)
from OpenProblemtimeDiffTable op
inner join CalendarInformation ci on op.problem_new >= ci.[date]
and op.[now] <= ci.[date]
and ci.businessday = 1
group by
problem_number
I usually try to avoid the use of cursors and working with data in a procedural manner, especially if I can handle the task as above. Dont think of the data as 1000's of individual rows, but think of the data as only two sets of data. How do they relate?
I have a requirement to display spend estimation for last 30 days. SpendEstimation is calculated multiple times a day. This can be achieved using simple SQL query:
SELECT DISTINCT ON (date) date(time) AS date, resource_id , time
FROM spend_estimation
WHERE
resource_id = '<id>'
and time > now() - interval '30 days'
ORDER BY date DESC, time DESC;
Unfortunately I can't seem to be able to do the same using SQLAlchemy. It always creates select distinct on all columns. Generated query does not contain distinct on.
query = session.query(
func.date(SpendEstimation.time).label('date'),
SpendEstimation.resource_id,
SpendEstimation.time
).distinct(
'date'
).order_by(
'date',
SpendEstimation.time
)
SELECT DISTINCT
date(time) AS date,
resource_id,
time
FROM spend
ORDER BY date, time
It is missing ON (date) bit. If I user query.group_by - then SQLAlchemy adds distinct on. Though I can't think of solution for given problem using group by.
Tried using function in distinct part and order by part as well.
query = session.query(
func.date(SpendEstimation.time).label('date'),
SpendEstimation.resource_id,
SpendEstimation.time
).distinct(
func.date(SpendEstimation.time).label('date')
).order_by(
func.date(SpendEstimation.time).label('date'),
SpendEstimation.time
)
Which resulted in this SQL:
SELECT DISTINCT
date(time) AS date,
resource_id,
time,
date(time) AS date # only difference
FROM spend
ORDER BY date, time
Which is still missing DISTINCT ON.
Your SqlAlchemy version might be the culprit.
Sqlalchemy with postgres. Try to get 'DISTINCT ON' instead of 'DISTINCT'
Links to this bug report:
https://bitbucket.org/zzzeek/sqlalchemy/issues/2142
A fix wasn't backported to 0.6, looks like it was fixed in 0.7.
Stupid question: have you tried distinct on SpendEstimation.date instead of 'date'?
EDIT: It just struck me that you're trying to use the named column from the SELECT. SQLAlchemy is not that smart. Try passing in the func expression into the distinct() call.
I'm modifying an existing query for a client, and I've encountered a somewhat baffling issue.
Our client uses SQL Server 2008 R2 and the database in question provides the user the ability to specify custom fields for one of its tables by making use of an EAV structure. All of the values stored in this structure are varchar(255), and several of the fields are intended to store dates. The query in question is being modified to use two of these fields and compare them (one is a start, the other is an end) against the current date to determine which row is "current".
The issue I'm having is that part of the query does a CONVERT(DateTime, eav.Value) in order to turn the varchar into a DateTime. The conversions themselves all succedd and I can include the value as part of the SELECT clause, but part of the question is giving me a conversion error:
Conversion failed when converting date and/or time from character string.
The real kicker is this: if I define the base for this query (getting a list of entities with the two custom field values flattened into a single row) as a view and select against the view and filter the view by getdate(), then it works correctly, but it fails if I add a join to a second table using one of the (non-date) fields from the view. I realize that this might be somewhat hard to follow, so I can post an example query if desired, but this question is already getting a little long.
I've tried recreating the basic structure in another database and including sample data, but the new database behaves as expected, so I'm at a loss here.
EDIT In case it's useful, here's the statement for the view:
create view Festival as
select
e.EntityId as FestivalId,
e.LookupAs as FestivalName,
convert(Date, nvs.Value) as ActivityStart,
convert(Date, nve.Value) as ActivityEnd
from tblEntity e
left join CustomControl ccs on ccs.ShortName = 'Activity Start Date'
left join CustomControl cce on cce.ShortName = 'Activity End Date'
left join tblEntityNameValue nvs on nvs.CustomControlId = ccs.IdCustomControl and nvs.EntityId = e.EntityId
left join tblEntityNameValue nve on nve.CustomControlId = cce.IdCustomControl and nve.EntityId = e.EntityId
where e.EntityType = 'Festival'
The failing query is this:
select *
from Festival f
join FestivalAttendeeAll fa on fa.FestivalId = f.FestivalId
where getdate() between f.ActivityStart and f.ActivityEnd
Yet this works:
select *
from Festival f
where getdate() between f.ActivityStart and f.ActivityEnd
(EntityId/FestivalId are int columns)
I've encountered this type of error before, it's due to the "order of operations" performed by the execution plan.
You are getting that error message because the execution plan for your statement (generated by the optimizer) is performing the CONVERT() operation on rows that contain string values that can't be converted to DATETIME.
Basically, you do not have control over which rows the optimizer performs that conversion on. You know that you only need that conversion done on certain rows, and you have predicates (WHERE or ON clauses) that exclude those rows (limit the rows to those that need the conversion), but your execution plan is performing the CONVERT() operation on rows BEFORE those rows are excluded.
(For example, the optimizer may be electing to a do a table scan, and performing that conversion on every row, before any predicate is being applied.)
I can't give a specific answer, without a specific question and specific SQL that is generating the error.
One simple approach to addressing the problem would be to use the ISDATE() function to test whether the string value can be converted to a date.
That is, replace:
CONVERT(DATETIME,eav.Value)
with:
CASE WHEN ISDATE(eav.Value) > 0 THEN CONVERT(DATETIME, eav.Value) ELSE NULL END
or:
CONVERT(DATETIME, CASE WHEN ISDATE(eav.Value) > 0 THEN eav.Value ELSE NULL END)
Note that the ISDATE() function is subject to some significant limitations, such as being affected by the DATEFORMAT and LANGUAGE settings of the session.
If there is some other indication on the eav row, you could use some other test, to conditionally perform the conversion.
CASE WHEN eav.ValueIsDateTime=1 THEN CONVERT(DATETIME, eav.Value) ELSE NULL END
The other approach I've used is to try to gain some modicum of control over the order of operations of the optimizer, using inline views or Common Table Expressions, with operations that force the optimizer to materialize them and apply predicates, so that happens BEFORE any conversion in the outer query.