I'm looking at this example provided by MS as I'm trying to learn Try...Catch. I understand the syntax and Output (for the most part) but I have one question:
The Output will show the Error_Line as '4'. This is fine but if I remove the line break between GO and BEGIN TRY it'll show the Error_Line as '3'. I just want to understand the logic here.
What I imagine is happening is that SQL Server is counting the lines by beginning the batch immediately after GO, even if that line is blank but I do not know this for certain. Can anyone clarify? If that theory is correct, wouldn't that make finding errors difficult if scripts are written with line breaks like this?
-- Verify that the stored procedure does not already exist.
IF OBJECT_ID ( 'usp_GetErrorInfo', 'P' ) IS NOT NULL
DROP PROCEDURE usp_GetErrorInfo;
GO
-- Create procedure to retrieve error information.
CREATE PROCEDURE usp_GetErrorInfo
AS
SELECT
ERROR_NUMBER() AS ErrorNumber
,ERROR_SEVERITY() AS ErrorSeverity
,ERROR_STATE() AS ErrorState
,ERROR_PROCEDURE() AS ErrorProcedure
,ERROR_LINE() AS ErrorLine
,ERROR_MESSAGE() AS ErrorMessage;
GO
--Line 1
BEGIN TRY --Line 2
-- Generate divide-by-zero error. --Line 3
SELECT 1/0; --Line 4
END TRY
BEGIN CATCH
-- Execute error retrieval routine.
EXECUTE usp_GetErrorInfo;
END CATCH;
You can't really rely on ERROR_LINE(), especially when the error is thrown in internal stored procedure or there is dynamic T-SQL statement which is executed.
But do you really need the exact error line?
in real production code, the fix for the line causing the error may not be so obvious as in your example;
it will be better to debug the stored procedure or the function with the corresponding input parameter in order to reproduce the error
In this way it will be easier to fix an issue. In order to debug a SQL routine:
just script it
remove the drop and create stuff
add declare in front of the input parameters and initialized them with the values causing the error
Basically, instead of the exact error line (which can be easily fine having the correct input parameters and executing the routine) you may found useful two things:
which routing is causing the error (for example, you can add additional parameter to user usp_GetErrorInfo SP which is yielding the SP name as well
the input parameters which are causing the error (this can be done using separated table for logging the errors in the CATCH clause - you simple insert the input parameters in the table and information about the error)
Having this information, it will be easy to reproduce and then fix an issue (in many cases).
Related
I have a Python script that runs a pgSQL file through SQLAlchemy's connection.execute function. Here's the block of code in Python:
results = pg_conn.execute(sql_cmd, beg_date = datetime.date(2015,4,1), end_date = datetime.date(2015,4,30))
And here's one of the areas where the variable gets inputted in my SQL:
WHERE
( dv.date >= %(beg_date)s AND
dv.date <= %(end_date)s)
When I run this, I get a cryptic python error:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) argument formats can't be mixed
…followed by a huge dump of the offending SQL query. I've run this exact code with the same variable convention before. Why isn't it working this time?
I encountered a similar issue as Nikhil. I have a query with LIKE clauses which worked until I modified it to include a bind variable, at which point I received the following error:
DatabaseError: Execution failed on sql '...': argument formats can't be mixed
The solution is not to give up on the LIKE clause. That would be pretty crazy if psycopg2 simply didn't permit LIKE clauses. Rather, we can escape the literal % with %%. For example, the following query:
SELECT *
FROM people
WHERE start_date > %(beg_date)s
AND name LIKE 'John%';
would need to be modified to:
SELECT *
FROM people
WHERE start_date > %(beg_date)s
AND name LIKE 'John%%';
More details in the pscopg2 docs: http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries
As it turned out, I had used a SQL LIKE operator in the new SQL query, and the % operand was messing with Python's escaping capability. For instance:
dv.device LIKE 'iPhone%' or
dv.device LIKE '%Phone'
Another answer offered a way to un-escape and re-escape, which I felt would add unnecessary complexity to otherwise simple code. Instead, I used pgSQL's ability to handle regex to modify the SQL query itself. This changed the above portion of the query to:
dv.device ~ E'iPhone.*' or
dv.device ~ E'.*Phone$'
So for others: you may need to change your LIKE operators to regex '~' to get it to work. Just remember that it'll be WAY slower for large queries. (More info here.)
For me it's turn out I have % in sql comment
/* Any future change in the testing size will not require
a change here... even if we do a 100% test
*/
This works fine:
/* Any future change in the testing size will not require
a change here... even if we do a 100pct test
*/
The problem
I've got a SQLRPGLE program that executes queries that look like this:
SELECT orapdt, oraptm, orodr#, c.ccctls, orbill, b.cuslmn, b.cusvrp, orocty, orost, o.cubzip, o.cucnty, ordcty, ordst, d.cubzip, d.cucnty
FROM order
LEFT JOIN cmtctlf c ON orbill = c.cccode
LEFT JOIN custmast b ON orbill = b.cucode
LEFT JOIN custmast o ON orldat = o.cucode
LEFT JOIN custmast d ON orcons = d.cucode
WHERE
orstat != 'C' AND
orbill IN ('ABCDE', 'VWXYZ', 'JKFRTE') AND
orapdt BETWEEN 2012365 AND 2013362 AND
o.cucnty = 'USA' AND
(o.cubzip LIKE '760%' OR o.cubzip LIKE '761%' OR o.cubzip LIKE '762%') AND
d.cubzip = '38652' AND
ordcty = 'NA' AND
ordst = 'MS' AND
d.cucnty = 'USA'
ORDER BY orapdt, oraptm, orodr#
Field definitions:
orapdt 7 0
oraptm 4a
orodr# 7a
c.ccctls 6a
orbill 6a
b.cuslmn 2a
b.cusvrp 3a
orocty 4a
orost 2a
o.cubzip 5a
o.cucnty 3a
ordcty 4a
ordst 2a
d.cubzip 5a
d.cucnty 3a
c.cccode 6a
b.cucode 6a
o.cucode 6a
d.cucode 6a
I see the following errors in my job log:
Field HVR0001 and value 1 not compatible. Reason 7.
Conversion error on host variable or parameter *N.
When I prompt for additional message information I'm told:
The attributes of variable field HVR0001 in query record format FORMAT0001 are not compatible with the attributes of value number 1. The value is *N. The reason code is 7.
7 -- Value contains numeric data that is not valid
and
Host variable or parameter *N or entry 1 in a descriptor area contains a value that cannot be converted to the attributes required by the statement. Error type 6 occurred.
6 -- Numeric data that is not valid.
These errors are triggered by opening the cursor:
...
exec sql PREPARE S1 FROM :sql_stmt;
exec sql DECLARE C1 SCROLL CURSOR FOR S1;
exec sql OPEN C1;
...
I also have QSQSVCDMP files in my outq filled with dump information. The only useful thing I see in there is a reference to CPF4278 and CPD4374
CPF4278 means Query definition template &1 not valid.
CPD4374 means Field &1 and value &3 not compatible. Reason &5.
Unfortunately the error message itself isn't there, only the strings "CPF4278" and "CPD4374".
In the program I monitor for SQL error codes and they are all the same:
SQLSTATE: 22023
SQLCODE: -302
SQLERRMC: <non-displayable character>*N
The error state/code means "A parameter or variable value is invalid."
What I've tried...
After much Googling I've tried:
removing the ORDER BY clause (on OPEN, data is fetched and
ordered when there is an ORDER BY clause)
changing all LEFT JOIN's to INNER JOIN's (did this to make sure there were no NULL's
in the result records from the right side)
adding " AND orapdt IS NOT NULL" to the WHERE clause
many more things that I've forgotten
What I'm asking...
How do I find out which field has bad data in it? I know that HVR0001 is invalid but which field is represented by HVR0001? I tried SELECTing fields in a different order but it's always HVR0001 that has an invalid value.
Ideally I'd like to be able to print out all HVR* fields/values so I can inspect them.
When I look at the compile listing there are no HVR* fields listed. There are some SQL_* fields listed and I can see that SQL_00011 is used to temporarily hold data that gets put into orapdt. SQL_00011 is defined exactly like orapdt (7,0 packed). That's the only numeric field in my query...
I feel like my problem is being caused by how the files are being joined, that somehow an invalid value (probably NULL) is being placed into my orapdt field.
I also think my problem has something to do with executing many of these queries one after the other (some of the WHERE specifics change for each query) because I can take one of the queries that fail and put it into it's own program and run it and it works fine.
This is on DB2 for i (V6R1) and all files involved were created using DDS
Edit:
Here is the host variable (data structure) and the two external data structures needed for the LIKE statements:
d eds_custmast e ds extname('CUSTMAST') inz
d eds_order e ds extname('ORDER') inz
d o ds
d orapdt like(ORAPDT)
d oraptm like(ORAPTM)
d orodr# like(ORODR#)
d orctls like(CUCODE)
d orbill like(ORBILL)
d orslmn like(CUSLMN)
d orcsr like(CUSVRP)
d orocty like(OROCTY)
d orost like(OROST)
d orozip like(CUBZIP)
d orocntry like(CUCNTY)
d ordcty like(ORDCTY)
d ordst like(ORDST)
d ordzip like(CUBZIP)
d ordcntry like(CUCNTY)
// Define an array to indicate nulls...
d o1nv s 3i 0 dim(15)
And here's the fetch statement that actually gets the data:
dow sqlcode = *zeros;
exec sql FETCH NEXT FROM C1 INTO :o :o1nv;
if sqlcode = *zeros;
// process the data.
endif;
enddo;
exec sql CLOSE C1;
I didn't include this before simply because the error occurs when I'm OPENing the cursor, not FETCHing a row. The OPEN statement shouldn't know anything about the o data structure.
As for what changes in the WHERE clause - all of it is dynamically built (and thus can change) other than:
orstat != 'C' AND orapdt BETWEEN 2012365 AND 2013362
It's not at all easy to find out what the actual error is. I tend to copy statements like these into IBM i Navigator and use Visual Explain to try to get a grasp of what decisions the optimiser is making. Another way to do this is to do a STRDBG and look at the job log. When STRDBG is in effect, the optimiser puts informational messages into the job log. But even then, it cam be tough to puzzle out.
In this case, there's only one numeric column, orapdt. Try the query without that column and see if that's the culprit.
Since ORAPDT is your only numeric column, so the problem must lie there.
The issue is in the way DDS defined files work. The validity of values is not checked when being written into DDS defined files, so it appears you have non-numeric data in ORAPDT on one or more records. SQL does not like this, and throws an error.
SQL (DDL) defined tables validate the values before they are written, thus protecting the integrity of your database better.
To solve your problem, find the offending record(s) and fix them or delete them.
Assuming error comes from orapdt, you could monitor it by creating new varible or replacing null or garbage values with other number e.g. null = 9999999, non-numeric = 8888888
SELECT case when orapdt is null
then 9999999
when TRANSLATE(SUBSTR(orapdt,1,LENGTH(orapdt)-1),' ','0123456789',' ') <>' '
then 8888888
else orapdt
end
, oraptm,
or check thru strsql or run sql script for offending records
SELECT orapdt, oraptm, orodr#,
...
WHERE ( orapdt is null or TRANSLATE(SUBSTR(orapdt,1,LENGTH(orapdt)-1),' ','0123456789',' ') <>' ' ) AND
orstat != 'C' AND
......
What seems to be the issue...
The code I posted in my question is in program A. Program A calls (via CALLP) program B. Nothing out of the ordinary there.
Program A uses embedded SQL declaring a prepared statement called S1 and a scrollable cursor called C1. Program B also happens to declare a prepared statement called S1 and a scrollable cursor called C1.
What appears to be happening is the cursor's are interfering with each other because they have the same name. My belief is the query being executed in program B is fetching data that is valid for itself – but is invalid for the query defined in program A. So when program A scrolls through the results of it's query and calls program B the query executed by program B attempts to put invalid values in fields associated with program A – and this only happens when the cursor names are the same in both programs.
All I did was give the cursors in both programs unique names (PGMA_C1 and PGMB_C1 for instance) and the errors stopped happening. Nothing else changed, just the cursor names. This goes against the information I found here (http://pic.dhe.ibm.com/infocenter/iseries/v6r1m0/index.jsp?topic=/rzala/rzalaccl.htm)
“Scope of a cursor: The scope of cursor-name is the source program in which it is defined; that is, the program submitted to the precompiler. Thus, a cursor can only be referenced by statements that are precompiled with the cursor declaration. For example, a program called from another separately compiled program cannot use a cursor that was opened by the calling program.”
Of course that statement seems to be contradicted by this one:
A cursor can only be referred to in the same instance of the program
in the program stack unless CLOSQLCSR(*ENDJOB), CLOSQLCSR(*ENDSQL), or
CLOSQLCSR(*ENDACTGRP) is specified on the CRTSQLxxx commands.
If CLOSQLCSR(*ENDJOB) is specified, the cursor can be referred to by any instance of the program on the program stack.
If CLOSQLCSR(*ENDSQL) is specified, the cursor can be referred to by any instance of the program on the program stack until the last
SQL program on the program stack ends.
If CLOSQLCSR(*ENDACTGRP) is specified, the cursor can be referred to by all instances of the module in the activation group until the
activation group ends.
But in our case both program A and B have CLOSQLCSR(*ENDMOD) – so the two cursors shouldn't be aware of each other.
Unfortunately I don't have the time to dig into this any deeper. I have confirmed that simply giving each program a unique cursor name solves our problem.
Before I figured out that using unique cursor names would fix our problem I did comprehensive testing of all our data. Every field in every record in every file used by these two programs contains valid data. Based on the error message I was expecting there to be a NULL or some other invalid character somewhere but that wasn't the case.
I appreciate your replies and suggestions, +1 all around :-)
I want to use T-SQL to perform a sequence of merges. I understand that if one fails, it rolls back, but I would like to print a message to the effect - so I know I need to go and check it. I can't seem to find any examples of this to start with. My first thought was to put the the transaction in a try, but then I'm not sure what happens with the rollback statement ... it seems weird putting that in the catch. I'm sure others must have had wanted to do this previously. Does anyone have an example of this?
NOTE: I need to use the MERGE command.
Try below code. Try Catch help from msdn
BEGIN TRY
-- Table does not exist; object name resolution
-- error not caught.
SELECT * FROM Table;
END TRY
BEGIN CATCH
Print ERROR_NUMBER() ;
Print ERROR_MESSAGE() ;
END CATCH
You can use OUTPUT with a merge. Sample
I had been using MySQL as database and had planned to move to postgresql. I had used aes_encrypt and aes_decrypt functions in MySQL extensively throughout my application. So whenever the encryption/decrytion fails, MySQL automatically returns 'null'.
I am unsure how to handle the same in postgresql. Tried using the pgp_sym_encrypt/pgp_sym_decrypt functions. If the encryption key is wrong, it throws error "Wrong key/corrupt data". I tried searching for some functions that could capture this error and return 'null' as in MySQL so that I need not modify my code. I had been searching but could not find one.
Has anybody used any error handling mechanism for individual queries? I had found that error handling can be done for procedures. But, I had to completely rewrite the entire application for that.
If you could share some details, it would be of great help. Thanks.
If you wish to avoid modifying your code and have the functions return NULL on error, you can do this by wrapping them in a PL/PgSQL function that uses a BEGIN ... EXCEPTION block to trap the error.
To do this, first I get the SQLSTATE for the error:
regress=# \set VERBOSITY verbose
regress=# SELECT pgp_sym_decrypt('fred','key');
ERROR: 39000: Wrong key or corrupt data
LOCATION: decrypt_internal, pgp-pgsql.c:607
I could use this directly in the error handler, but I prefer to use a symbolic name, so I look up the error name associated with 39000 in Appendix A - Error codes, finding that it's the generic function call error external_routine_invocation_exception. Not as specific as we would've liked, but it'll do.
Now a wrapper function is required. Something like this must be defined, with one function for each overloaded signature of pgp_sym_decrypt that you wish to support. For the (bytea,text) form that returns text, for example:
CREATE OR REPLACE FUNCTION pgp_sym_decrypt_null_on_err(data bytea, psw text) RETURNS text AS $$
BEGIN
RETURN pgp_sym_decrypt(data, psw);
EXCEPTION
WHEN external_routine_invocation_exception THEN
RAISE DEBUG USING
MESSAGE = format('Decryption failed: SQLSTATE %s, Msg: %s',
SQLSTATE,SQLERRM),
HINT = 'pgp_sym_encrypt(...) failed; check your key',
ERRCODE = 'external_routine_invocation_exception';
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
I've chosen to preseve the original error in a DEBUG level message. Here's a comparison of the original and wrapper, with full message verbosity and debug level output.
Enable debug output to show the RAISE. Note that it also shows the *original query text of the pgp_decrypt_sym call, including parameters.
regress=# SET client_min_messages = DEBUG;
New wrapped function still reports the error if detailed logging is enabled, but returns NULL:
regress=# SELECT pgp_sym_decrypt_null_on_err('redsdfsfdsfd','bobsdf');
LOG: 00000: statement: SELECT pgp_sym_decrypt_null_on_err('redsdfsfdsfd','bobsdf');
LOCATION: exec_simple_query, postgres.c:860
DEBUG: 39000: Decryption failed: SQLSTATE 39000, Msg: Wrong key or corrupt data
HINT: pgp_sym_encrypt(...) failed; check your key
LOCATION: exec_stmt_raise, pl_exec.c:2806
pgp_sym_decrypt_null_on_err
-----------------------------
(1 row)
compared to the original, which fails:
regress=# SELECT pgp_sym_decrypt('redsdfsfdsfd','bobsdf');
LOG: 00000: statement: SELECT pgp_sym_decrypt('redsdfsfdsfd','bobsdf');
LOCATION: exec_simple_query, postgres.c:860
ERROR: 39000: Wrong key or corrupt data
LOCATION: decrypt_internal, pgp-pgsql.c:607
Note that both forms show the parameters the function was called with when it failed. The parameters won't be shown if you've used bind parameters ("prepared statements"), but you should still consider your logs to be security critical if you're using in-database encryption.
Personally, I think it's better to do crypto in the app, so the DB never has access to the keys.
I am using Sybase ASE 12.5 at the moment. I have a code below:
create procedure test_launcher_fail_wrapper
as
begin
select convert(numeric(2),1234345)
if ##error != 0
begin
select "SP failed to execute"
return 1
end
end
Here, I am trying to convert a very large value/amount (1234345) to Numeric size 2. Which is not possible and it generates error.
Questions:
Is having ##error useful here? I ran this SP and it never went into
error handling
How to error handle these kind of scenarios?
I treat error handling in procs similarly to error handling in applications -- if there's an opportunity for you to contribute some actual value by handling the error, then by all means, do so, but if you can't really do anything to help, then you're better off just letting it go.
As an example of adding value, I've got one or two procs that add contextual information in the error message, like a list of ID values that conflict with an update operation. In this particular case, I know that the upstream consumer of the proc will log this error, and the text will be available to an operator who will find this information valuable when debugging the problem. I also know that while this condition is a real error, it's been known to happen from time-to-time, and the effort to format the error is worthwhile.
Does this catch your error?
create procedure test_launcher_fail_wrapper
as
begin
declare #database_err int
set #database_err = 0
select convert(numeric(2),1234345)
set #database_err = ##error
if #database_err <> 0
begin
PRINT 'SP failed to execute'
return 1
end
end
##error is the way to go but beware since:
Every Transact-SQL statement, including print statements and if tests, resets ##error, so the status check must immediately follow the batch for which success is in question.
As for a suggestion on how to handle error management in similar scenarios, have you considered using raiserror ?
An example:
create procedure proc1 as
begin
select convert(numeric(2),1234345)
if ##error <> 0
begin
raiserror 20001 "Error during convert in proc1"
return 1
end
end