I had been using MySQL as database and had planned to move to postgresql. I had used aes_encrypt and aes_decrypt functions in MySQL extensively throughout my application. So whenever the encryption/decrytion fails, MySQL automatically returns 'null'.
I am unsure how to handle the same in postgresql. Tried using the pgp_sym_encrypt/pgp_sym_decrypt functions. If the encryption key is wrong, it throws error "Wrong key/corrupt data". I tried searching for some functions that could capture this error and return 'null' as in MySQL so that I need not modify my code. I had been searching but could not find one.
Has anybody used any error handling mechanism for individual queries? I had found that error handling can be done for procedures. But, I had to completely rewrite the entire application for that.
If you could share some details, it would be of great help. Thanks.
If you wish to avoid modifying your code and have the functions return NULL on error, you can do this by wrapping them in a PL/PgSQL function that uses a BEGIN ... EXCEPTION block to trap the error.
To do this, first I get the SQLSTATE for the error:
regress=# \set VERBOSITY verbose
regress=# SELECT pgp_sym_decrypt('fred','key');
ERROR: 39000: Wrong key or corrupt data
LOCATION: decrypt_internal, pgp-pgsql.c:607
I could use this directly in the error handler, but I prefer to use a symbolic name, so I look up the error name associated with 39000 in Appendix A - Error codes, finding that it's the generic function call error external_routine_invocation_exception. Not as specific as we would've liked, but it'll do.
Now a wrapper function is required. Something like this must be defined, with one function for each overloaded signature of pgp_sym_decrypt that you wish to support. For the (bytea,text) form that returns text, for example:
CREATE OR REPLACE FUNCTION pgp_sym_decrypt_null_on_err(data bytea, psw text) RETURNS text AS $$
BEGIN
RETURN pgp_sym_decrypt(data, psw);
EXCEPTION
WHEN external_routine_invocation_exception THEN
RAISE DEBUG USING
MESSAGE = format('Decryption failed: SQLSTATE %s, Msg: %s',
SQLSTATE,SQLERRM),
HINT = 'pgp_sym_encrypt(...) failed; check your key',
ERRCODE = 'external_routine_invocation_exception';
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
I've chosen to preseve the original error in a DEBUG level message. Here's a comparison of the original and wrapper, with full message verbosity and debug level output.
Enable debug output to show the RAISE. Note that it also shows the *original query text of the pgp_decrypt_sym call, including parameters.
regress=# SET client_min_messages = DEBUG;
New wrapped function still reports the error if detailed logging is enabled, but returns NULL:
regress=# SELECT pgp_sym_decrypt_null_on_err('redsdfsfdsfd','bobsdf');
LOG: 00000: statement: SELECT pgp_sym_decrypt_null_on_err('redsdfsfdsfd','bobsdf');
LOCATION: exec_simple_query, postgres.c:860
DEBUG: 39000: Decryption failed: SQLSTATE 39000, Msg: Wrong key or corrupt data
HINT: pgp_sym_encrypt(...) failed; check your key
LOCATION: exec_stmt_raise, pl_exec.c:2806
pgp_sym_decrypt_null_on_err
-----------------------------
(1 row)
compared to the original, which fails:
regress=# SELECT pgp_sym_decrypt('redsdfsfdsfd','bobsdf');
LOG: 00000: statement: SELECT pgp_sym_decrypt('redsdfsfdsfd','bobsdf');
LOCATION: exec_simple_query, postgres.c:860
ERROR: 39000: Wrong key or corrupt data
LOCATION: decrypt_internal, pgp-pgsql.c:607
Note that both forms show the parameters the function was called with when it failed. The parameters won't be shown if you've used bind parameters ("prepared statements"), but you should still consider your logs to be security critical if you're using in-database encryption.
Personally, I think it's better to do crypto in the app, so the DB never has access to the keys.
Related
Common sense dictates that SQL query strings should never be assembled by hand. Thus, all database interfaces offer parameter substitution, and all users use it, without exceptions.*
I'm using PostgreSQL v10.5, nodejs v8.12.0, node-postgres 7.6.1.
Parameter substitution works as expected for SELECT statements:
> await db.query("select from users where id = 'mic'");
(success, 1 row returned)
> await db.query("select from users where id = $1", ["mic"]);
(success, 1 row returned)
But it doesn't work for LISTEN statements:
> await db.query("listen topicname");
(success)
> await db.query("listen $1", ["topicname"]);
(error: syntax error at or near "$1")
The name of the topic I want to listen to is dynamic. It is coming from semi-trustworthy sources, which should not be user-controllable. But why go against all established best practice and take any chances?
Unfortunately, from my tests I fear that PostgreSQL simply can't do parameter substitution for LISTEN queries.
Is there any solution or workaround for this?
*) This statement may only be true in some utopic future society.
I don't have enough reputation to comment on the answer, but the proposed solution doesn't work for me.
Using %L results in a quoted string, which causes the following error:
ERROR: syntax error at or near "'topic'"
The %I format should be used instead (SQL identifier, this is documented for table and column names, but it also works for the channel name,). You can also use the quote_ident function. See the documentation on creating dynamic queries here.
The following PL/pgSQL function works for us:
CREATE OR REPLACE FUNCTION listenForChannel(
channel_ TEXT
) RETURNS VOID AS $$
BEGIN
EXECUTE format('LISTEN %I', channel_);
END
$$ LANGUAGE PLPGSQL;
You are right that this cannot be done in PostgreSQL.
As a workaround, write a PL/pgSQL function that uses dynamic SQL like this:
EXECUTE format('LISTEN %L', topicname);
The format function escapes strings properly; in this case, the %L format that produces a properly quoted string Literal is the appropriate one.
I am struggling to figure out how to best handle the return of results or errors to my application from Postgres stored functions.
Consider the following contrived psudeocode example:
app.get_resource(_username text)
RETURNS <???>
BEGIN
IF ([ ..user exists.. ] = FALSE) THEN
RETURN 'ERR_USER_NOT_FOUND';
END IF;
IF ([ ..user has permission.. ] = FALSE) THEN
RETURN 'ERR_NO_PERMISSION';
END IF;
-- Return the full user object.
RETURN QUERY( SELECT 1
FROM app.resources
WHERE app.resources.owner = _username);
END
The function can fail with a specific error or succeed and return 0 or more resources.
At first I tried creating a custom type to always use as a standard return type in eachh function:
CREATE TYPE app.appresult AS (
success boolean,
error text,
result anyelement
);
Postgres does not allow this however:
[42P16] ERROR: column "result" has pseudo-type anyelement
I then discovered OUT parameters and attempted the following uses:
CREATE OR REPLACE FUNCTION app.get_resource(
IN _username text,
OUT _result app.appresult -- Custom type
-- {success bool, error text}
)
RETURNS SETOF record
AS
$$
BEGIN
IF 1 = 1 THEN -- just a test
_result.success = false;
_result.error = 'ERROR_ERROR';
RETURN NULL;
END IF;
RETURN QUERY(SELECT * FROM app.resources);
END;
$$
LANGUAGE 'plpgsql' VOLATILE;
Postgres doesn't like this either:
[42P13] ERROR: function result type must be app.appresult because of OUT parameters
Also tried a similar function but reversed: Returning a custom app.appresult object and setting the OUT param to "SETOF RECORD". This was also not allowed.
Lastly i looked into Postgres exception handling using
RAISE EXCEPTION 'ERR_MY_ERROR';
So in the example function, i'd just raise this error and return.
This resulted in the driver sending back the error as:
"ERROR: ERR_MY_ERROR\nCONTEXT: PL/pgSQL function app.test(text) line 6 at RAISE\n(P0001)"
This is easy enough to parse but doing things this way feels wrong.
What is the best way to solve this problem?
Is it possible to have a custom AppResult object that i could return?
Something like:
{ success bool, error text, result <whatever type> }
//Edit 1 //
I think I'm leaning more towards #Laurenz Albe solution.
My main goal is simple: Call a stored procedure which can return either an error or some data.
Using RAISE seems to accomplish this and the C++ driver allows easy checking for an error condition returned from a query.
if ([error code returned from the query] == 90100)
{
// 1. Parse out my overly verbose error from the raw driver
// error string.
// 2. Handle the error.
}
I'm also wondering about using custom SQLSTATE codes instead of parsing the driver string.
Throwing '__404' might mean that during the course of my SPs execution, it could not continue because some record needed was not found.
When calling the sql function from my app, i have a general idea of what it failing with a '__404' would mean and how to handle it. This avoids the additional step of parsing driver error string.
I can also see the potential of this being a bad idea.
Bedtime reading:
https://www.postgresql.org/docs/current/static/errcodes-appendix.html
This is slightly opinion based, but I think that throwing an error is the best and most elegant solution. That is what errors are for!
To distinguish various error messages, you could use SQLSTATEs that start with 6, 8 or 9 (these are not used), then you don't have to depend on the wording of the error message.
You can raise such an error with
RAISE EXCEPTION SQLSTATE '90001' USING MESSAGE = 'my own error';
We do something similar to what you're trying to do, but we use TEXT rather than ANYELEMENT, because (almost?) any type can be cast to TEXT and back. So our type looks something like:
(errors our_error_type[], result TEXT)
The function which returns this stores errors in the errors array (it's just some custom error type), and can store the result (cast to text) in the result field.
The calling function knows what type it expects, so it can first check the errors array to see if any errors were returned, and if not it can cast the result value to the expected return type.
As a general observation, I think exceptions are more elegant (possibly because I come from a c# background). The only problem is in plpgsql exception handling is (relatively) slow, so it depends on the context - if you're running something many times in a loop, I would prefer a solution that doesn't use exception handling; if it's a single call, and/or especially when you want it to abort, I prefer raising an exception. In practice we use both at various points throughout our call stacks.
And as Laurenz Albe pointed out, you're not meant to "parse" exceptions, so much as raise an exception with specific values in specific fields, which the function that catches the exception can then extract and act on directly.
As an example:
Setup:
CREATE TABLE my_table (id INTEGER, txt TEXT);
INSERT INTO my_table VALUES (1,'blah');
CREATE TYPE my_type AS (result TEXT);
CREATE OR REPLACE FUNCTION my_func()
RETURNS my_type AS
$BODY$
DECLARE
m my_type;
BEGIN
SELECT my_table::TEXT
INTO m.result
FROM my_table;
RETURN m;
END
$BODY$
LANGUAGE plpgsql STABLE;
Run:
SELECT (m.result::my_table).*
FROM my_func() AS m
Result:
| id | txt |
-------------
| 1 | blah |
I'm looking at this example provided by MS as I'm trying to learn Try...Catch. I understand the syntax and Output (for the most part) but I have one question:
The Output will show the Error_Line as '4'. This is fine but if I remove the line break between GO and BEGIN TRY it'll show the Error_Line as '3'. I just want to understand the logic here.
What I imagine is happening is that SQL Server is counting the lines by beginning the batch immediately after GO, even if that line is blank but I do not know this for certain. Can anyone clarify? If that theory is correct, wouldn't that make finding errors difficult if scripts are written with line breaks like this?
-- Verify that the stored procedure does not already exist.
IF OBJECT_ID ( 'usp_GetErrorInfo', 'P' ) IS NOT NULL
DROP PROCEDURE usp_GetErrorInfo;
GO
-- Create procedure to retrieve error information.
CREATE PROCEDURE usp_GetErrorInfo
AS
SELECT
ERROR_NUMBER() AS ErrorNumber
,ERROR_SEVERITY() AS ErrorSeverity
,ERROR_STATE() AS ErrorState
,ERROR_PROCEDURE() AS ErrorProcedure
,ERROR_LINE() AS ErrorLine
,ERROR_MESSAGE() AS ErrorMessage;
GO
--Line 1
BEGIN TRY --Line 2
-- Generate divide-by-zero error. --Line 3
SELECT 1/0; --Line 4
END TRY
BEGIN CATCH
-- Execute error retrieval routine.
EXECUTE usp_GetErrorInfo;
END CATCH;
You can't really rely on ERROR_LINE(), especially when the error is thrown in internal stored procedure or there is dynamic T-SQL statement which is executed.
But do you really need the exact error line?
in real production code, the fix for the line causing the error may not be so obvious as in your example;
it will be better to debug the stored procedure or the function with the corresponding input parameter in order to reproduce the error
In this way it will be easier to fix an issue. In order to debug a SQL routine:
just script it
remove the drop and create stuff
add declare in front of the input parameters and initialized them with the values causing the error
Basically, instead of the exact error line (which can be easily fine having the correct input parameters and executing the routine) you may found useful two things:
which routing is causing the error (for example, you can add additional parameter to user usp_GetErrorInfo SP which is yielding the SP name as well
the input parameters which are causing the error (this can be done using separated table for logging the errors in the CATCH clause - you simple insert the input parameters in the table and information about the error)
Having this information, it will be easy to reproduce and then fix an issue (in many cases).
I am trying to call a stored procedure with Time variable as in parameter. But whenever i try to call the procedure i m getting error as:
db2 'call PASS_FAIL_CHECKDATE('2014-01-21','13:42:25','CSS1',Null,'4500096651','10',Null)'
SQL0104N An unexpected token ":42" was found following "CKDATE(2014-01-21,
13". Expected tokens may include: "+". SQLSTATE=42601
My Procedures input parameter are :
PASS_fail_checkdate (in post_date date,in post_time time,in destplant varchar(4), in destloc varchar(4), in transnum varchar(10), in translineitemnum varchar(6), in inboundconsignment varchar(35))
I am not sure if my declaration for time variable is correct in procedure or if i am calling the time variable correctly in the procedure.
Please give me suggestions on the same.
As already offered, in words, try the following example as a revision to what was noted in the OP as tried already but failing; i.e. change to use double-quote vs the apostrophe, specified as the delimiter for the DB2 SQL statement string:
db2 "call PASS_FAIL_CHECKDATE('2014-01-21','13:42:25','CSS1',Null,'4500096651','10',Null)"
Though I have been using SQL Server, Oracle from last decade, I have been asked to
do some research on PostgreSQL and after some initial investigation it is evident that I am now stuck on retrieving data from the PostgreSQL database using Function.
Using following piece of code to retrieve the data and getting error
('ERROR [26000] ERROR: prepared statement "mytabletest" does not exist;
'Error while executing the query)
Code Snippets
Dim oDBCommand As DbCommand = GetDBCommand(oConnectionType, "mytabletest", CommandType.StoredProcedure)
Dim dstResults As DataSet = GetDataSet(ConnectionTypes.ODBC, oDBCommand)
Public Function GetDataReader(dbType As ConnectionTypes, command As DbCommand) As DbDataReader
Try
Dim oConnection As DbConnection = GetDBConnection(dbType)
Dim oDBTransaction As DbTransaction = oConnection.BeginTransaction
command.Connection = oConnection
command.Transaction = oDBTransaction
'GETTING ERROR ON FOLLOWING LINE
'ERROR [26000] ERROR: prepared statement "mytabletest" does not exist;
'Error while executing the query
return command.ExecuteReader()
Catch ex As Exception
Throw ex
Finally
End Try
Return Nothing
End Function
Environement I am currently working on is following:-
32 Bit Machine.
Visual Studio 2010 + SP1
ODBC Prodiver: PostgreSQL Unicode 9.01.02.00
ADO.Net (System.Data.Odbc)
Please note that I am open to any suggestions i.e. if I am completely doing it wrong
OR partially etc. Please feel free to write.
In order to make it easier for you to create a same environment, please use following table/function definition.
--- Simple table to make things easier to understand. <br>
CREATE TABLE mytable
(
messagetypeid integer NOT NULL,
messagetype character varying(100) NOT NULL
)
-- Function to retrieve data. <br>
CREATE OR REPLACE FUNCTION mytabletest() <br>
RETURNS SETOF refcursor AS $$
DECLARE
ref1 refcursor;
BEGIN
OPEN ref1 FOR SELECT * FROM mytable;
RETURN NEXT ref1;
END;
$$ LANGUAGE plpgsql;
Please Note:
If I use <br>
Dim oDBCommand As DbCommand = GetDBCommand(oConnectionType, "SELECT * FROM mytable", CommandType.Text)
then system manages to retrieve information from the datbase without any issue, however, as I mentioned as soon we use "Function" it throws an exception.
During my failed efforts to search any solution from the internet someone mentioned that Table should be created with the lower case it so just for the sake of it I recreated with the lower case, however, problem persists.
I am unfamiliar with .net but I suspect you meant something more like:
GetDBCommand(oConnectionType, "SELECT myfunc()", CommandType.Text)
Or in the case of SETOF functions etc..
GetDBCommand(oConnectionType, "SELECT * FROM myfunc()", CommandType.Text)
PostgreSQL does not have 'stored procedures' per-ce. It does have functions and I believe that the client/server protocol has a method for preparing statements that can then be executed multiple times with different variables (to save on the cost of parsing the SQL), but this should be exposed via your client library.