We have the following script (script.sql):
BEGIN;
SET client_min_messages = warning;
\COPY foo_table FROM 'foo.csv' csv header DELIMITER ';';
\COPY bar_table FROM 'bar.csv' csv header DELIMITER ';';
COMMIT;
We launch this script in an ansible playbook with
community.postgresql.postgresql_db
- name: 'Restore SQL dump(s) on database(s)'
become: yes
become_user: 'postgres'
postgresql_db:
name: 'db'
target: 'script.sql'
state: 'restore'
How can we detect if the transaction fail with a rollback to get an error on the deployment ?
The common way is to use -v ON_ERROR_STOP=on on psql command line, or alternatively \set ON_ERROR_STOP on in the SQL script.
According to the documentation:
ON_ERROR_STOP
By default, command processing continues after an error. When this
variable is set to on, processing will instead stop immediately. In
interactive mode, psql will return to the command prompt; otherwise,
psql will exit, returning error code 3 to distinguish this case from
fatal error conditions, which are reported using error code 1. In
either case, any currently running scripts (the top-level script, if
any, and any other scripts which it may have in invoked) will be
terminated immediately. If the top-level command string contained
multiple SQL commands, processing will stop with the current command.
Related
I am new to Ansible and willing to write a small playbook that switches to another user, gets into Postgres data base, do some changes, exit the data base and switch back to the original user.
This is what I would do manually, but I want to put these commands in the playbook:
sudo su - postgres
psql postgres
DROP DATABASE scm;
CREATE DATABASE scm OWNER scm ENCODING 'UTF8';
\q ##This will quit the database
exit ##This will quit postgres user back to original user
I started writing it into a playbook but it seems to not work:
---
- name: TEST
hosts: master_servers
tasks:
- name: Delete DB
shell:
cmd: psql postgres | DROP DATABASE scm; | CREATE DATABASE scm OWNER scm ENCODING 'UTF8'; | \q | exit
become: yes
become_user: postgres
Here is the error I get:
fatal: [xx.xx.xx.xx]: FAILED! => {
"msg": "Failed to change ownership of the temporary files Ansible needs to create despite connecting as a privileged user. Unprivileged become user would be unable to read the file."
}
Thanks in advance!
I'd suggest that you use postgresql_db module (maintained by the Ansible Core Team).
You can create, delete, or otherwise manipulate PostgreSQL databases in an idempotent way (you should always refrain from using shell and cmd modules whenever possible, since they are not idempotent by design).
There is no problem with your ansible syntax but there is a flaw in shell command you're trying to execute.
| "pipe" connects the stdout of one command to the stdin of another, the way you're executing your command doesn't make any sense.
I would suggest creating a script and copying it over to achieve your goal.
I'm maintaining a psql script which I usually want to immediately abort with a non-zero exit code when any part of it fails.
Thus I'm considering to either place
\set ON_ERROR_STOP on
at the beginning, or to instruct users to run the script with
psql -v ON_ERROR_STOP=on -f my_script.sql
However, there is a part of the script that deliberately fails (and gets rolled back). As the script is for education and demonstration purposes, and as that part demonstrates a CONSTRAINT actually working as it should (by making a subsequent constraint-violating INSERT fail), I can't really "fix" that part to not fail, so the accepted answer from How to save and restore value of ON_ERROR_STOP? doesn't solve my problem.
Thus, I'd like to disable ON_ERROR_STOP before that part and restore the setting after the part. If I know that ON_ERROR_STOP is enabled in general, this is easy:
\set ON_ERROR_STOP off
BEGIN;
-- [ part of the script that purposfully fails ]
ROLLBACK;
\set ON_ERROR_STOP on
or
\unset ON_ERROR_STOP
BEGIN;
-- [ part of the script that purposefully fails ]
ROLLBACK;
\set ON_ERROR_STOP on
However, this blindly (re-)enables ON_ERROR_STOP, whether it was enabled before or not.
\set previous_ON_ERROR_STOP :ON_ERROR_STOP
\unset ON_ERROR_STOP
BEGIN;
-- [ part of the script that purposefully fails ]
ROLLBACK;
\set ON_ERROR_STOP :previous_ON_ERROR_STOP
works if ON_ERROR_STOP has previously been explicitly disabled (e.g., set to off) but fails if it was unset (and thus just implicitly disabled).
I'd like the script to remain backwards compatible to PostgreSQL 9.x, so I can't yet use the \if meta commands introduced in PostgreSQL 10.
I don't think you can do that.
What you could do, however, is to use a PL/pgSQL block to run the statement and catch and report the error, somewhat like this:
DO
$$BEGIN
INSERT INTO mytab VALUES (...);
EXCEPTION
WHEN integrity_constraint_violation THEN
RAISE NOTICE 'Caught error: %', SQLERRM;
END;$$;
That will report the error, but it won't cause psql to stop.
\set CACHE_ON_ERROR_STOP :ON_ERROR_STOP
SELECT CASE
WHEN :'CACHE_ON_ERROR_STOP' = ':CACHE_ON_ERROR_STOP' THEN 'off'
WHEN :'CACHE_ON_ERROR_STOP'::BOOLEAN is true THEN 'on'
ELSE 'off'
END::boolean AS CACHE_ON_ERROR_STOP
\gset
\set ON_ERROR_STOP off
-- your code here
\set ON_ERROR_STOP :CACHE_ON_ERROR_STOP
I have a very big SQL dump I'm working on. The overall structure looks like this:
BEGIN;
SET CONSTRAINTS ALL DEFERRED;
INSERT …;
-- … ~300K insert operations
INSERT …;
COMMIT;
The problem is if there is an error in any single statement, then error is shown and the message current transaction is aborted, commands ignored until end of transaction block is generated for EACH statement that follows it.
Why does it behave so weirdly? Is there a way to suppress the following messages? It's enough to just show the real error message and to skip transaction execution. I don't want to see ~300K meaningful error messages.
Do I need to structure my dump differently? Or is there a flag/option I can use?
Presumably you're using psql to send the queries to the server.
You may set the ON_ERROR_STOP built-in variable to on.
From https://www.postgresql.org/docs/current/static/app-psql.html:
ON_ERROR_STOP
By default, command processing continues after an error. When this
variable is set to on, processing will instead stop
immediately. In interactive mode, psql will return to the command
prompt; otherwise, psql will exit, returning error code 3 to
distinguish this case from fatal error conditions, which are
reported using error code 1.
It may be set from outside psql with psql -v ON_ERROR_STOP=on -f script.sql, or from inside the script or interactively with the meta-command \set ON_ERROR_STOP on.
pg_dump does not have an option to add this automatically to a dump (as far as I know, it doesn't emit any psql meta-command anyway, only pure SQL commands).
I've tried several different ways to get this to run, all unsuccessfully!
Currently, I have a groovy script that tries to execute the Postgresql (9.2) 'createdb' command like this:
def createDbCmdLine = "-p 5433 --encoding=UTF8 --template=template0 myDatabaseName"
ant.exec(executable:'fullpath/bin/createdb') {
arg(line: "$createDbCmdLine")
env(key:"PGPASSWORD", value:"myPassword")
}
However, this just hangs forever. When I tried creating a string and executing:
["sh", "-c", theStringHere].execute()
the result was the same - hangs forever. In this case though, I printed the string. When I ran that on the command line (directly or via 'sh -c') it worked perfectly - after the command completes, I can enter postgresql via 'psql' do a \l and see the database created.
Anyone know what the problem is?
The most likely issue would be createdb not seeing the environment variable and hanging waiting for the password.
I think you have two options. The first is to continue the shell escapes and try to use a .pgpass file, but the other is to connect to the postgres (or other existing db) and create the database manually. To do this, issue the following SQL:
CREATE DATABASE myDatabaseName WITH TEMPLATE template0 ENCODING utf8;
Is there a way to specify that when executing a sql script it stops when encountering the first error on the script, it usually continues, regardless of previous errors.
I think the solution to add following to .psqlrc is far from perfection
\set ON_ERROR_STOP on
there exists much more simple and convenient way - use psql with parameter:
psql -v ON_ERROR_STOP=1
better to use also -X parameter turning off .psqlrc file usage.
Works perfectly for me
p.s. the solution found in great post from Peter Eisentraut. Thank you, Peter!
http://petereisentraut.blogspot.com/2010/03/running-sql-scripts-with-psql.html
I assume you are using psql, this might be handy to add to your ~/.psqlrc file.
\set ON_ERROR_STOP on
This will make it abort on the first error. If you don't have it, even with a transaction it will keep executing your script but fail on everything until the end of your script.
And you probably want to use a transaction as Paul said. Which also can be done with psql --single-transaction ... if you don't want to alter the script.
So a complete example, with ON_ERROR_STOP in your .psqlrc:
psql --single-transaction --file /your/script.sql
It's not exactly what you want, but if you start your script with begin transaction; and end with end transaction;, it will actually skip everything after the first error, and then it will rollback everything it did before the error.
I always like to reference the manual directly.
From the PostgreSQL Manual:
Exit Status
psql returns 0 to the shell if it finished normally, 1 if a fatal
error of its own occurs (e.g. out of memory, file not found), 2 if the
connection to the server went bad and the session was not interactive,
and 3 if an error occurred in a script and the variable
ON_ERROR_STOP was set.
By default if the sql code you are running on the PostgreSQL server error psql won't quit an error. It will catch the error and continue. If, as mentioned above, you set the ON_ERROR_STOP setting to on, when psql catches an error in the sql code it will exit and return 3 to the shell.