I get a little confused some times when working with psql between when to use a set vs. \set vs. \pset. I think that:
set is for session variables on my connection to the db. For example SET ROLE dba;
\set is for local variables for this psql session. For example \set time 'select current_timestamp'
\pset is for psql settings for this psql session. For example '\pset border 2'
But, I've never found what I thought was a good explanation of each. Are my assumptions above correct?
I'm using PostgreSQL 9.4
Basically correct. The important difference is that SET is an SQL command while the other two are psql meta-commands - indicated by the leading \.
SET is an SQL command to change run-time parameters. It is executed on the server and has nothing to do with psql per se.
\set is a psql meta-command:
Sets the psql variable name to value [...]
Note: This command is unrelated to the SQL command SET.
\pset is another psql meta-command:
This command sets options affecting the output of query result tables
Related
I am running a script interactively to test it. The meta-command \c is not playing well with the standard sql commands: it seems to want to be run by itself. Here is what happens when it is run in the script
\set username steve
\c pubkey :username
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO :username;
GRANT ALL PRIVILEGES ON ALL sequences IN SCHEMA public TO :username;
Results:
invalid integer value "ALL" for connection option "port"
Previous connection kept
But if the "\c" command is run by itself then the subsequent grant's are successful. Is there a way to embed the "\c" connection commands in a psql script?
It seems that running in psql has different behavior than in a script itself: running the above from the command line using -f does work
psql postgres -f create_dbs.sql -a > db.out 2>&1
Whereas running the '\c` command from psql seems to need to be isolated.
I've created a SQL file I run through the psql command that roughly looks like as follows:
truncate table my_table;
\set content `cat /workdir/test.json` insert into my_table values ('test_row', :'content');
The first line is somewhat irrelevant to the problem, except for the fact it does print out "TRUNCATE TABLE", so it is reading and running the SQL file correctly, at least initially. However, the insert row is never created, the table is always empty. Yet no error message pops up.
The JSON file has a valid value (even if I pare it down to super basic {}). I've also tried passing the sql command directly (just to cover my bases, tried it with just one '' and same, with three it gives invalid command error):
psql [...] -c "\\set content `cat /workdir/test.json` insert into my_table values ('test_row', :'content')"
Again, no output message, no new rows created. However not using the meta-command \set does work. E.g.:
psql [...] -c "insert into my_table values ('test_row', '{}')"
Seems like there's something it doesn't like about the meta-command \set, but without any error info, not sure what I'm doing wrong.
Both the script and database are running on the same VM. That is, script can call host via 'localhost' and the filesystem/filepaths should be the same, I think, should that matter.
A psql meta-command (something that starts with a backslash) are terminated by the end of line; you cannot have an SQL statement on the same line.
Write the \set in one line and the INSERT in another.
If you want to use the -c option of psql, use several -c options:
psql -c "\\set ..." -c "INSERT ..."
I run a daily backup of my database using pg_dump and pg_restore that recently stopped working after I pushed an update.
I have a function validate_id that's a Case/When statement just as a quick check for some the data that has integrity issues. Looks something like this:
CREATE OR REPLACE FUNCTION validate_id(
_string text,
_type type
) RETURNS boolean AS
$$
SELECT
CASE WHEN (stuff) THEN TRUE
WHEN (other stuff) THEN TRUE
When (more stuff) THEN raise_err('Not an accepted type, the accepted types are: x y z')
ELSE FALSE
$$
LANGUAGE SQL;
Since I added this function, when I dump using this command:
pg_dump -U postgres -h ipaddress -p 5432 -w -F t databaseName > backupsfolder/databaseName.tar
When I use this command:
pg_restore -U postgres -h localhost -p 5432 -d postgres -C "backupsfolder/databaseName.tar"
As of two days ago, this now throws an error:
pg_restore: error: could not execute query: ERROR: function raise_err(unknown) does not exist
I'm pretty lost on what to do. I think what might be going on is that it's trying to restore this function before it restores the raise_err function. Which I thought was built-in to postgres (I can SELECT raise_err('Hello, World');). Is this possible? Is it my CASE statement because I need to return only Booleans? All of the permissions seem correct and restoring with previous backups works fine.
The problem is that raise_err is not schema qualified in your function code.
This is potentially dangerous: a malicious user could create his own function raise_err and set search_path so that the wrong function is called.
Since pg_restore is typically run by a superuser, this can be a security problem. Imagine such a function being used in an index definition!
For these reasons pg_dump and pg_restore set an empty search_pathin current versions of PostgreSQL.
The solution to your problem is to explicitly use the function's schema in your SQL statement.
I ended up solving this issue by explicitly setting the search paths for both functions, raise_err() and validate_id() to public:
ALTER FUNCTION validate_id(text,text) SET search_path=public;
ALTER FUNCTION raise_err(text,text) SET search_path=public;
I'm using a windows batch file to connect to postgres using psql. I'm issuing commands like this....
SET PGPASSWORD=postgres
psql -U postgres -d postgres -c "DROP USER IF EXISTS foo;"
This works fine for running one, short SQL command against the database. But I'm having trouble with two related issues
How to continue a single long SQL command over multiple lines, and
How to run multiple commands.
Example 1.....
psql -U postgres -d postgres -c "CREATE DATABASE foo
WITH OWNER = bar
ENCODING = 'UTF8'
TABLESPACE = mytabspace;"
Example 2.....
psql -U postgres -d postgres -c "
ALTER TABLE one ALTER COLUMN X TYPE INTEGER;
ALTER TABLE two ALTER COLUMN Y TYPE INTEGER;"
Neither of these will work as shown, I've done some googling and found some suggestions for doing this with linux, and have experimented with various carats, backslashes and underscores, but just don't seem to be able to split the commands across lines.
I'm aware of the -f option to run a file, but I'm trying to avoid that.
Any suggestions?
The line continuation character in batch is the ^. See this Q&A
So end the line with space+caret ^ and make sure the following line begins with a space.
You will also have to escape double quoted areas that span several lines with a caret for this to work.
Since the line is unquoted then for the batch parser you will also have to escape any special chararacters like <>|& also with a caret.
psql -U postgres -d postgres -c ^"CREATE DATABASE foo ^
WITH OWNER = bar ^
ENCODING = 'UTF8' ^
TABLESPACE = mytabspace;"
psql -U postgres -d postgres -c ^" ^
ALTER TABLE one ALTER COLUMN X TYPE INTEGER; ^
ALTER TABLE two ALTER COLUMN Y TYPE INTEGER;"
I use Ruby to generate a bunch of SQL commands, and store this into a file.
I then login to my PostgreSQL database. Then I do something like:
\i /tmp/bla.sql
And this populates my database.
This all works fine as it is, no problem here.
I dislike the manual part where I have to use \i, though (because I need this to work in a cron job eventually, and I think commands like \i are only available when you are directly in the interactive psql prompt).
So my question now is:
Is it possible to use a psql command from the command line that directly will start to read in an external file?
You can directly use the psql command as shown below.
Works for me with Ubuntu and Mint. On Windows it should be quite the same...
psql -U user -d database -f filepath
Example:
psql -U postgres -d testdb -f /home/you/file.sql
For more information take a lock at the official documentation: http://www.postgresql.org/docs/current/static/app-psql.html
When you try to execute an sql file using cron, you will also need to set the environment - database name, password etc. This is a short shell script snippet that does it all
source /var/lib/pgsql/scripts/.pgenv
echo $PATH
psql << AAA
select current_date;
select sp_pg_myprocedure(current_date);
AAA
In .pgenv, you set the values such as
export PGPORT=<yourport>
export PGHOST=<yourhost>
export PGDATA=<yourdatadir>
Also have a .pgpass file so that the password is supplied.
http://www.postgresql.org/docs/current/static/libpq-pgpass.html
Replace the part where SELECT is being done with whatever you want to do, or do it as #Kuchi has shown.