How to execute multiple sql files in postgreSQL linux? - command-line

I have many .sql files in a folder (/home/myHSH/scripts) in linux debian. I want to know the command to execute or run all sql files inside the folder into postgreSQL v9.1 database.
PostgreSQL informations:
Database name=coolDB
User name=coolUser
Nice to have: if you know how to execute multiple sql files through GUI tools too like pgAdmin3.

From your command line, assuming you're using either Bash or ZSH (in short, anything but csh/tcsh):
for f in *.sql;
do
psql coolDB coolUser -f "$f"
done

The find command combined with -exec or xargs can make this really easy.
If you want to execute psql once per file, you can use the exec command like this
find . -iname "*.sql" -exec psql -U username -d databasename -q -f {} \;
-exec will execute the command once per result.
The psql command allows you to specify multiple files by calling each file with a new -f argument. e.g. you could build a command such as
psql -U username -d databasename -q -f file1 -f file2
This can be accomplished by piping the result of the find to an xargs command once to format the files with the -f argument and then again to execute the command itself.
find . -iname "*.sql" | xargs printf -- ' -f %s' | xargs -t psql -U username -d databasename -q

Related

How to get dump only function creation & Stored Process script via command line in PostgreSQL

I am able to get the script for table creation via pg_dump command in postgress sql similar way I need to generate for functions, SP's & views to export into SQL file via pg_dump.
Can you please help on this?
For a Table:
pg_dump -U postgres -d postgres -t clientlocationregions > C:\clientlocationregions.sql
Thanks
Mahesh
This will do:
pg_dump -Fc -s -n <schemaname> -f temp.dump <database name>
pg_restore -l temp.dump | grep FUNCTION > functionalist
pg_restore -L functionalist temp.dump > yourfunctions.sql

How to create a folder to save output in Postgresql?

If I'd like to save the output from file1.sql to a new file file2.sql, I would use this command in terminal/cmd:
psql -U postgres -f file1.sql -o file2.sql
What if, though, I want file2.sql to be in a different folder?
If I try this command:
psql -U postgres -f file1.sql -o New/file2.sql,
that won't automatically make a new folder and will give an error. The New folder needs to exist before I can do this.
I need to this over many output files and many new folders. One obvious alternative would be to pre-create the required folders using Python, but really, is there no way Postgresql can create folders for me?
Use mkdir -p.
It will try to create a directory New if it doesn't exist or does nothing if it already exists.The && ensures that
your psql command runs only if the mkdir command succeeds.
mkdir -p New && psql -U postgres -f file1.sql -o New/file2.sql
If you want to run os commands inside psql, simply use \! <command> option within file1.sql and then output via \o option.
\! mkdir -p New
\o New/file2.sql

How to use postgres user in a shell script on Ubuntu 16

I am trying to adapt a shell script set made for running on Debian 7 to work on Ubuntu 16.
I got to change successfully all except a part that executes PosgreSQL database commands.
Former version of script has these lines:
service postgresql restart
psql -q -U postgres -c "DROP DATABASE IF EXISTS db_crm" -o $log &> /dev/null
psql -q -U postgres -c "CREATE DATABASE db_crm" -o $log &> /dev/null
When I tried to run psql as above on Ubuntu 16, OS didn't recognize command. It is important to say that script is called with sudo.
I got to find a way to run only database script on Ubuntu 16 changing code so:
service postgresql restart
su postgres <<EOF
psql -c "DROP DATABASE IF EXISTS db_crm" -o $log &> /dev/null
psql -c "CREATE DATABASE db_crm" -o $log &> /dev/null
However, this same script doesn't work when it is called by main script. Following messages are presented:
here-document at line 41 delimited by end-of-file (wanted 'EOF')
syntax error: unexpected end of file
Even replacing EOF to beggining of next line, error continues.
If there is a way to use psql in shell script without to use EOF would be better.
The reason your script is failing, is you forgot the EOF at the end of input.
su postgres <<EOF
psql -c "DROP DATABASE IF EXISTS db_crm" -o $log &> /dev/null
psql -c "CREATE DATABASE db_crm" -o $log &> /dev/null
EOF #<<<--- HERE
An easy way to do this is to put your commands into a temporary file, then re-direct that into psql. Obviously you don't want this to stop for a password prompt, in this case either use a user that doesn't need it, or set $PGPASSWORD - or prompt at the beginning of the script - there's lots of ways around.
#! /usr/bin/env bash
# PGPASSWORD='' #(set this to stop password prompts, but it's insecure)
PSQL="psql -q -U postgres -o $log" #TODO define $log
TMPFILE="/tmp/sql-tmp.`date +%Y%m%d_%H%M%S_%N`.sql"
# TODO - check $TMPFILE does not exist already
echo "DROP DATABASE IF EXISTS db_crm;" > "$TMPFILE"
echo "CREATE DATABASE db_cr;" >> "$TMPFILE"
# ... etc.
# run the command, throw away any stdout, stderr
PSQL < "$TMPFILE" 2>&1 > /dev/null
# exit with the psql error code
err_code=$?
exit $?

Why does pg_restore return "options -d/--dbname and -f/--file cannot be used together"?

The following Windows batch script fails on the line with database restore:
#Echo off
set Path=C:\Program Files (x86)
set Backup_Path=C:\Program Files (x86)
c:
cd \
cd C:\Program Files (x86)\PostgreSQL\9.1\bin
#echo "Wait ..."
setlocal
set PGPASSWORD=1234
psql.exe -U postgres -c "create database Mydata"
"C:\Program Files (x86)\PostgreSQL\9.1\bin\pg_restore.exe" -h localhost -U postgres -d Mydata -f "Mydata.backup"
pause
endlocal
The error is:
pg_restore : -d / - dbname and -f / - file can not be used together
Try " pg_restore --help "
-f is the output filename, not the input file.
The input file does not have any parameter switch.
c:\>pg_restore --help
pg_restore restores a PostgreSQL database from an archive created by pg_dump.
Usage:
pg_restore [OPTION]... [FILE]
General options:
-d, --dbname=NAME connect to database name
-f, --file=FILENAME output file name
-F, --format=c|d|t backup file format (should be automatic)
-l, --list print summarized TOC of the archive
-v, --verbose verbose mode
-V, --version output version information, then exit
-?, --help show this help, then exit
So you need to use:
pg_restore.exe -h localhost -U postgres -d Mydata "Mydata.backup"
More details in the manual: http://www.postgresql.org/docs/current/static/app-pgrestore.html

psql -o not what I expected (how to output db response to an output file)

I am creating a PostgreSQL database from the command line (i.e. using psql).
There are some errors in my SQL statements and I want to find out where the errors are occuring (too many objects to fill the screen buffer - so I need to save thios to file)
I have tried just about everything, from using the -o option, the -L option and using tee - I still cant capture the information that scrolls past on the screen.
How do I log this?
This is what I have tried so far:
psql -U -o dbcreate.log -f file.sql
psql -U -L dbcreate.log -f file.sql
psql -U -a -f file.sql | tee dbcreate.log
NONE of which results in the data flashing accross the screen being logged to file - how do I do this?
You need to redirect stderr. On Un*x and Linux:
psql ... 2>error.log
or both stdout and stderr:
psql ... &>error.log
On the other hand if you like to investigate the errors one by one:
psql -v ON_ERROR_STOP=1 ...
A helpful article about executing SQL scripts with psql - here.