I am using 4D for front-end and postgresql for back-end. So i have the requirement to take database backups from front-end.
Here what i have done so far for taking backups in 4D.
C_LONGINT(i_pg_connection)
i_pg_connection:=PgSQL Connect ("localhost";"admin";"admin";"test_db")
LAUNCH EXTERNAL PROCESS("C:\\Program Files\\PostgreSQL\\9.5\\bin\\pg_dump.exe -h localhost -p 5432 -U admin -F c -b -v -f C:\\Users\\Admin_user\\Desktop\\backup_test\\db_backup.backup test_db")
PgSQL Close (i_pg_connection)
But the it's not taking the backup.
The backup command is ok because it works perfectly while firing on command prompt.
What's wrong in my code?
Thanks in advance.
Unneeded commands in your code
If you are using LAUNCH EXTERNAL PROCESS to do the backup then you do not need the PgSQL CONNECT and PgSQL CLOSE.
These plug-in commands do not execute in the same context as LAUNCH EXTERNAL PROCESS so they are unneeded in this situation.
Make sure you have write access
If the 4D Database is running as a Service, or more specifically as a user that does not have write access to C:\Users\Admin_user\..., then it could be failing due to a permissions issue.
Make sure that you are writing to a location that you have write access to, and also be sure to check the $out and $err parameters to see what the Standard Output and Error Streams are.
You need to specify a password for pg_dump
Another problem is that you are not specifying the password.
You could either use the PGPASSWORD environment variable or use a pgpass.conf file in the user's profile directory.
Regarding the PGPASSWORD environment variable; the documentation has the following warning:
Use of this environment variable is not recommended for security reasons, as some operating systems allow non-root users to see process environment variables via ps; instead consider using the ~/.pgpass file
Example using pgpass.conf
The following example assumes you have a pgpass.conf file in place:
C_TEXT($c;$in;$out;$err)
$c:="C:\\Program Files\\PostgreSQL\\9.5\\bin\\pg_dump.exe -h localhost -p 5432 -U admin -F"
$c:=$c+" c -b -v -f C:\\Users\\Admin_user\\Desktop\\backup_test\\db_backup.backup test_db"
LAUNCH EXTERNAL PROCESS($c;$in;$out;$err)
TRACE
Example using PGPASSWORD environment variable
The following example sets the PGPASSWORD environment variable before the call to pg_dump and then clears the variable after the call:
C_TEXT($c;$in;$out;$err)
SET ENVIRONMENT VARIABLE ( "PGPASSWORD" ; "your postgreSQL password" )
$c:="C:\\Program Files\\PostgreSQL\\9.5\\bin\\pg_dump.exe -h localhost -p 5432 -U admin -F"
$c:=$c+" c -b -v -f C:\\Users\\Admin_user\\Desktop\\backup_test\\db_backup.backup test_db"
LAUNCH EXTERNAL PROCESS($c;$in;$out;$err)
SET ENVIRONMENT VARIABLE ( "PGPASSWORD" ; "" ) // clear password for security
TRACE
Debugging
Make sure to use the debugger to check the $out and $err to see what the underlying issue is.
Related
I am using postgres 9.3 . I want to make a script to create my database cluster and supply the password inline in the terminal. I know you can do it from file, but is there a way to do it command line?
that is the line I am using right now : 'initdb -D path/to/cluster -W -A password'
it then prompt me for password, I tried to provide it inline, but it does not work. Any ideas?
thanks
You can accomplish this with a shell trick. Assuming bash shell:
initdb -D path/to/cluster -A password --pwfile=<(echo secretpassword)
(Although you should never use -A password, use at least md5.)
As for your comment, it is hard to say what is going on. You don't show us starting the server at all, or setting the port to start on to 5555, nor creating a user named 'dbuser'.
thanks everyone!
It worked when I changed 'password' to 'md5' in my initdb statement.
although in the password.txt file, I can only store the password. If I follow the documentation from postgres
host:port:username:password it does not work anymore
I solved the connection problem with .pgpass. I've created the .pgpass file in the home directory and I was able to connect using: "psql -U username -d database -pXXXX"
Ok, my initial question was to supply the password inline in the terminal. But ,the proper way of doing is using the password file for security reason. Second, in the initdb statement use at least md5 for encryption. The password will be for the superuser for the default database. You statement should look like this:
"initdb -D /path/to/dbCluster -A md5 --pwfile=/path/to/password.txt"
Now, if you want to automatically connect to the database with psql without password prompt, you have to create a .pgpass file (linux) or pgpass.conf file (windows) with your user and password info in this format: host:port:db_name:user_name:password
Where you put those file is important:
Windows : /Users/user_name/AppData/Roaming/postgresql/pgpass.conf
(If the postgresql folder does not exist, you have to create it)
Linux : /home/user/.pgpass (with chmod 0600 permission on the file)
How to force psql to detect .pgpass file on Windows 10 system?
I am migrating an application into Docker. One of the issues that I am bumping into is what is the correct way to load the initial data into PostgreSQL running in Docker? My typical method of restoring a database backup file are not working. I have tried the following ways:
gunzip -c mydbbackup.sql.gz | psql -h <docker_host> -p <docker_port> -U <dbuser> -d <db> -W
That does not work, because PostgreSQL is prompting for a password, and I cannot enter a password because it is reading data from STDOUT. I cannot use the $PGPASSWORD environment variable, because the any environment variable I set in my host is not set in my container.
I also tried a similar command above, except using the -f flag, and specify the path to a sql backup file. This does not work because my file is not on my container. I could copy the file to my container with the ADD statement in my Dockerfile, but this does not seem right.
So, I ask the community. What is the preferred method on loading PostgreSQL database backups into Docker containers?
I cannot use the $PGPASSWORD environment variable, because the any
environment variable I set in my host is not set in my container.
I don't use docker, but your container looks like a remote host in the command shown, with psql running locally. So PGPASSWORD never has to to be set on the remote host, only locally.
If the problems boils down to adding a password to this command:
gunzip -c mydbbackup.sql.gz |
psql -h <docker_host> -p <docker_port> -U <dbuser> -d <db> -W
you may submit it using several methods (in all cases, don't use the -W option to psql)
hardcoded in the invocation:
gunzip -c mydbbackup.sql.gz |
PGPASSWORD=something psql -h <docker_host> -p <docker_port> -U <dbuser> -d <db>
typed on the keyboard
echo -n "Enter password:"
read -s PGPASSWORD
export PGPASSWORD
gunzip -c mydbbackup.sql.gz |
psql -h <docker_host> -p <docker_port> -U <dbuser> -d <db>
Note about the -W or --password option to psql.
The point of this option is to ask for a password to be typed first thing, even if the context makes it unnecessary.
It's frequently misunderstood as the equivalent of the -poption of mysql. This is a mistake: while -p is required on password-protected connections, -W is never required and actually goes in the way when scripting.
-W, --password
Force psql to prompt for a password before connecting to a
database.
This option is never essential, since psql will automatically
prompt for a password if the server demands password
authentication. However, psql will waste a connection attempt
finding out that the server wants a password. In some cases it is
worth typing -W to avoid the extra connection attempt.
For a while i have my db running on a command window because im not figuring out how to run it as a windows service.
Since i have the zip file version downloaded. how can i register the pg_ctl command as a windows service?
By the way, im using the following line to start the server:
"D:/Program Files/PostgreSQL/9.0.4/bin/pg_ctl.exe" -D "D:/Program Files/PostgreSQL/9.0.4/db_data" -l logfile start
Thanks in advance.
Use the register parameter for the pg_ctl program.
The data directory should not be stored in Program Files, the location of %ProgramData% is e.g. a good choice.
pg_ctl.exe register -N PostgreSQL -U some_windows_username -P windows_password -D "%ProgramData%/db_data" ...
In newer versions of Postgres, a separate Windows account is no longer necessary, so the following is also sufficient
pg_ctl.exe register -N PostgreSQL -D "%ProgramData%/db_data" ...
Details are in the manual: http://www.postgresql.org/docs/current/static/app-pg-ctl.html
You need to make sure the directory D:/Program Files/PostgreSQL/9.0.4/db_data has the correct privileges for the windows user you specify with the -U flag.
Btw: it is a bad idea to store program data in Program Files. You should move the data directory somewhere outside of Program Files because Program Files is usually highly restricted for regular users - with a very good reason.
Just run 'Command Prompt' as windows administrator and run the below command:
pg_ctl.exe register -N PostgreSQL -D "D:/Program Files/PostgreSQL/9.0.4/db_data"
You don't need to specify a User and Password, as previous answers have suggested.
During the installation of my app, I want to create a PostgreSQL-Database and some tables and functions.
For that purpose I use PSQL.EXE that ships with PostgreSQL. I have 2 scripts. The first one creates the database and a corresponding user that has rights to execute scripts on that database. I want to execute the second script as this just created user. Unfortunately I can't find a way to pass the password for that user as a command line argument. Omitting the password leads to a stop of execution and a prompt for the user to enter the password, which I would like to avoid - since this is executed during installtion of my app.
Is there any way to pass the password as argument or is there any other command line tool I could use?
To explain the environment a bit further. I use WiX 3.5 setup as a "MSI-Builder".
You can either use a pgpass file as dbenhur answerd, or you can set the environment variable PGPASSWORD before calling psql:
SET PGPASSWORD=my_very_secret_password
psql somedb someuser
All supported environment variables are documented in the manual: http://www.postgresql.org/docs/current/static/libpq-envars.html
You can't supply password via cmdline arg (and don't want to as that's poor security practice).
You can provide a .pgpass file to support automatic script authentication. Here's the docs.
Better still, if you have access to create the db role then you already have all the access you need without having to carefully log in with a password. Have the second script operate under the same user as the first but include the following line to switch user:
set role my_new_user;
Where my_new_user is the name of the role you want to run it as.
If you only divided the scripts because of the different logins then with this they can go in the same file and just switch role mid way through.
Note:
On the off chance that you are not creating the DB and new role as a super user this may be a little more complex. If this is the case you will need to create the new role with:
create role my_new_role ... ADMIN my_role;
Where my_new_role is the role you're creating and my_role is your current user. Then when you're finished simply:
revoke my_new_role from my_role;
For completion, you can also use URI (doc link)
List dbs
psql "postgresql://username:password#localhost/postgres" -l
I also crafted this command to have only names (please tell me if you know a better way):
psql "postgresql://username:password#localhost/postgres" -l | awk -F '|' '{print $1}'| sed -e '/^\s*$/ d' -e '1,3d'|sed '$d'|awk '{print $1}'
You can also use unix socket to connect:
# ss -x -a |grep postgres|awk '{print $5}'
/var/run/postgresql/.s.PGSQL.5432
Note that the parent directory of the socket is used:
# sudo -u postgres psql -d "postgresql:///postgres?host=/var/run/postgresql/" -l
You can only do this if you have this line in your pg_hba.conf:
local all postgres ident
"ident" uses unix user for authent
dump a db
Here I added a different port number
pg_dump -Fc "postgresql://username:password#localhost:9001/${db}" > "backup_${db}.pgdump"
With dumpall you need a super user or role (with CREATE ROLE ... SUPERUSER). And it must have access to all DB. By default postgres can.
but in my case I couldn't use pg_dumpall with postgres because his password was removed by devs.
So I used:
sudo -u postgres pg_dumpall -d "postgresql:///?host=/var/run/postgresql/" > all.dump
tested version
# cat /opt/postgresql/PG_VERSION
9.6
hth
DB: PostgreSQL 9.0
Client: Windows 7
Server Windows 2008, 64bit
I'm trying to connect remotely to a PostgreSQL instance for purposes of performing a pg_dump to my local machine.
Everything works from my client machine, except that I need to provide a password at the password prompt, and I'd ultimately like to batch this with a script.
I've followed the instructions here:
http://www.postgresql.org/docs/current/static/libpq-pgpass.html
But it's not working.
To recap, I've created a file on the server: C:/Users/postgres/AppData/postgresql/pgpass.conf, where PostgreSQL is the db user.
The file has one line with the following data:
\*:5432:\*postgres:[mypassword]
I've also tried replacing each * with [localhost|myip] and [mydatabasename] respectively.
From my client machine, I connect using:
pg_dump -h <myip> -U postgres -w [mydbname] > [mylocaldumpfile]
I'm presuming that I need to provide the -w switch in order to ignore password prompt, at which point it should look in the AppData directory on the server.
It just comes back with:
connection to database failed: fe_sendauth: no password supplied.
As a hack workaround, if there was a way I could tell the Windows batch file on my client machine to inject the password at the PostgreSQL prompt, that would work as well.
It works for me:
Use command line
cd %appdata%
mkdir postgresql
cd postgresql
notepad pgpass.conf
inside pgpass.conf paste your connection string (*:5432:*postgres:[mypassword]) and save the file.
To connect to postgres use:
psql/pg_dump -U <username> -h <host> -w <other params you want to use>
I have solved similar problem (only in Linux) to use ip address in pgpass and psql.
.pgpass
127.0.0.1:5432:db:dbuser:123
psql params
psql -d db -U dbuser -h 127.0.0.1 -w
pg_hba conf with default settings:
# IPv4 local connections:
84 host all all 127.0.0.1/32 md5
Create pgpass.conf file
Windows > Start > Run
type %APPDATA%
press OK
Create a folder: postgresql
Create a file : pgpass.conf (under postgresql folder)
Open pgpass.conf file
Now you should have below file ready, open it via below (making sure it exists):
Windows > Start > Run
type %APPDATA%\postgresql\pgpass.conf
press OK
Paste pgpass.conf file contents
Paste the below
# serverDomainOrIP:PORT:databaseName:userName:password
# 127.0.0.1:5432:myDbName:postgres:myPassword
# *:5432:*:*:myPassword
You can do one of the below:
- Remove # for the 3rd line, and give your password in the place of "myPassword"
OR
- Remove # for the 2nd line, and give ip (or, yourDomain.com), dbname, username & password
Hope that helps.
I've had a similar problem which I didn't manage to resolve - I couldn't get the script to recognise the pgpass.conf file. I however used a work-around of setting the PGPASSWORD environment variable in the batch file I was using (PostgreSQL 9.6).
Inside a batch file:
SET PGPASSWORD=<<password>>
pg_dump.exe -h <<host>> -p <<port>> -U <<user>> -Fc -b -v -f <<output file path>> <<database>>
I have gotten it to work with the following:
pgpass.conf:
127.0.0.1:5432:*:username:password
However, I have it stored here:
C:\Users\<user>\AppData\Roaming\postgresql
For some reason, on a previous iteration of Postgres, the database had generated the pgpass file and stored it there. I was running into the same issue you were having, moved it to that directory and it worked. I'm not sure why though.
Then, all you'll need to do is:
pg_dump -h myip mydb > mylocaldumpfile
...ensuring that myip and the ip in pgpass.conf are identical. If they are not, it will prompt you for a password.
You could use pgAdmin III to store the password (in the server's properties).
That operation automatically creates a correct pgpass.conf file. You can then schedule a task to run a simple batch file that would read:
"C:\path\to\pg_dump.exe" -U <user> -w <database> > C:\path\to\database.backup
Make sure you are logged in as the user corresponding with the folder where the pgpass.conf file lives.
If you are using UTF-8 encoding, please ensure that you are using without BOM mode.
Otherwise leave the first line as a comment:
# This line may contain hidden BOM bytes
localhost:5432:database:username:password
Also you don't need to escape asterisks \*, just put * to enable wildcard matching.