Postgresql Import dump with user dfferent from postgres - postgresql

i'm tring to backup my database from production machine to a standby machine.
My production machine is a debian (unspecified version) distro
My standby machine is a ubuntu 20.10 distro
After some readings across the internet here is what i was able to accomplish
pg_dump -h 123.123.123.123 -p 5432 my_db > my_db.bak -> check
psql < createDateabase.sql -> check this make a simple drop/create database my_db owner my_db_backup;
psql < my_db.back -> check
so far so good
connect with my sqlworkbench to db -> check
use table imported -> ehm, not check
the problem is the ownership of the tables and sequeces. When imported, postgres is their owner but i want my_db_backup as owner insted.
I have tryed, as a workaround, to
alter table xxx OWNER TO my_db_backup; - > check for each table
alter sequence if exists xxx OWNER TO my_db_backup; -> it takes forever and it is unable to finish;
the command for the final stage (change ownership) is
psql < finalize.sql
and finalize.sql is
\connect my_db;
ALTER TABLE xxx OWNER TO my_db_backup;
ALTER SEQUENCE IF EXISTS seq_xxx OWNER TO my_db_backup; -- this takes forever
what i'm missing?
I would prefer to import directly with the right owner instead of altering owner after, but that's still acceptable as long as i can change all objects in my_db
tnx for the help

I was finally able to accomplish the import with a different user as follow
psql -U my_db_backup < my_db.bak
to avoid asking for password i had to make an entry in pg_hba.conf like this
local my_db my_db_backup trust
and to avoid request for password during the pg_dump i had to add this entry in my production machine
host my_db postgres 456.456.456.456/32 trust
where 456.456.456.456 id the ip of the standby machine.
i also had to change a bit the pg_dump to avoid errors for different user import
pg_dump -h 123.123.123.123 -p 5432 --no-owner --no-acl my_db > my_db.bak
so the complete script looks like
pg_dump -h 123.123.123.123 -p 5432 --no-owner --no-acl my_db > my_db.bak
psql < createDatabase.sql
psql -U my_db_backup < my_db.bak
EDIT: to make the script usable from any user there are a few things in need to be done
pg_dump -h 123.123.123.123 -p 5432 -U postgres --no-owner --no-acl -w my_db > my_db.bak
psql -U postgres < createDatabase.sql
psql -U my_db_backup < my_db.bak
Adding -U postgres is needed only because pg_dump try to login with the current user so if you are running the script as, for example, root pg_dump and psql will try to login as root and will obviously fail.
The other thing is the -w option that is needed to make pg_dump not ask for password.

Related

Postgres ERROR: permission denied for table

I have a dump file generated by using pg_dump from a remote database. I am using the file to restore and populate a database using psql.
I did not have the user and database name. I peeked into the sql file and figured out the user and database.
Here are my steps:
1. psql -h <> -u <> -d ( I used the superuser here).
2. created the user as the application needs to access using this user only.
3. changed the owner of the database to the user created in step 2 using Pgadmin.
I am unable to access any tables and getting the error -- ERROR: permission denied for table
Please Help !
Thanks
It is better if you use dump and restore instead of psql for restoring the database.
e.g
e.g suppose you have database dbname_source in remoteDB and dbname_target in localDB then you can use the below commands for dump and restore
./pg_dump -U postgres -p 5432 -h <remote_IP> -Ft dbname_source > dump_of_source.sql.tar
./pg_restore -U postgres -h localhost -p 5432 -d dbname_target dump_of_source.sql.tar

Modifying postgresql.conf through shell?

I want to modify a postgresql.conf parameter through the shell. From the documentation I can see that I can use the postgres command with the -c flag.
However, on my attempt, for example,
postgres -c autovacuum=off
postgres returns:
Execution of PostgreSQL by a user with administrative permissions is not permitted.
The server must be started under an unprivileged user ID to prevent possible system security compromises. See the documentation for more information on how to properly start the server.
How can I overcome this or what is the correct procedure? Also, I don't really mind for security compromises.
Given the differences on the underlying OS, I usually prefer to do this via PostgreSQL itself, which comes handy when you're dealing with a managed service that do not give you filesystem access, like so:
sudo -u postgres psql -U postgres -d database_name -c "alter system set postgresql_parameter = 'new_value';"
As an example when I have to install TimeScaleDB extension, I can do:
sudo -u postgres psql -U postgres -d database_name -c "alter system set shared_preload_libraries = 'timescaledb';"
sudo service postgresql restart
sudo -u postgres psql -U postgres -d database_name -c "create extension if not exists timescaledb;"

pg_dump fails with error: `FATAL: role "username" does not exist`

I have a db named backupdb, I want to import this to my local rails app so I want to take a dump of it.
When I am running pg_dump backupdb, I am getting below error.
pg_dump: [archiver (db)] connection to database "backupdb" failed: FATAL: role "username" does not exist
what's wrong here. Please help.
I downloaded the db from my email and then trying to create a dump so I can import it to my local rails app
Try this way, it works!!
$ pg_dump -h localhost -U postgres -Fc mydb > db.dump
This command will take the backup of complete database
pg_dump -h localhost -U "dbuser" "dbname" -Fc > "pathfilename.backup"
**ex:** pg_dump -h localhost -U mani manidb - Fc > "thamesdb.backup"
for more pg_dump formats please refer to this answer
You're giving "username" as username, which does not exist. You need to pass a username that exists (postgres would probably do).
add parameters --username=postgres --password and it will ask you for the password for user postgres. (you might have security set to trust in your pg_hba.conf in which case leaving out --password would work.
If someone has the same problem using intellij, here is what helped me:
In my case the Problem was, that i right clicked the datasource, but you have to open it and right click the database itself. So in the image below don't click on localDB. Right click the database name and then do a dump.
Hope this will help someone to solve this confusing UX problem. :D
I was having the same problem but after trying out different permutations of the command options while referencing the documentation, I was able to get it running with this order of options. The command I used is:
pg_dump.exe -h localhost -U postgres -t "shop*" -t "core*" -t "profiles*" -f "path/to/output/dir_or_file" mydbname
Where:
option prefix
option
params
-h
host
localhost
-U
username
postgres
-t
Tables
Any tables whose name starts with (shop, core, or profiles (NOTE: for django app tables in my case))
-f
file out
path where I want to save the dumped file
(End of cmd)
database
mydbname
Note: as I have multiple versions of postgresql on my computer, I had to call the newest pg_dump executable. (Just for reference)

Create database from command line

I am trying to create a database from command line.
My OS is centos and postgres version is 10.9.
sudo -u postgres psql createdb test
Password for user test:
Why is it prompting me for the password?
Change the user to postgres :
su - postgres
Create User for Postgres (in the shell and NOT with psql)
$ createuser testuser
Create Database (same)
$ createdb testdb
Acces the postgres Shell
psql ( enter the password for postgressql)
Provide the privileges to the postgres user
$ alter user testuser with encrypted password 'qwerty';
$ grant all privileges on database testdb to testuser;
Try:
sudo -u postgres psql -c 'create database test;'
createdb is a command line utility which you can run from bash and not from psql. To create a database from psql, use the create database statement like so:
create database [databasename];
Note: be sure to always end your SQL statements with ;
As some of the answers point out, createdb is a command line utility that could be used to create database.
Assuming you have a user named dbuser, the following command could be used to create a database and provide access to dbuser:
createdb -h localhost -p 5432 -U dbuser testdb
Replace localhost with your correct DB host name, 5432 with correct DB port, and testdb with the database name you want to create.
Now psql could be used to connect to this newly created database:
psql -h localhost -p 5432 -U dbuser -d testdb
Tested with createdb and psql versions 9.4.15.
As the default configuration of Postgres, a user called postgres is made and the user postgres has full super admin access to entire PostgreSQL instance running on your OS.
sudo -u postgres psql
The above command gets you the psql command line interface in admin mode.
Creating user
sudo -u postgres createuser <username>
Creating Database
sudo -u postgres createdb <dbname>
NOTE: < > are not to be used while writing command, they are used just to signify the variables
PostgreSQL Create Database - Steps to create database in Postgres.
Login to server using postgres user.su - postgres
Connect to postgresql database.
bash-4.1$ psql
psql (12.1)
Type "help" for help.
postgres=#
Execute below command to create database.
CREATE DATABASE database_name;
Check for detailed information below:
https://orahow.com/postgresql-create-database/
PGPORT=5432
PGHOST="my.database.domain.com"
PGUSER="postgres"
PGDB="mydb"
createdb -h $PGHOST -p $PGPORT -U $PGUSER $PGDB
With a single command line:
su -c "createuser dbuser;createdb -h localhost -p 5432 -E UTF8 -O dbuser dbname;" - postgres
It is pretty simple but sometimes I find the answers tricky.
[For windows users]
Open Windows cmd
psql -U <username>
Once connected to psql, enter the following command to create a new database: CREATE DATABASE <database_name>;
To verify that the database has been created, you can run the \l command to list all available databases. Your new database should be listed in the output.------[Additional]------
You can now connect to the new database using the \c command followed by the database name, like this: \c <database_name>
You can now run SQL commands on the new database to create tables, insert data, and so on.
Note: Make sure to replace <username>, <database_name> with your actual Postgres username and database name.
If you are using pgAdmin:
In query editor you can try like this :
CREATE DATABASE <databasename>
WITH
OWNER = <dbowner>
ENCODING = <encoding>
CONNECTION LIMIT = <numberofsimulaneousconnections>;
an example snippet :
CREATE DATABASE twitterdb
WITH
OWNER = postgres
ENCODING = 'UTF8'
CONNECTION LIMIT = -1;

pg_dump postgres database from remote server when port 5432 is blocked

I'm trying to pg_dump a SQL database on a remote server in our DMZ. There are 2 problems.
there is not a lot of space left on the remote server so the normal command run to locally backup the database
pg_dump -C database > sqldatabase.sql.bak won't work due to space issues.
I also can't run the other version of pg_dump command to dump database from remote server to local server using:
pg_dump -C -h remotehost -U remoteuser db_name | psql localhost -U localuser db_name
as the server is in our DMZ and port 5432 is blocked. What I'm looking to see is if it is possible to pg_dump the database and immediatly save it (ssh or some other form) as a file to a remote server.
What I was trying was: pg_dump -C testdb | ssh admin#ourserver.com | > /home/admin/testdb.sql.bak
Does anyone know if what i am trying to achieve is possible?
You can connect with ssh to your remote server, do with the connect the pg_dump call and send the output back to stdout of local machine.
ssh user#remote_machine "pg_dump -U dbuser -h localhost -C --column-inserts" \
> backup_file_on_your_local_machine.sql
let's create a backup from remote postgresql database using pg_dump:
pg_dump -h [host address] -Fc -o -U [database user] <database name> > [dump file]
later it could be restored at the same remote server using:
sudo -u postgres pg_restore -C mydb_backup.dump
Ex:
pg_dump -h 67.8.78.10 -Fc -o -U myuser mydb > mydb_backup.dump
complete (all databases and objects)
pg_dumpall -U myuser -h 67.8.78.10 --clean --file=mydb_backup.dump
restore from pg_dumpall --clean:
psql -f mydb_backup.dump postgres #it doesn't matter which db you select here
Copied from: https://codepad.co/snippet/73eKCuLx
You can try to dump part of the table to a file in your local machine like this (assume your local machine has psql installed):
psql -h ${db_host} -p 5432 -U ${db_user} -d ${db_name} \
-c "\copy (SELECT * FROM my_table LIMIT 10000) to 'some_local_file.csv' csv;"
And you can import the exported csv into another db later like this:
COPY my_table FROM '/path/to/some_local_file.csv' WITH (FORMAT csv);
One possible solution - pipe through ssh - has been mentioned.
You also could make your DB server listen on the public inet address, add a hostssl entry for your backup machine to pg_hba.conf, maybe configure a client certificate for security, and then simply run the dump on the client/backup machine with pg_dump -h dbserver.example.com ...
This is simpler for unattended backups.
For the configuration of the connection (sslmode) see also the supported environment variables.
If you would like to periodically backup a database PostgreSQL that is inside of a container in the remote server to your local host by using pg_dump over ssh, this is useful for you:
https://github.com/omidraha/periodic-pgdump-over-ssh