I search for a while to find this answer but with no luck.
The situation:
I have Postgresql currently running on my production environment. I am preparing to scale my database and move it to a large server instance. I made the mistake of setting up the initial database with the postgres user who has all permissions, and I would like the new database to be controlled by a custom user I have created. ie The current database's owner is postgres, and I want the new database owner to be pooper.
To dump, I am running:
pg_dump -d database_name > database_name.sql
To restore on separate machine, I am running:
psql database_name < database_name.sql
If the user is the same, ie both postgres, then it will work just fine, but when switching users, my app does not load correctly. Is there a secret to the madness. Nothing stood out to me.
My system:
Debian Wheezy
Postgresql 9.1
Postgis Extension
pg_dump with the --no-owner flag (see pg_dump --help)
Create the new db with the new owner CREATE DATABASE foo OWNER pooper;,
Load via psql -U pooper -d database_name -f database_name.sql.
Related
Problem
A simple 2x2 table of data explains my problem. Both databases can be made to work, but they behave differently and I need them to be the same.
PostreSQL Query | Local DB | Amazon-RDS |
--------------------------+------------+-----------------------------------------+
SELECT * from mydb.users; | Success | Success |
--------------------------+------------+-----------------------------------------+
SELECT * from users; | Success | ERROR: relation "users" does not exist |
---------------------------------------------------------------------------------+
Details
The databases should be identical. Amazon-RDS is literally pg_restore'd from a pg_dump of the local database. Exact commands:
$ pg_dump --format=c ---no-privileges --no-owner --verbose \
--host=localhost --port=5432 --username=gary mydb;
$ pg_restore --no-owner --no-tablespaces --dbname=mydb --verbose \
--host=127.0.0.1 --port=47737 \ #ssh tunnel
--username=XXXXXX --format=c
The problem is not with the data dump itself. I've wiped the local database, restored it from the dump, and it still behaves the way it's supposed to.
The problem doesn't just manifest not just with my raw SQL queries, there's a sizable Node/Express app that is supposed to front-end the database. It generates queries without the database prefix in front of the tables too. The app uses Sequelize for an ORM and has been running with MySQL on Amazon-RDS in
production for years. The issue I'm seeing now has only appeared while migrating from MySQL to PostgreSQL.
I have no experience with Postgres.
I don't think it should matter, but in full disclosure, I'm using DBeaver to handle all my database connections, and do the db dump and restore.
Questions
Why does one database successfully infer the database from the name of the table alone, and not the other cannot?
Is there a configuration setting somewhere to make them both work? mydb is the only database in the RDS instance.
mydb is not a database, it's a schema. And it appears that it is not in the schema search_path on RDS.
It could be configured in the cluster settings, the settings of your database, the settings of your (login) user, or locally on the DBeaver connection.
I am having some difficulties with restoring the schema of a table. I dumped my Heroku Postgres db and I used pg_restore to restore one table from it into my local db (it has more than 20 tables). It was successfully restored, but I was having issues when I tried to insert new data into the table.
When I opened up my database using psql, I found out that the restored table is available with all the data, but its schema has zero rows. Is there anyway I could import both the table and its schema from the dump? Thank you very much.
This is how I restored the table into my local db:
pg_restore -U postgres --dbname my_db --table=message latest.dump
Edit:
I tried something like this following the official docs, but it just gets blocked and nothing happened. My db is small, no more than a couple of megabytes and the table's schema I am trying to restore has no more than 100 row.
pg_restore -U postgres --dbname mydb --table=message --schema=message_id_seq latest.dump
As a more general answer (I needed to restore a single table from a huge backup), you may want to take a look at this post: https://thequantitative.medium.com/restoring-individual-tables-from-postgresql-pg-dump-using-pg-restore-options-ef3ce2b41ab6
# run the schema-only restore as root
pg_restore -U postgres --schema-only -d new_db /directory/path/db-dump-name.dump
# Restore per table data using something like
pg_restore -U postgres --data-only -d target-db-name -t table_name /directory/path/dump-name.dump
From the Heroku DevCenter here
Heroku Postgres is integrated directly into the Heroku CLI and offers
many helpful commands that simplify common database tasks
You can check here if your environment is correctly configured.
In this way, you can use the Heroku CLI pg:pull command to pull remote data from a Heroku Postgres database to a local database on your machine.
For example:
$ heroku pg:pull HEROKU_POSTGRESQL_MAGENTA mylocaldb --app sushi
I used following command to backup the DB on the source server:
pg_dump --no-acl -h "server" --port "1234" -U "admin" --no-owner dbname > dbname_dump.sql
Then on new server I created empty database newdb and used this command to restore database
psql -U postgres -d newdb -f dbname_dump.sql
created new user, assigned all permissions to new database. When I connected with that new user to restored database and when I wanted to do select on any table inside public schema I am getting access denied. Why?
So I managed to get around the problem, in short the problem was admin user which was used for restored was owner of everything restored and creating new DB user for app usage and transferring everything to new user is a nightmare. So here are the steps to do this proper way:
as admin backup source database with --no-acl and --no-owner switch
on new server create new empty database
on new server create db user that will be used for app to connect to newly created database
grant all permissions to created user on newly created database
restore database with that new user into newly created database
Now app is working and I can run database migration scripts so everything is working as expected. Since I am PGSQL novice I wasn't aware only owner can make deletes or alterations on the table but after I read the manual things got much clearer.
We have the database named 'itreport' on production server and database named 'itreport_dev' on development server.
1)On Production server, 52 users are present in the database 'itreport'.
2)On Development server, 60 users are in present the database 'itreport_dev'.
3)I have taken the dump of production server database 'itreport'. Dump file name is backup_12082017.sql
My question is
If I restore the above dump(backup) file to Development server database 'itreport_dev, Users(60) present will present in the Development database?
If not what option we have to give in the restore process?
What are the pre steps and post steps to be performed on Develpement server?
Short answer: No, roles are not part of a single-database backup.
If you dump only the database using pg_dump it will only restore tables and data. not any roles. any objects owned by missing roles will end up owned by the user performing the restore (this user should be a superuser)
If you do pg_dumpall roles and all databases will be backed up.
Roles can be backed up separately using pg_dumpall -r
if you do pgdumpall --clean the resore will destroy and replace any databases and roles on the dev server that also exist in the dump. any names that are not in both will be unaffected, (the special role "postgres" and template databases also are untouched)
pgdumpall backups are SQL backups and should be restores using psql
su postgres -c psql < all-database-backupfile.sql
or
zcat all-database-backupfile.sql.gz | su postgres -c psql
etc.
(for windows use runas instead of su, I'm not sure of the exact syntax needed)
First of all, I am new to PostgreSQL.
So am I right thinking that one cannot run most of the psql util commands nor non-db-specified sql commands if there isn't a db with same name of the current user's?
That is saying e.g., if I run psql "show databases;" as user postgres while there isn't a db called "postgres", I won't be able to run the command.
Question is that in this case, one cannot find the list of the dbs before knowing any of db exits, is that how it works?
You have to connect to a database. By default, the databases "template1" and "postgres" will exist and will accept connections.
If your PostgreSQL admin has changed things in such a way that you can't connect to either of those databases, you'll have to do one of two things.
Ask the PostgreSQL admin what database you're supposed to connect to.
Create a database, then connect to it. There's more than one way to do this.
If you have CREATEDB privileges, you can create a database on the psql command line. For example, I have CREATEDB privileges here, so I can do this, which creates the database "mike" and exits.
$ psql -h localhost -p 5435 -U mike -c "create database mike"
Now I can connect to "mike" by either taking advantage of the default database name, or by specifying it.
$ psql -h localhost -p 5435 -U mike
$ psql -h localhost -p 5435 -U mike mike
You can. If you connect (with proper user, usually postgres) to the postgres database there are several tables on the pg_catalog (PostgreSQL) among those is pg_database table a simple select * from pg_database will show all databases.
Here is an image showing that on pgAdmin III Tool
There is no way of doing exactly what you want without, at least, knowing the database catalog. The postgres database is default and will exist in all installed instances (unless someone had droped it). All RDBMs is the same they all have the catalog (also named information_schema, or other names depending on vendor) which holds all information about the databases, tables, constraints, etc.