Problem
A simple 2x2 table of data explains my problem. Both databases can be made to work, but they behave differently and I need them to be the same.
PostreSQL Query | Local DB | Amazon-RDS |
--------------------------+------------+-----------------------------------------+
SELECT * from mydb.users; | Success | Success |
--------------------------+------------+-----------------------------------------+
SELECT * from users; | Success | ERROR: relation "users" does not exist |
---------------------------------------------------------------------------------+
Details
The databases should be identical. Amazon-RDS is literally pg_restore'd from a pg_dump of the local database. Exact commands:
$ pg_dump --format=c ---no-privileges --no-owner --verbose \
--host=localhost --port=5432 --username=gary mydb;
$ pg_restore --no-owner --no-tablespaces --dbname=mydb --verbose \
--host=127.0.0.1 --port=47737 \ #ssh tunnel
--username=XXXXXX --format=c
The problem is not with the data dump itself. I've wiped the local database, restored it from the dump, and it still behaves the way it's supposed to.
The problem doesn't just manifest not just with my raw SQL queries, there's a sizable Node/Express app that is supposed to front-end the database. It generates queries without the database prefix in front of the tables too. The app uses Sequelize for an ORM and has been running with MySQL on Amazon-RDS in
production for years. The issue I'm seeing now has only appeared while migrating from MySQL to PostgreSQL.
I have no experience with Postgres.
I don't think it should matter, but in full disclosure, I'm using DBeaver to handle all my database connections, and do the db dump and restore.
Questions
Why does one database successfully infer the database from the name of the table alone, and not the other cannot?
Is there a configuration setting somewhere to make them both work? mydb is the only database in the RDS instance.
mydb is not a database, it's a schema. And it appears that it is not in the schema search_path on RDS.
It could be configured in the cluster settings, the settings of your database, the settings of your (login) user, or locally on the DBeaver connection.
Related
Is there a simple way to create a copy of a database or schema in PostgreSQL 8.1?
I'm testing some software which does a lot of updates to a particular schema within a database, and I'd like to make a copy of it so I can run some comparisons against the original.
If it's on the same server, you just use the CREATE DATABASE command with the TEMPLATE parameter. For example:
CREATE DATABASE newdb WITH TEMPLATE olddb;
pg_dump with the --schema-only option.
If you have to copy the schema from the local database to a remote database, you may use one of the following two options.
Option A
Copy the schema from the local database to a dump file.
pg_dump -U postgres -Cs database > dump_file
Copy the dump file from the local server to the remote server.
scp localuser#localhost:dump_file remoteuser#remotehost:dump_file
Connect to the remote server.
ssh remoteuser#remotehost
Copy the schema from the dump file to the remote database.
psql -U postgres database < dump_file
Option B
Copy the schema directly from the local database to the remote database without using an intermediate file.
pg_dump -h localhost -U postgres -Cs database | psql -h remotehost -U postgres database
This blog post might prove helpful for you if you want to learn more about options for copying the database using pg_dump.
This can be done by running the following command:
CREATE DATABASE [Database to create] WITH TEMPLATE [Database to copy] OWNER [Your username];
Once filled in with your database names and your username, this will create a copy of the specified database. This will work as long as there are no other active connections to the database you wish to copy. If there are other active connections you can temporarily terminate the connections by using this command first:
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = '[Database to copy]'
AND pid <> pg_backend_pid();
A good article that I wrote for Chartio's Data School which goes a bit more in depth on how to do this can be found here:
https://dataschool.com/learn/how-to-create-a-copy-of-a-database-in-postgresql-using-psql
I am having some difficulties with restoring the schema of a table. I dumped my Heroku Postgres db and I used pg_restore to restore one table from it into my local db (it has more than 20 tables). It was successfully restored, but I was having issues when I tried to insert new data into the table.
When I opened up my database using psql, I found out that the restored table is available with all the data, but its schema has zero rows. Is there anyway I could import both the table and its schema from the dump? Thank you very much.
This is how I restored the table into my local db:
pg_restore -U postgres --dbname my_db --table=message latest.dump
Edit:
I tried something like this following the official docs, but it just gets blocked and nothing happened. My db is small, no more than a couple of megabytes and the table's schema I am trying to restore has no more than 100 row.
pg_restore -U postgres --dbname mydb --table=message --schema=message_id_seq latest.dump
As a more general answer (I needed to restore a single table from a huge backup), you may want to take a look at this post: https://thequantitative.medium.com/restoring-individual-tables-from-postgresql-pg-dump-using-pg-restore-options-ef3ce2b41ab6
# run the schema-only restore as root
pg_restore -U postgres --schema-only -d new_db /directory/path/db-dump-name.dump
# Restore per table data using something like
pg_restore -U postgres --data-only -d target-db-name -t table_name /directory/path/dump-name.dump
From the Heroku DevCenter here
Heroku Postgres is integrated directly into the Heroku CLI and offers
many helpful commands that simplify common database tasks
You can check here if your environment is correctly configured.
In this way, you can use the Heroku CLI pg:pull command to pull remote data from a Heroku Postgres database to a local database on your machine.
For example:
$ heroku pg:pull HEROKU_POSTGRESQL_MAGENTA mylocaldb --app sushi
I'm looking to load a database from a backup.gz. The backup is raw sql generated from pg_dump -U postgres app_development -f backup.gz -Z9.
I've tried dropping the db with psql -Upostgres -c "drop database app_development" but I get:
ERROR: database "app_development" is being accessed by other users
DETAIL: There are 3 other sessions using the database.
The same thing happens when I use dropdb.
I don't want to dump to a non-ascii version so I don't think I can use pg_restore.
Also, I'm not sure if it helps, but all this is happening in docker.
First of all, I am new to PostgreSQL.
So am I right thinking that one cannot run most of the psql util commands nor non-db-specified sql commands if there isn't a db with same name of the current user's?
That is saying e.g., if I run psql "show databases;" as user postgres while there isn't a db called "postgres", I won't be able to run the command.
Question is that in this case, one cannot find the list of the dbs before knowing any of db exits, is that how it works?
You have to connect to a database. By default, the databases "template1" and "postgres" will exist and will accept connections.
If your PostgreSQL admin has changed things in such a way that you can't connect to either of those databases, you'll have to do one of two things.
Ask the PostgreSQL admin what database you're supposed to connect to.
Create a database, then connect to it. There's more than one way to do this.
If you have CREATEDB privileges, you can create a database on the psql command line. For example, I have CREATEDB privileges here, so I can do this, which creates the database "mike" and exits.
$ psql -h localhost -p 5435 -U mike -c "create database mike"
Now I can connect to "mike" by either taking advantage of the default database name, or by specifying it.
$ psql -h localhost -p 5435 -U mike
$ psql -h localhost -p 5435 -U mike mike
You can. If you connect (with proper user, usually postgres) to the postgres database there are several tables on the pg_catalog (PostgreSQL) among those is pg_database table a simple select * from pg_database will show all databases.
Here is an image showing that on pgAdmin III Tool
There is no way of doing exactly what you want without, at least, knowing the database catalog. The postgres database is default and will exist in all installed instances (unless someone had droped it). All RDBMs is the same they all have the catalog (also named information_schema, or other names depending on vendor) which holds all information about the databases, tables, constraints, etc.
I search for a while to find this answer but with no luck.
The situation:
I have Postgresql currently running on my production environment. I am preparing to scale my database and move it to a large server instance. I made the mistake of setting up the initial database with the postgres user who has all permissions, and I would like the new database to be controlled by a custom user I have created. ie The current database's owner is postgres, and I want the new database owner to be pooper.
To dump, I am running:
pg_dump -d database_name > database_name.sql
To restore on separate machine, I am running:
psql database_name < database_name.sql
If the user is the same, ie both postgres, then it will work just fine, but when switching users, my app does not load correctly. Is there a secret to the madness. Nothing stood out to me.
My system:
Debian Wheezy
Postgresql 9.1
Postgis Extension
pg_dump with the --no-owner flag (see pg_dump --help)
Create the new db with the new owner CREATE DATABASE foo OWNER pooper;,
Load via psql -U pooper -d database_name -f database_name.sql.