I have a postgres database and I'm trying to use pg_dumpall to create the full script to reproduce my database (also using github to keep track of versions of the dump file).
My question is, why does pg_dumpall lose my database schemas? For instance, I have a schema called "go1". My pg_dumpall script should produce a file with "go1.users" for instance for my users table in schema "go1". But the dump file only produces "users". It loses the schema name.
Is there a way for pg_dumpall to preserve the schemas in the dump file?
It does not lose it. View the generated file - you will find search_path before creating schema objects, like:
...
SET search_path = instruction, pg_catalog;
SET default_tablespace = '';
SET default_with_oids = false;
--
-- Name: file; Type: TABLE; Schema: instruction; Owner: postgres
--
CREATE TABLE file (
details jsonb,
...
would mean it will create instruction.file table
Related
Given a database dump how to specify a tablespace to be used by all tables during restore? The database has multiple tablespaces used by its table. Old tablespaces should be ignored (they are not relevant on new computer) and all tablespaces my by replaced by a new one.
dump with "--no-tablespaces" parameter to have tablespaces-free dump - but you can also use the same parameter on pg_restore if you cannot change dump commands
set global parameter "default_tablespace" on target DB to what is needed for restore (for example by using alter database xxxxx set DEFAULT_TABLESPACE='xxx')
run all pg_restore tasks
if necessary reset default_tablespace to original value
I am having an issue with pg_restore --clean not clearing the database.
Or do I misunderstand what the --clean does, I am expecting it to truncate the database tables and reinitialize the indexes/primary keys.
I am using 9.5 on rds
This is the full command we use
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U superuser -d mydatabase backup.dump
Basically what is happening is this.
I do a nightly backup of my production db, and restore it to an analytics db for the analyst to churn and run their reports.
I found out recently that the rails application used to view the reports was complaining that the primary keys were missing from the restored analytics database.
So I started investigating the production db, the analytics db etc. Which was when I realized that multiple rows with the same primary key existed in the analytics database.
I ran a few short experiments and realized that every time the pg_restore script is run it inserts duplicate data into the tables, this leads me to think that the --clean is not dropping and restoring the data. Because if I were to drop the schema beforehand, I don't get duplicate data.
To remove all tables from a database (but keep the database itself), you have two options.
Option 1: Drop the entire schema
You will need to re-create the schema and its permissions. This is usually good enough for development machines only.
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO public;
Applications usually use the "public" schema. You may encounter other schema names when working with a (legacy) application's database.
Note that for Rails applications, dropping and recreating the database itself is usually fine in development. You can use bin/rake db:drop db:create for that.
Option 2: Drop each table individually
Prefer this for production or staging servers. Permissions may be managed by your operations team, and you do not want to be the one who messed up permissions on a shared database cluster.
The following SQL code will find all table names and execute a DROP TABLE statement for each.
DO $$ DECLARE
r RECORD;
BEGIN
FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname = current_schema()) LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(r.tablename) || ' CASCADE'; -- DROP TABLE IF EXISTS instead DROP TABLE - thanks for the clarification Yaroslav Schekin
END LOOP;
END $$;
Original:
https://makandracards.com/makandra/62111-how-to-drop-all-tables-in-postgresql
Is there any way of storing database other than the fixed data directory in postgres? I have a situation where I need to store database at any location irrespective of data directory.
You can add a tablespace.
A tablespace is basically a location to store databases and/or tables. You create a tablespace using CREATE TABLESPACE:
CREATE TABLESPACE mytablespace LOCATION '/path/to/some/location';
You can then create tables directly in that tablespace:
CREATE TABLE whatever (thing integer) TABLESPACE mytablespace;
Or set the default tablespace:
SET default_tablespace = mytablespace;
You can also set the default tablespace at database creation time:
CREATE DATABASE mydatabase TABLESPACE mytablespace;
You are looking for the 'Data Directory' in linux for example its in
/usr/local/pgsql/data
if you install from source code. Each distribution is different though, try reading up on
File Locations in postgres and Creating a Database Cluster
also try using the query
show data_directory;
in windows try looking around
C:\Program Files (x86)\PostgreSQL\[VERSION]\data\global
I have created a tablespace named hdd_tablespace and I wan't all new databases to be automatically created there. Basically when I execute:
CREATE DATABASE FOO;
I want this database to be created in hdd_tablespace.
I have updated postgresql.conf with:
default_tablespace = 'hdd_tablespace'
However, new databases are still created in pg_default.
Actually it turned out I had also to update template1 database that is a template database.
I had to;
ALTER DATABASE template1 SET TABLESPACE hdd_tablespace;
I tried pg_dump and then on a separate machine I tried to import the sql and populate the database, I see
CREATE TABLE
ERROR: role "prod" does not exist
CREATE TABLE
ERROR: role "prod" does not exist
CREATE TABLE
ERROR: role "prod" does not exist
CREATE TABLE
ERROR: role "prod" does not exist
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
WARNING: no privileges could be revoked for "public"
REVOKE
ERROR: role "postgres" does not exist
ERROR: role "postgres" does not exist
WARNING: no privileges were granted for "public"
GRANT
which means my user and roles and grant information is not in pg_dump
On the other hand we have pg_dumpall, I read conversation, and this does not lead me anywhere?
Question
- Which one should I be using for database backups? pg_dump or pg_dumpall?
- the requirement is that I can take the backup and should be able to import to any machine and it should work just fine.
The usual process is:
pg_dumpall --globals-only to get users/roles/etc
pg_dump -Fc for each database to get a nice compressed dump suitable for use with pg_restore.
Yes, this kind of sucks. I'd really like to teach pg_dump to embed pg_dumpall output into -Fc dumps, but right now unfortunately it doesn't know how so you have to do it yourself.
Up until PostgreSQL 11 there was also a nasty caveat with this approach: Neither pg_dump, nor pg_dumpall in --globals-only mode would dump user access GRANTs on DATABASEs. So you pretty much had to extract them from the catalogs or filter a pg_dumpall. This is fixed in PostgreSQL 11; see the release notes.
Make pg_dump dump the properties of a database, not just its contents (Haribabu Kommi)
Previously, attributes of the database itself, such as database-level GRANT/REVOKE permissions and ALTER DATABASE SET variable settings, were only dumped by pg_dumpall. Now pg_dump --create and pg_restore --create will restore these database properties in addition to the objects within the database. pg_dumpall -g now only dumps role- and tablespace-related attributes. pg_dumpall's complete output (without -g) is unchanged.
You should also know about physical backups - pg_basebackup, PgBarman and WAL archiving, PITR, etc. These offer much "finer grained" recovery, down to the minute or individual transaction. The downside is that they take up more space, are only restoreable to the same PostgreSQL version on the same platform, and back up all tables in all databases with no ability to exclude anything.