Given a database dump how to specify a tablespace to be used by all tables during restore? The database has multiple tablespaces used by its table. Old tablespaces should be ignored (they are not relevant on new computer) and all tablespaces my by replaced by a new one.
dump with "--no-tablespaces" parameter to have tablespaces-free dump - but you can also use the same parameter on pg_restore if you cannot change dump commands
set global parameter "default_tablespace" on target DB to what is needed for restore (for example by using alter database xxxxx set DEFAULT_TABLESPACE='xxx')
run all pg_restore tasks
if necessary reset default_tablespace to original value
Related
I dumped the database with:
pg_dump dsmartgpp -f /data1/master/backup/db.sql
and I copied the database with
nohup psql -h gpnm1 -d dsmartgpp_1 -U gpadmin < /data1/master/backup/db.sql
the log information showed some errors about the function and datatype of postgis, such as
ALTER FUNCTION
ERROR: function "gidx_out" already exists with same argument types.
ALTER FUNCTION
ERROR: type "gidx" already exists
some tables between the two database have the same number or records, but about 1MB difference in size.
That is as expected. Restoring a pg_dump creates a logical copy of the database. In particular, the copy won't be bloated (don't worry – a bit of bloat is healthy).
If you want to avoid these errors, create the new database (dsmartgpp_1) with template0 as template (empty database). You seem to have PostGIS installed in template1, which gets copied into every new database. Or you installed PostGIS in your new database, before importing the dump.
Either way, create the empty database (dsmartgpp_1) and let the dump install the PostGIS functions.
Oh, one more thing, you can use "-f /data1/master/backup/db.sql" instead of the "<" redirect.
If you want to be super careful, also add "-v ON_ERROR_STOP=1". This will stop at the first error.
I have server1 and server2 in my environment, both have db2 v11.1 installed.
I have already done an online tablespaces (TS1, TS2, TS3) incremental backup of my database GS_DB and obtained the below 3 images.
Image1 at timestamp1: 20190215162151 (full online backup of TS1,TS2,TS3)
Image2 at timestamp2: 20190215162254 (incremental online backup of TS1,TS2,TS3)
Image3 at timestamp3: 20190215162725 (incremental online backup of TS1,TS2,TS3)
In server1, suppose I want to restore my db to image2 (20190215162254), I can do:
db2ckrst -d GS_DB -t 20190215162254 -r tablespace
Suggested restore order of images using timestamp 20190215162254 for database gs_db.
====================================================================
restore db gs_db tablespace ( TS1, TS2, TS3 ) incremental taken at 20190215162254
restore db gs_db incremental taken at 20190215162151
restore db gs_db incremental taken at 20190215162254
====================================================================
If I follow the order and restore to the existing GS_DB in server1, it is working fine.
Now I transferred 3 images to server2 and created an empty database GS_DB in server2, then try to use the above command to restore tablespaces TS1,TS2,TS3 to GS_DB in server2:
db2 restore db gs_db2 tablespace ( TS1, TS2, TS3 ) incremental taken at 20190215162254
SQL2560N The table space restore operation failed because the target database is not identical to the source database.
Already stuck at the first command, does it mean we cannot restore tablespace backup image across two different db? Any way I can do it?
Thanks in advance!
Every database has an unique internal identifier called Seed. You can't create another database with the same Seed as the existing one, even you create it with the same name. These databases are different from the DB2's point of view.
Citation from the Restoring to an existing database article:
The database manager assigns the seed when you create the database.
Db2® always uses the seed from the backup image.
You can restore a table space into an existing database only if the table space exists and if the table spaces are the same, meaning that you did not drop
the table space and then re-create it between the backup and the
restore operations.
The database on disk and in the backup image must
be the same.
So, yes, you are not able to restore tablespace backup image across two different db in the way you try.
Read about the Database schema transporting feature.
I am starting to use PostgreSQL and I am confused about the two ways to create a database. When I installed it the first time, the instructions said I have to create a default database with initdb /usr/local/var/postgres When I lookup my databases, I can see that I have a database called postgres. Now I am able to create a database with two other commands whereas the former is the command line script and the latter the SQL command. In the case of a "postgres" called database it would be:
createdb postgres
CREATE DATABASE postgres
Both are setting up a database in my list of databases. When I try to create another database with initdb /usr/local/var/[someDbName] though, it doesn't appear in my list of databases. So what's the difference between initdb and createdb then?
initdb is not used to create a "new database".
As documented in the manual you need it to create a "cluster" or "data directory" which then stores databases created with create database.
Quote from the manual:
Before you can do anything, you must initialize a database storage area on disk. We call this a database cluster. (The SQL standard uses the term catalog cluster.) A database cluster is a collection of databases that is managed by a single instance of a running database server
[...]
In file system terms, a database cluster is a single directory under which all data will be stored. We call this the data directory or data area
In short: initdb creates the necessary directory layout on the harddisk to be able to create and manage databases.
It's a necessary part of the installation process of a Postgres server.
I tried pg_dump and then on a separate machine I tried to import the sql and populate the database, I see
CREATE TABLE
ERROR: role "prod" does not exist
CREATE TABLE
ERROR: role "prod" does not exist
CREATE TABLE
ERROR: role "prod" does not exist
CREATE TABLE
ERROR: role "prod" does not exist
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
WARNING: no privileges could be revoked for "public"
REVOKE
ERROR: role "postgres" does not exist
ERROR: role "postgres" does not exist
WARNING: no privileges were granted for "public"
GRANT
which means my user and roles and grant information is not in pg_dump
On the other hand we have pg_dumpall, I read conversation, and this does not lead me anywhere?
Question
- Which one should I be using for database backups? pg_dump or pg_dumpall?
- the requirement is that I can take the backup and should be able to import to any machine and it should work just fine.
The usual process is:
pg_dumpall --globals-only to get users/roles/etc
pg_dump -Fc for each database to get a nice compressed dump suitable for use with pg_restore.
Yes, this kind of sucks. I'd really like to teach pg_dump to embed pg_dumpall output into -Fc dumps, but right now unfortunately it doesn't know how so you have to do it yourself.
Up until PostgreSQL 11 there was also a nasty caveat with this approach: Neither pg_dump, nor pg_dumpall in --globals-only mode would dump user access GRANTs on DATABASEs. So you pretty much had to extract them from the catalogs or filter a pg_dumpall. This is fixed in PostgreSQL 11; see the release notes.
Make pg_dump dump the properties of a database, not just its contents (Haribabu Kommi)
Previously, attributes of the database itself, such as database-level GRANT/REVOKE permissions and ALTER DATABASE SET variable settings, were only dumped by pg_dumpall. Now pg_dump --create and pg_restore --create will restore these database properties in addition to the objects within the database. pg_dumpall -g now only dumps role- and tablespace-related attributes. pg_dumpall's complete output (without -g) is unchanged.
You should also know about physical backups - pg_basebackup, PgBarman and WAL archiving, PITR, etc. These offer much "finer grained" recovery, down to the minute or individual transaction. The downside is that they take up more space, are only restoreable to the same PostgreSQL version on the same platform, and back up all tables in all databases with no ability to exclude anything.
I have backups of my postgres database - the entire database instance in a single nightly backup. Is it possible to restore just one of the database from within that backup? Or if I want to access individual databases (for migration or restoring), do I need to change my database backup scheme to do individual dumps?
You can access individual databases from a full database cluster dump in text format (pg_dumpall) by using some text processing like this:
awk '/^\\connect database_name/ {flag=1;print;next}
/^\\connect/ {flag=0}
flag { print }' \
< all_databases.sql \
> database_name.sql
This would get from pg_dumpall file everything between "\connect database_name" and next "\connect". But it is not very efficient.
But I'd recommend dumping every database separately like this:
# Dumping global data (for example roles)
pg_dumpall -g > /var/lib/pgsql/backups/globals.sql
#Dumping indidual databases in tar (uncompressed binary) format
for dbname in
`
psql -qXtc "
select datname from pg_catalog.pg_database
where datname<>'template0'" template1
`
do
pg_dump -b -F t "$dbname" > "/var/lib/pgsql/backups/$dbname.dump"
done
I'm assuming you mean "Can I restore just one database from a pg_dumpall of the entire database cluster?"
Yes, if your backup is "an archive created by pg_dump in one of the non-plain-text formats." - use the "--schema=" option to pg_restore.
On the other hand I'm not sure you're using the correct terminology here - you refer to a DB cluster as "entire database instance"; in the explanation you ask about "database" but in the title you wrote "schema", etc.
Edit: Now, after some deliberation, I believe you have a cluster backup, created with "pg_dumpall". "pg_dumpall" creates only plain-text (SQL-commands) backups. In that case it's not possible to restore only one database (or only one schema from a database).
Then yes, you need to change your backup procedure to backup individual databases using non-plain-text format (--format=custom is the recommended one).
Actually the backup procedure I use on my DB servers does just that.