When trying to restore backup in Hawq database first time. Getting this error.
Input command
I tried running this command
[gpadmin#mdsby backup]$ pg_restore -d gpadminrestore gpadmin_10-01-2017
Error :
pg_restore: [custom archiver] unexpected end of file
probably wrong format.
try to cat the backup file. if it is all sql statement, you need to use psql -f to load the data.
BTW, if the data size is big, don't use pg_backup and pg_restore. Those backup and restore go through master and will be slow.
Related
I have sql-extension db-file and I need to load it using the --no-security-labels flag.
I usually load data from file with the command:
psql -U postgres -d db_test < db_test_dump.sql
If I try
psql -U postgres -d db_test --no-security-labels < db_test_dump.sql
then I get an error:
psql: unrecognized option `--no-security-labels'
I also tried like this:
pg_restore -d db_test -U postgres --no-security-labels db_test_dump.sql
but that doesn't work either:
pg_restore: error: input file appears to be a text format dump. Please use psql
How to upload data to db from .sql file with flag --no-security-labels?
What is "sql-extension db-file"? And how does that relate to the rest of your question?
That option is only available on pg_dump and pg_restore. You can redo the dump with that flag, or redo the dump in custom format (or one of the other formats supported by pg_restore) and then supply that flag to pg_restore. But either way, you need to redo the dump. If you don't have access to the original database, perhaps you can set up a scratch database, load the dump to it, and dump from there with the correct option or in the correct format.
Or you could manually edit the existing dump file to remove the security label bits.
I'm currently trying to restore a database from a backup created by pg_dump (which is small ~100mb) from a tar file.
However attempting to restore with
pg_restore -U postgres -f "example.tar"
just keeps hanging and never seems to work.
What's the best way to try to diagnose and fix the issue?
It's waiting for input on stdin.
-f specifies an output file, not an input file. With no input it's waiting for input on stdin. If you type ctrl-d to end input it will say something like pg_restore: error: input file is too short (read 0, expected 5).
The proper invocation is...
pg_restore --dbname=DatabaseName -Upostgres --format=tar example.tar
--dbname is the name of the database to restore to. If it does not exist, add --create to tell pg_restore to create it.
The --format=tar is not strictly necessary, pg_restore can guess the file type, but why risk it?
See the pg_restore docs.
I was given a mydb.pg_dump file. I have created a new database test1 with pgAdmin. When I click right on the database and select Restore..., I can select my file after selecting "All files". But when I attempt to restore the database I immediately obtain this error:
pg_restore: error: input file appears to be a text format dump. Please
use psql.
Can I use pgAdmin to restore a .pg_dump file? How?
You cannot use pgAdmin for that, because in a plain-text dump COPY statements are mixed with COPY data.
You need psql for that, which fortunately is already installed on your machine:
psql -d mydb -U postgres -f mydb.pg_dump
So I regularly backup and restore databases and schema's using pgadmin4. I would like to do it with a batch file using commands as pg_dump and pg_restore. I however always fail to succeed in this and could use some help. The way I try to dump one schema (with data) is the following:
pg_dump -U postgres -F c -d database -n schema > mw2
Then I try to restore it with pg_restore:
pg_restore -U postgres -d otherdatabase -n schema mw2
First I tried to use .dump in stead of tar but the result stays the same; which is an empty schema in my database.
Or the following error message:
pg_restore: [archiver] input file does not appear to be a valid archive
https://www.postgresql.org/docs/current/static/app-pgrestore.html
--format=format Specify format of the archive. It is not necessary to specify the format, since pg_restore will determine the format
automatically. If specified, it can be one of the following:
custom, directory and tar
https://www.postgresql.org/docs/current/static/app-pgdump.html
--format=format
Selects the format of the output. format can be one of the following:
p plain Output a plain-text SQL script file (the default).
others are custom, directory and tar
in short - you used defualt plain format, which is meant for using with psql, not pg_restore. So either specify different format with pg_dump or use your file as
psql -f mw2.tar
I used this command to backup 200GB database (postgres 9.1, win7 x64):
pg_dump -Z 1 db_name > backup
It created 16GB file, which is fine I think because previous backups which works (and were packed by ext. tools) had similar size. Now, when I'm trying to restore into PG9.2 using pg_restore, I'm getting the error:
input file does not appear to be a valid archive
With pg_restore -Ft:
[tar archiver] corrupt tar header found in ▼ (expected 13500752, com
puted 78268) file position 512
Gzip also shows it's corrupted. When I open the backup file in Total Commander, the inner file has only 1.8GB.
When I was looking for a solution, dump should be done with -Cf parameter probably.
Which format has the file right now? Is it only tar or gzip (winrar shows gzip)?
Is there any way to restore this properly or is it corrupted somehow (no error when dumped)? Could it be due to file size limitations of tar or gzip?
What you have as output in "backup" is just zipped plain sql.
You could check it by prompting:
gzip -l backup
Unfortunately pg_retore do not provide possibility to restore PLAIN SQL,
so you just need to decompress the file and use psql -f <FILE> command:
zcat backup > backup.sql
psql -f backup.sql
It is not possible to make dump with pg_dump -Fc from postgres 9.1 as proposed by "Frank Heikens",
because dump formats are not compatible between primary versions, like 9.0 -> 9.1 -> 9.2
and "pg_restore" will give you an error on 9.2
Mostly this error mean that your restore action used invalid format
From manual of pg_dump ( pg_dump --help )
-F, --format=c|d|t|p output file format (custom, directory, tar,
plain text (default))
This mean that if you create dump with pg_dump without option --format / -F that your dump will be created in plain text format
NOTE: Plain text format cannot be restored with pg_restore tool. Use psql < dump.sql instead.
Examples:
# plain text export/import
pg_dump -Fp -d postgres://<db_user>:<db_password>#<db_host>:<db_port>/<db_name> > dump.sql
psql -d postgres://<target_db_user>:<target_db_password>#<target_db_host>:<target_db_port>/<target_db_name> -f dump.sql
# custom format
pg_dump -Fc -d postgres://<db_user>:<db_password>#<db_host>:<db_port>/<db_name> > dump.sql.custom
pg_restore -Ft postgres://<target_db_user>:<target_db_password>#<target_db_host>:<target_db_port>/<target_db_name> dump.sql.custom
# tar format
pg_dump -Ft -d postgres://<db_user>:<db_password>#<db_host>:<db_port>/<db_name> > dump.sql.tar
pg_restore -Ft postgres://<target_db_user>:<target_db_password>#<target_db_host>:<target_db_port>/<target_db_name> dump.sql.tar
Error from subject also can occur when restoring format not match backup.
For example created dump will be in custom format but for restore specified tar
Your dump is plain SQL, it's not a tar format, like you try to use in pg_restore. Use --format=custom or -Fc when you want a compressed format and use this setting in pg_restore as well. Check the manual.
This is an old thread though I had the exact same issue and managed to fix the somewhat corrupted dump with fixgz:
Short answer: run fixgz http://www.gzip.org/fixgz.zip on compressed dump.
fixgz.exe bad.gz fixed.gz
Long answer:
So if you used pg_dump with --compresss or -Z without specifying custom format option (-Fc) what you actually get is a compressed file in ASCII mode instead of BINARY mode.
Quoting from http://www.gzip.org/#faq1
If you have transferred a file in ASCII mode and you no longer have
access to the original, you can try the program fixgz to remove the
extra CR (carriage return) bytes inserted by the transfer. A Windows
9x/NT/2000/ME/XP binary is here. But there is absolutely no guarantee
that this will actually fix your file. Conclusion: never transfer
binary files in ASCII mode.
I got this problem when restoring using PGAdmin III. The problem doesn't occur with PGAdmin 4.