I need to import my psql dump into my fresh psql database. When I execute the following command, I get errors.
psql -U user new_database < filename.sql
Error I got:
ERROR: out of memory
DETAIL: Cannot enlarge string buffer containing 0 bytes by 1208975751 more bytes.
How do i fix this. Aldo, is there any method to log the import process?
Thanks.
I think the most common cause is a "corrupt" SQL file. There's no easy fix. Split the file in half (man split), fix the SQL statements at the tail end of one resulting file and at the head end of the other, and run again. In the past, I seem to recall seeing "Invalid byte sequence for UTF-8", or something like that. I'm sure there are other causes.
PostgreSQL has a lot of logging options; set them in postgresql.conf and restart PostgreSQL. Look at
log_destination
logging_collector
client_min_messages
log_error_verbosity
log_statement
Related
Ok, I screwed up.
I dumped one of my psql (9.6.18) staging database with the following command
pg_dump -U postgres -d <dbname> > db.out
And after doing some testing, I "restored" the data using the following command.
psql -f db.out postgres
Notice the absence of -d option? yup. And that was supposed to be the username.
Annnd as the database happend to have the same name as its user, it overwrote the 'default' database (postgres), which had data that other QAs are using.
I cancelled the operation quickly as soon as I realised my mistake, but the damage was still done. Around 1/3 ~ 1/2 of the database is roughly identical to the staging database - at least in terms of the schema.
Is there any way to revert this? I am still looking for any other dumps if any of these guys made one. But I don't think there is any past two to three months. Seems like I got no choice but to own up and apologise to them in the morning.
Without a recent dump or some sort of PITR replication setup, you can't un-revert this easily. The only option is to manually go through the log of what was restored and remove/alter it in the postgres database. This will work for the schema, the data is another matter. FYI, the postgres database should not really be used as a 'working' database. It is there to be a database to connect to for doing other operations, such as CREATE DATABASE or to bootstrap your way into a cluster. If left empty then the above would not have been a problem. You could have done, from another database, DROP DATABASE postgres; and then CREATE DATABASE postgres.
Do you have a capture of the output of the psql -f db.out postgres run?
Since the pg_dump didn't specify --clean or -c, it should not have overwritten anything, just appended. And if your tables have unique or primary keys, most of the data copy operations should have failed with unique key violations and rolled back. Even one overlapping row (per table) would roll back the entire dataset for that table.
Without having the output, it will be hard to figure out what damage has actually been done.
You should also immediately copy the pg_xlog data someplace safe. If it comes down to it, you might be able to use pg_xlogdump to figure out what changes committed and what did not.
I made a backup of a database of postgres, it is not the first time I do it, I used this command:
pg_dump db -f /backup/agosto_31.sql
And I do the restore with this:
psql -d August_31 -f August_31.sql
But this time I did not import any trigger, and there are many. I checked in the file August_31.sql and they are. How could I import them again? Only the triggers.
Thanks everyone, greetings!
There are not any possibility to import only triggers. But, because your dump (backup file) is in SQL format (plain text), you can cut trigger definition manually in any editor. For this case is practical dump data and schema to separate files.
Is possible so import (restore from backup) fails, probably due broken dependencies. Check psql output if there are some errors. psql has nice possibility to stop on first error:
psql -v ON_ERROR_STOP=1
Use this option. Without error specification is not possible to help more.
I am very new to PostgreSQL so I apologize if the question is elementary.
During PostgreSQL database restore, from sql file, I am getting an error "invalid input syntax for type bytea" and I believe the data is not copied to the table, i.e. the table is empty.
This is the error message:
2015-02-20 08:56:14 EST ERROR: invalid input syntax for type bytea
2015-02-20 08:56:14 EST CONTEXT: COPY ir_ui_menu, line 166, column web_icon_data: "\x6956424f5277304b47676f414141414e5355684555674141414751414141426b43414d41414142485047566d4141414143..."
2015-02-20 08:56:14 EST STATEMENT: COPY ir_ui_menu (id, parent_id, name, icon, create_uid, create_date, write_date, write_uid, web_icon_data, web_icon, sequence, web_icon_hover, web_icon_hover_data) FROM stdin;
The database backup dump is created like this:
pg_dump -U user_name database_name -f backup_file.sql
The database restore is done like this:
psql -U user_name -d destination_db -f backup_file.sql
Source database (to get backup from) is PostgreSQL version 9.1.15 on one server and destination (to restore to) database is PostgreSQL 8.3.4, on another server.
Is there any way to resolve this issue? Thanks in advance for your help.
Restoring a dump from a newer version of Postgres onto an older is quite often problematic, and there is no automated way that I am aware of. Making this work will most likely require editing the dump file manually.
Specifically, Postgres 9.0 changed the handling of escape strings used with bytea: previous versions treated \ in regular string literals such as '\' as escape characters, whereas newer versions use the escape string syntax E'\'.
If you have access to your 9.X server configuration, you can change bytea_output variable to 'escape' on postgresql.conf:
bytea_output = 'escape' # hex, escape
Then restart Postgres 9.X server and dump the database as you do it normally. Finally, restore it on the 8.X server.
You can also change client connection variable just for the database dump action, but it may be outside scope.
Hope it helps.
Unfortunately, PostgreSQL 9 backups are not backward compatible, and there are no options to enable this compatibility. I've been stuck on this problem for hours.
You'll need to figure out which statements are not compatible, and find their equivalent (if there's one) in PostgreSQL 8.
Another option, the better one, would be to upgrade your v8 to the latest v9.
I know this issue has already been raised by others, but even trying previous suggestions I still get this error...
When I try to populate a table copying from a csv file, I get a permission error.
COPY Eurasia FROM '/Users/Oritteropus/Desktop/eurasia1.csv' CSV HEADER;
ERROR: could not open file "/Users/Oritteropus/Desktop/eurasia1.csv" for reading: Permission denied
SQL state: 42501
As previously suggested in these cases, I changed the permission of the file (chmod 711 eurasia1.csv or chmod a+r eurasia1.csv) and I also changed the user rights with:
ALTER USER postgres WITH SUPERUSER; #where postgres is my user
However, I still get the same error.
I also tried to manually change the privileges from pgAdmin but seems avery privilege is already given.
I'm working on a Mac Os and I'm using PostGreSQL 9.2.4.
Any suggestion? Thanks
The best option is to change and use COPY FROM STDIN as that avoids quite a number of permissions issues.
Alternatively you can make sure that the postgres user can access the file. This rarely better than COPY FROM STDIN however for a couple reasons.
COPY TO STDOUT can conceivably corrupt your data. Because this involves file I/O by PostgreSQL if bugs exist in COPY FROM STDIN that could be a problem too.
If you are doing it on the server side because of automation/stored proc concerns, this is rarely a win, as you are combining transactional and non-transactional effects. COPY TO STDOUT and COPY FROM STDIN do not have these issues. (For example, you don't have to wonder whether the atime of the inode actually means the file was properly processed).
I am trying to move a POSTGRESQL database from one server to another. In order to do so, I did a pg_dump and then after creating a new database on the new server, I tried to restore the pg_dumped file. For the most part, the restore was alright, but then one table did not copy over.
pg_restore: [archiver (db)] COPY failed: ERROR: invalid byte sequence for encoding "UTF8": 0x92
HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".
Now, after checking database properties, it turns out that the original table was encoded in SQL_ASC2, but the new one that I created is UTF8.
I don't know anything about encoding, but isn't UTF8 backward compatible with ASC2? So, why is there an invalid byte sequence?
Would changing the new database to one that uses SQL_ASC2 fix this problem?
If I have to change the encoding of the new database, how do I do it? Can I just change it, or do I have to start from scratch and remake the entire database?
Thanks for the help!
Before connecting to the database you could set client_encoding to 'LATIN9' (what it probably is; anyway: it will be accepted) You can do this by:
1) issuing pg_restore with a -f my_filename flag.
2) Edit the resulting file (there probably already is a "SET client_encoding = 'UTF8';" present near the top.)
3) submit with "psql -U username dbname < filename". (in most cases you'll have to supply a different username or dbname; the "\connect newdbname" is situated at the top of the script and will take over. Sometimes you even need to create a user first)
If you are on a *nix box and your pg_dump file is plain text, then you could try running the dump file through iconv prior to importing it into postgres.