I have two Postgres 9.3 servers in synchronous replication.
I had needed to restart the slave in order to load a new archive_cleanup_command in the recovery.conf.
The server restarted correctly and it's now perfectly in sync with the master server.
But when I open "Server status" panel for the slave server in PgAdmin III (which executable is located on the master server), I get some errors like this:
invalid byte sequence for encoding “UTF8” plus some hex codes
It might be because I put a tilde ~ in the archive_cleanup_command, but it didn't worked, then I removed it and the command worked correctly.
Maybe that ~ has been written somewhere and it's not a valid char... but I also deleted logs...
Log of the slave server has a lot of lines like the followings:
2015-02-13 11:11:32 CET ERROR: invalid byte sequence for encoding “UTF8”: 0xe8 0x20 0x73
2015-02-13 11:11:32 CET STATEMENT: SELECT pg_file_read('pg_log/postgresql-2015-02-13_111038.log', 0, 50000)
Note that postgresql-2015-02-13_111038.log is the last log, the one from which I got these lines.
The problem you have is that the locale setting lc_messages is set to an encoding that is different to the encoding of the database(s). As a result, some messages are being written into the log using Windows-1252 encoding, while when you try to use PgAdmin to view the log, it tries to interpret the file using UTF-8. Some of the byte sequences written in the log are not valid UTF-8, leading to the error.
In fact, the way in which different locales interact in postgresql can result in mixed encodings in the log file. There is a Bug Report on this, but it does not look like it has been resolved.
Probably the easiest way to resolve this is to set lc_messages to English_United States.UTF-8.
It would also be preferable to have lc_messages aligned across all of the databases on the server (or at least all using the same encoding).
Be sure to remove any existing log files as they will already contain the incorrect encoding.
It is due to your postgresql.log corrupted as stated in the statement 'select pf_file_read ....'.
If you do a "touch" (after a backup of your log perhaps) on you server log, and reconnect, you'll not see this unicode error anymore and thus, you'll be able to use pgadmin III furthermore.
Related
pg_dump is failing with the error message:
"pg_dump FATAL: segment too big"
What does that mean?
PostgreSQL 10.4 on Ubuntu 16.04.
It appears that pg_dump passes the error messages it receives from the queries it is running into the logs.
The following line in the logs (maybe buried deeper if you have busy logs), shows the query that failed.
In this case, we had a corrupted sequence. Any query on the sequence, whether it was interactive, via a column default, or via pgdump, returned the "segment too big" error, and killed the querying process.
I figured out the new start value for the sequence, dropped the dependencies, and created a new sequence starting where the old one left off and then put the dependencies back.
pg_dump worked fine after that.
It is not clear why or how a sequence could get so corrupted that you would have a session killing error when it was accessed. We did have a recent database hard-crash though, so it may be related. (Although that sequence is accessed very rarely and it is unlikely we went down in the middle of incrementing it.)
I'm using NodeJS to get some data from a PostgreSQL database and render it on the web. When I make a query on SQL Shell(psql) everything looks fine but when I console.log that same data from NodeJS, all the special characters are replaced with gibberish.
The encoding for this database is
- Encoding: UTF8
- Collation: French_France.1252
- Ctype: French_France.1252
I tried to set Client_Encoding to UTF8 but when I reconnect to the database I find out that it is still not set.
Also I get this warning each time I connect to the database (Just in case it may cause smth)
WARNING: Console code page (850) differs from Windows code page (1252)
8-bit characters might not work correctly. See psql reference
page "Notes for Windows users" for details.
My OS is Windows 8 and PostgreSQL version is 10.x.
After a long search I have found a solution that I want to share now.
Change your ODBC DSN connection string to include this: ConnSettings=SET CLIENT_ENCODING TO 'UTF8';
I am very new to PostgreSQL so I apologize if the question is elementary.
During PostgreSQL database restore, from sql file, I am getting an error "invalid input syntax for type bytea" and I believe the data is not copied to the table, i.e. the table is empty.
This is the error message:
2015-02-20 08:56:14 EST ERROR: invalid input syntax for type bytea
2015-02-20 08:56:14 EST CONTEXT: COPY ir_ui_menu, line 166, column web_icon_data: "\x6956424f5277304b47676f414141414e5355684555674141414751414141426b43414d41414142485047566d4141414143..."
2015-02-20 08:56:14 EST STATEMENT: COPY ir_ui_menu (id, parent_id, name, icon, create_uid, create_date, write_date, write_uid, web_icon_data, web_icon, sequence, web_icon_hover, web_icon_hover_data) FROM stdin;
The database backup dump is created like this:
pg_dump -U user_name database_name -f backup_file.sql
The database restore is done like this:
psql -U user_name -d destination_db -f backup_file.sql
Source database (to get backup from) is PostgreSQL version 9.1.15 on one server and destination (to restore to) database is PostgreSQL 8.3.4, on another server.
Is there any way to resolve this issue? Thanks in advance for your help.
Restoring a dump from a newer version of Postgres onto an older is quite often problematic, and there is no automated way that I am aware of. Making this work will most likely require editing the dump file manually.
Specifically, Postgres 9.0 changed the handling of escape strings used with bytea: previous versions treated \ in regular string literals such as '\' as escape characters, whereas newer versions use the escape string syntax E'\'.
If you have access to your 9.X server configuration, you can change bytea_output variable to 'escape' on postgresql.conf:
bytea_output = 'escape' # hex, escape
Then restart Postgres 9.X server and dump the database as you do it normally. Finally, restore it on the 8.X server.
You can also change client connection variable just for the database dump action, but it may be outside scope.
Hope it helps.
Unfortunately, PostgreSQL 9 backups are not backward compatible, and there are no options to enable this compatibility. I've been stuck on this problem for hours.
You'll need to figure out which statements are not compatible, and find their equivalent (if there's one) in PostgreSQL 8.
Another option, the better one, would be to upgrade your v8 to the latest v9.
I am trying to copy a table from on MySQL database on a remote machine to another MySQL database on my local machine. I noticed that after importing the dump to my local machine, there were characters like ’ instead of single quotes.
I assumed this was an encoding issue, so I went into both databases and ran show create table posts, near the end of both, I saw CHARSET=utf8. Also, I ran file -i on the dump file, both before and after scping it to my local machine, and they were both utf8.
However, when I import this file, I get this before:
attendees—policy makers,
and after:
attendees—policy makers,
I am not sure why this is happening, everything is using utf8, what am I missing?
EDIT: I am using mysql Ver 14.12 Distrib 5.0.75, for debian-linux-gnu (x86_64) remotely, and mysql Ver 14.14 Distrib 5.5.25a, for osx10.7 (i386) locally.
On both systems you must check that your connection encoding is correct:
SHOW VARIABLES LIKE 'character_set_%'
Usually seeing characters like that is the result of double-encoding. Make sure you can match up the connection and client encoding to be exactly the same. There is a number of command line options that can facilitate this, or if you're using a driver or client, something in there can tweak it.
I am trying to move a POSTGRESQL database from one server to another. In order to do so, I did a pg_dump and then after creating a new database on the new server, I tried to restore the pg_dumped file. For the most part, the restore was alright, but then one table did not copy over.
pg_restore: [archiver (db)] COPY failed: ERROR: invalid byte sequence for encoding "UTF8": 0x92
HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".
Now, after checking database properties, it turns out that the original table was encoded in SQL_ASC2, but the new one that I created is UTF8.
I don't know anything about encoding, but isn't UTF8 backward compatible with ASC2? So, why is there an invalid byte sequence?
Would changing the new database to one that uses SQL_ASC2 fix this problem?
If I have to change the encoding of the new database, how do I do it? Can I just change it, or do I have to start from scratch and remake the entire database?
Thanks for the help!
Before connecting to the database you could set client_encoding to 'LATIN9' (what it probably is; anyway: it will be accepted) You can do this by:
1) issuing pg_restore with a -f my_filename flag.
2) Edit the resulting file (there probably already is a "SET client_encoding = 'UTF8';" present near the top.)
3) submit with "psql -U username dbname < filename". (in most cases you'll have to supply a different username or dbname; the "\connect newdbname" is situated at the top of the script and will take over. Sometimes you even need to create a user first)
If you are on a *nix box and your pg_dump file is plain text, then you could try running the dump file through iconv prior to importing it into postgres.