I`m trying to create DB cluster on PostgreSQL 14.1, Windows 10 Pro 19044 by following command through PowerShell:
.\pg_ctl.exe init -o "-E UTF8 --locale=ru_RU.UTF8 -U postgres" -D <cluster dir>
The creation fails with
invalid byte sequence for encoding UTF8: 0xd0 0xe0
The cluster dir does not contain any non-ASCII characters. Although it works correctly if I simply omit -E and --locale, but what the reason behind? May powershell cause this?
You are logged into Windows as an operating system user whose name is not all ASCII characters. Log on as a Windows user with all ASCII character in the name and try again.
Related
I make a backup with the command
docker exec -t arcane-aio pg_dump arcane-aio -c -U admin > arcane_aio_db.sql
I restore the backup with the command
cat arcane_aio_db.sql | docker exec -i arcane-aio psql -U admin -d arcane-aio
All is good, but all Russian symbols are replaced by "?".
The string value before the restore is Привет, hi!.
The string value after the restore of the backup is ??????, hi!.
I checked the encoding of the backup, the database before the backup, the database after the restore, and they are the same (en_US.utf8). Could it be that this encoding don't support the Russian language?
We are using Windows.
After a change of the system encoding from Cyrillic to UTF-8,
the values in the data dump become correct.
But after the restore, we still see "?" instead of Russian symbols in the database.
the cat command uses your shell character encoding.
Did you try running simply the first part:
cat arcane_aio_db.sql
I bet it also shows the ???.
You need to set the charset to the same encoding on both sides. You probably have UTF-8 on one side and some russian language on the other.
The pipe, that writes to file is binary and doesn't care about the encoding, but cat does.
You can check your encoding with
echo $LANG
make sure it is UTF-8 on both sides and that should fix your issue.
** EDIT
a work-around is to do the backup and restore within the container:
#get into the container
docker exec -it arcane-aio /bin/bash
# in the container run:
pg_dump arcane-aio -c -U admin > arcane_aio_db.sql
# try restore:
cat arcane_aio_db.sql | psql -U admin -d arcane-aio
if that works, then it's an encoding issue between your docker container and local machine.
You can do the dump / restore within the container and copy the file in/out with docker cp
On another thought, the SQL you 'cat' may contain quotes or $ or # or other characters that are problematic sent directly into a TTY.
So you may want to try this instead, to make sure the whole thing is quoted:
eval "echo \"$(cat arcane_aio_db.sql)\"" | docker exec -t arcane-aio psql -U admin -d arcane-aio
Since a pg_dump includes instructions to set the client_encoding correctly, the data in your target database will be correct. That is, unless the database encoding is SQL_ASCII, in which case you are lost anyway if you need Cyrillic characters.
The problem must be with your client software or your terminal encoding.
To ascertain that the characters are correct in the database, connect with psql and cast the string to bytea so that you can see the bytes:
SELECT charcol, CAST(charcol TO bytea)
FROM tab WHERE ...;
Hi my database is setup as UTF-8 encoding. I use cmd (windows) and ssh to connect to my postgres database. In CMD all polish letters work fine (ś,ą,ż...). the problem is when I connect with my database by:
psql -U database;
then polish letters don't exist, and I do not know how, any ideas why?
ps. when I use PGAdmin polish letters work fine, but if is possible I prefer to use command line
I'm working in a centralized monitoring system on Windows 2008 R2, I have installed a PostgreSQL 9.3 to use psql from the command line.
When I try to access to some remote Postgres (an 8.4 in my principal case) I have an error with the encoding:
command:
psql.exe -h 192.168.114.12 -p 5432 -d db_seros_transaccion -U postgres -f script.sql
error:
psql: FATAL: la conversión entre WIN1252 y LATIN1 no está soportada
I try adding the sentence
SET client_encoding = 'UTF8';
in my script but the problem persist (and with other encodings too, like LATIN1 & WIN1252).
After googling it I found people that update some rows in the server to make the connection, and this is a problem to me.
Can anyone help me to make a connection using psql without an update? Is it possible?
Thanks a lot Craig Ringer, works, finally works! You are my new idool now!
The steps are:
open the cmd
SET PGCLIENTENCODING=utf-8
chcp 65001
psql -h your.ip.addr.ess -U postgres
Windows 10 / Windows server 2016 or later:
Open Windows Control Panel
Select Region (and Language)
Click Change system locale
Beta: Use Unicode UTF-8 for worldwide language support
Click OK
or
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage]
"ACP"="65001"
"OEMCP"="65001"
"MACCP"="65001"
The PowerShell console and CMD will display Cyrillic correctly
As an option, try enter this command in psql :
postgres=# \! chcp 65001
65001 is equals UTF-8
1251 = WIN 1251
I have a UTF8 database and a UTF8 script to fill tables with data. However I want to run this script with psql -d instance -U user -f fillTables.sql. As my system has a Windows CP1252 encoding it looks like psql uses this to parse the file. I found this documentation and saw these backslash commands, but don't get it working
psql \encoding UTF8 -d instance -U user -f fillTables.sql
It looks like these are meant for starting psql and entering commands inside the psql console, right? How can I set different encoding for a batch processing of different files?
I got it working with export PGCLIENTENCODING=UTF8 (in cygwin, there is another syntax for windows), but would accept other answers if they can achieve the same with an option of psql.
I am trying to execute an SQL query which is stored in the file. I am using following command to execute:
psql -d DB_NAME -a -f QUERY_NAME.sql
I have some non English text in the SQL file like - સુરત
When the query is executed the text in the database looks like - à ª¸à «Âà ª°à ª¤
How do I execute the query from command line so that it runs correctly?
Make sure the client_encoding matches the encoding of your file. Check your system locale. Then use a matching command line argument for psql. Quoting the manual here:
If at least one of standard input or standard output are a terminal,
then psql sets the client encoding to "auto", which will detect the
appropriate client encoding from the locale settings (LC_CTYPE
environment variable on Unix systems). If this doesn't work out as
expected, the client encoding can be overridden using the environment
variable PGCLIENTENCODING.
Example for a Linux shell:
env PGCLIENTENCODING='WIN1258' psql DB_NAME -a -f QUERY_NAME.sql
List of available encodings in the manual.