I have a column in PostgreSQL that is of type bytea that usually has text data. I want to get the value of that column for a certain row with newlines and tabs intact rather than the octal escape characters that psql is outputting. For example, I run:
psql -Atc 'SELECT my_column from my_table limit 1;'
And I get output like:
Foo\015\012Bar\011This is some text.
Instead I want:
Foo
Bar This is some text.
I realize I can just use grep, which is what I'm doing, but I'm wondering if there's some simple way to do this via psql. I tried type casting the value to type text, but that didn't seem to help.
Try to convert your bytea in this way to ascii:
psql -Atc "SELECT convert_from (my_column, 'SQL_ASCII') from my_table limit 1;"
Related
With theses row in my table :
From intellij, when I run my PSQL request, lo_get result is correctly return (here a JSON content)
Select lo_get(cast(my_col as bigint)) from my_table
my_column is "text" format
But when i execute the same request in psql console on my postgres server, the result is not correctly return
example :
\x7b22646174654c6976726169736f6e223a22323032322d30322d32325432333a30303a30302e3030302b30303030222c22666f726d617443616e74696e6573223a5b5d2c226d6f6e74616e7450616e696572456e436f757273223a307d
Is there a way to get these value in psql columns ?
If I encode it in 'escape' method, I get something which looks like JSON, but which doesn't look anything at all like what you show in your image (which does not look like JSON in the least).
select encode('\x7b22646174654c6976726169736f6e223a22323032322d30322d32325432333a30303a30302e3030302b30303030222c22666f726d617443616e74696e6573223a5b5d2c226d6f6e74616e7450616e696572456e436f757273223a307d'::bytea,
'escape');
If you want to always encode it like this, you could set bytea_output TO escape. But while that might make sense in the psql interactive tool, it probably isn't preferable as a permanent server setting, i.e. when dealing with other clients.
Using psql on my shell, I am trying to display the output of a SELECT CLAUSE in a specific way.
I want each column value of my rows to be displayed on its own line.
Right now, my output looks like this (the display I don’t want):
Expanded display is on.
idOne nameOne createdAtOne
idTwo nameTwo createdAtTwo
But I want it to look like this
Expanded display is on.
idOne
nameOne
createdAtOne
idTwo
nameTwo
createdAtTwo
My command line is the following
psql -h hostName -d databaseName -U userName -p 5432 -c '\x on' -c "
COPY (
SELECT
e.id,
e.name,
e.created_at
FROM entity_name e
) TO STDOUT
;
"
I would like each value of a column in a row of my table to be displayed on its own separate line.
But although I have used the -c ‘\x on’ in my command line, and even if the sentence Expanded display is on is displayed, my output is still displayed the first way and not in an expanded way.
How can I do this? What am I missing?
My psql version on my remote server is psql (PostgreSQL) 9.6.6 and I am using MAC.
(ps: if it possible, I would even like my output to look like this
id idOne
name nameOne
created_at createdAtOne
id idTwo
name nameTwo
created_at createdAtTwo
https://www.postgresql.org/docs/current/static/app-psql.html#APP-PSQL-META-COMMANDS
This command sets options affecting the output of query result tables
in other words it affects only tables displayed, while COPY to STDOUT shows you the output of resulting generated tsc or csv. thus \x does not affect it.
Also from your query - I see no need in COPY - you just copy from query to stdout - thus omitting copy and just running
SELECT
e.id,
e.name,
e.created_at
FROM entity_name e
in expanded display will give expected result. You will have to grep -v 'RECORD' though
I got the answer to check for one certain BOM in a PostgreSQL text column. What I really like to do would be to have something more general, i.e. something like
select decode(replace(textColumn, '\\', '\\\\'), 'escape') from tableXY;
The result of a UTF8 BOM is:
\357\273\277
Which is octal bytea and can be converted by switching the output of bytea in pgadmin:
update pg_settings set setting = 'hex' WHERE name = 'bytea_output';
select '\357\273\277'::bytea
The result is:
\xefbbbf
What I would like to have is this result as one query, e.g.
update pg_settings set setting = 'hex' WHERE name = 'bytea_output';
select decode(replace(textColumn, '\\', '\\\\'), 'escape') from tableXY;
But that doesn't work. The result is empty, probably because the decode cannot handle hex output.
If the final purpose is to get the hexadecimal representation of all the bytes that constitute the strings in textColumn, this can be done with:
SELECT encode(convert_to(textColumn, 'UTF-8'), 'hex') from tableXY;
It does not depend on bytea_output. BTW, this setting plays a role only at the final stage of a query, when a result column is of type bytea and has to be returned in text format to the client (which is the most common case, and what pgAdmin does). It's a matter of representation, the actual values represented (the series of bytes) are identical.
In the query above, the result is of type text, so this is irrelevant anyway.
I think that your query with decode(..., 'escape') can't work because the argument is supposed to be encoded in escape format and it's not, per comments it's normal xml strings.
With the great help of Daniel-Vérité I use this general query now to check for all kind of BOM or unicode char problems:
select encode(textColumn::bytea, 'hex'), * from tableXY;
I had problem with pgAdmin and too long columns, as they had no result. I used that query for pgAdmin:
select encode(substr(textColumn,1,100)::bytea, 'hex'), * from tableXY;
Thanks Daniel!
I need to export content of a db2 table to CSV file.
I read that nochardel would prevent to have the separator between each data but that is not happening.
Suppose I have a table
MY_TABLE
-----------------------
Field_A varchar(10)
Field_B varchar(10)
Field_A varchar(10)
I am using this command
export to myfile.csv of del modified by nochardel select * from MY_TABLE
I get this written into the myfile.csv
data1 ,data2 ,data3
but I would like no ',' separator like below
data1 data2 data3
Is there a way to do that?
You're asking how to eliminate the comma (,) in a comma separated values file? :-)
NOCHARDEL tells DB2 not to surround character-fields (CHAR and VARCHAR fields) with a character-field-delimiter (default is the double quote " character).
Anyway, when exporting from DB2 using the delimited format, you have to have some kind of column delimiter. There isn't a NOCOLDEL option for delimited files.
The EXPORT utility can't write fixed-length (positional) records - you would have to do this by either:
Writing a program yourself,
Using a separate utility (IBM sells the High Performance Unload utility)
Writing an SQL statement that concatenates the individual columns into a single string:
Here's an example for the last option:
export to file.del
of del
modified by nochardel
select
cast(col1 as char(20)) ||
cast(intcol as char(10)) ||
cast(deccol as char(30));
This last option can be a pain since DB2 doesn't have an sprintf() function to help format strings nicely.
Yes there is another way of doing this. I always do this:
Put select statement into a file (input.sql):
select
cast(col1 as char(20)),
cast(col2 as char(10)),
cast(col3 as char(30));
Call db2 clp like this:
db2 -x -tf input.sql -r result.txt
This will work for you, because you need to cast varchar to char. Like Ian said, casting numbers or other data types to char might bring unexpected results.
PS: I think Ian points right on the difference between CSV and fixed-length format ;-)
Use "of asc" instead of "of del". Then you can specify the fixed column locations instead of delimiting.
I have a table with a bytea field, and it would be convenient if I could do queries via the command line (or pgAdmin's query executor). I've got the hex value as a string. Is there a built in function to convert hex to bytea?
I'd like to do something like:
SELECT * FROM table WHERE my_bytea_field=some_???_function('fa26e312');
where 'fa26e312' is the hex value of the bytea field I want.
Note: this is just to be helpful while I'm developing / debugging things, I can do it via code but I'd like to be able to do it by hand in a query.
Try using built-in decode(string text, type text) function (it returns bytea). You can run queries via CLI using psql in non-interactive mode, that is with -c switch (there are some formatting options if you like):
psql -c "SELECT * FROM table WHERE my_bytea_field=decode('fa26e312', 'hex');"
Example:
CREATE TABLE test(id serial, my_bytea_field bytea);
INSERT INTO test (my_bytea_field) VALUES
(E'\\320\\170'::bytea),
(E'\\100\\070'::bytea),
(E'\\377\\377'::bytea);
psql -tc "SELECT * FROM test WHERE my_bytea_field=decode('ffff', 'hex');"
3 | \377\377
SELECT * FROM table WHERE my_bytea_field=E'\\xfa26e312';
Just as in the example in the Binary Data Types docs (note the E'\\x' prefix):
SELECT E'\\xDEADBEEF';
under psql console, you can see its bytea content by default, such as:
also you can query like this (such as in pg_admin) :
xxx=# select id, encode(code_hash, 'hex') from your_table where id = 1195;
id | encode
------+------------------------------------------------------------------
1195 | c5e5dcf215925f7ef4dfaf5f4b4f105bc321c02776d6e7d52a1db3fcd9d011a4