How can I set numeric display format in psql?
I have many character varying and double precision columns in the select query. I can use to_char() or round() on each of the numeric columns, but with many columns this makes the code too repetitive.
Is there a shortcut? For example, are there any psql session settings, something like \pset numeric '9.9EEEEE' (I just made it up, don't try to use it). I could not quickly find such settings in the manual: PostgreSQL: Documentation: psql.
Example:
-- Got this:
=# select bar from (values (123456789), (0.123456789), (0.000000123456789)) table_foo (bar);
bar
-------------------
123456789
0.123456789
0.000000123456789
-- I have to use this workaround:
=# select to_char(bar, '9.9EEEEE') as bar from (values (123456789), (0.123456789), (0.000000123456789)) table_foo (bar);
bar
----------
1.2e+08
1.2e-01
1.2e-07
-- I want this:
-- set some session settings, and then magically:
=# select bar from (values (123456789), (0.123456789), (0.000000123456789)) table_foo (bar);
bar
----------
1.2e+08
1.2e-01
1.2e-07
There is not other possibility to specify number of format then specification of numeric locale. Values are formatted on server side already, psql does formatting to final output format (table, html, csv), but the values are formatted, and psql doesn't try to do reformat (numericlocale is an exception). I remember long discussion about possibility to specify output format of boolean type, but still without actual results. psql has not strong functionality for generating rich reports. It should be fast, safe and robust interactive client. Although I would to have some stronger possibility in this area, the modern spreadsheets and reporting tools are for some tasks better.
You could always define your own formatting function, but you'd have to be doing a lot of formatting on a lot of columns for this to be considered an improvement in readability. I wouldn't recommend it for your toy example but if your real queries have dozens of columns it might be neater.
sophia=> create function x(numeric) returns text language sql as $$ select to_char($1, '9.9EEEEE') $$;
CREATE FUNCTION
sophia=> select x(bar) from (values (123456789), (0.123456789), (0.000000123456789)) table_foo (bar);
x
----------
1.2e+08
1.2e-01
1.2e-07
(3 rows)
sophia=>
Related
The dump function in Oracle displays the internal representation of data:
DUMP returns a VARCHAR2 value containing the data type code, length in bytes, and internal representation of expr
Fore example:
SELECT DUMP(cast(1 as number ))
2 FROM DUAL;
DUMP(CAST(1ASNUMBER))
--------------------------------------------------------------------------------
Typ=2 Len=2: 193,2
SQL> SELECT DUMP(cast(1.000001 as number ))
2 FROM DUAL;
DUMP(CAST(1.000001ASNUMBER))
--------------------------------------------------------------------------------
Typ=2 Len=5: 193,2,1,1,2
It shows that the first 1 uses 2 byte for storing and the second example uses 5 bytes for storing.
I suppose the similar function in PostgreSQL is pg_typeof but it returns only the type name without information about byte usage:
SELECT pg_typeof(33);
pg_typeof
integer (1 row)
Does anybody know if there is an equivalent function in PostgreSQL?
I don't speak PostgreSQL.
However, Oracle functionality page says that there's Orafce which
implements in Postgres some of the functions from the Oracle database that are missing (or behaving differently)
It, furthermore, mentions the dump function
dump (anyexpr [, int]): Returns a text value that includes the datatype code, the length in bytes, and the internal representation of the expression
One of examples looks like this:
postgres=# select pg_catalog.dump('Pavel Stehule',10);
dump
-------------------------------------------------------------------------
Typ=25 Len=17: 68,0,0,0,80,97,118,101,108,32,83,116,101,104,117,108,101
(1 row)
To me, it looks like Oracle's dump:
SQL> select dump('Pavel Stehule') result from dual;
RESULT
--------------------------------------------------------------
Typ=96 Len=13: 80,97,118,101,108,32,83,116,101,104,117,108,101
SQL>
I presume you'll have to visit GitHub and install the package to see whether you can use it or not.
It is not a complete equivalent, but if you want to figure out the byte values used to encode a string in PostgreSQL, you can simply cast the value to bytea, which will give you the bytes in hexadecimal:
SELECT CAST ('schön' AS bytea);
This will work for strings, but not for numbers.
I've got a Postgres ORDER BY issue with the following table:
em_code name
EM001 AAA
EM999 BBB
EM1000 CCC
To insert a new record to the table,
I select the last record with SELECT * FROM employees ORDER BY em_code DESC
Strip alphabets from em_code usiging reg exp and store in ec_alpha
Cast the remating part to integer ec_num
Increment by one ec_num++
Pad with sufficient zeors and prefix ec_alpha again
When em_code reaches EM1000, the above algorithm fails.
First step will return EM999 instead EM1000 and it will again generate EM1000 as new em_code, breaking the unique key constraint.
Any idea how to select EM1000?
Since Postgres 9.6, it is possible to specify a collation which will sort columns with numbers naturally.
https://www.postgresql.org/docs/10/collation.html
-- First create a collation with numeric sorting
CREATE COLLATION numeric (provider = icu, locale = 'en#colNumeric=yes');
-- Alter table to use the collation
ALTER TABLE "employees" ALTER COLUMN "em_code" type TEXT COLLATE numeric;
Now just query as you would otherwise.
SELECT * FROM employees ORDER BY em_code
On my data, I get results in this order (note that it also sorts foreign numerals):
Value
0
0001
001
1
06
6
13
۱۳
14
One approach you can take is to create a naturalsort function for this. Here's an example, written by Postgres legend RhodiumToad.
create or replace function naturalsort(text)
returns bytea language sql immutable strict as $f$
select string_agg(convert_to(coalesce(r[2], length(length(r[1])::text) || length(r[1])::text || r[1]), 'SQL_ASCII'),'\x00')
from regexp_matches($1, '0*([0-9]+)|([^0-9]+)', 'g') r;
$f$;
Source: http://www.rhodiumtoad.org.uk/junk/naturalsort.sql
To use it simply call the function in your order by:
SELECT * FROM employees ORDER BY naturalsort(em_code) DESC
The reason is that the string sorts alphabetically (instead of numerically like you would want it) and 1 sorts before 9.
You could solve it like this:
SELECT * FROM employees
ORDER BY substring(em_code, 3)::int DESC;
It would be more efficient to drop the redundant 'EM' from your em_code - if you can - and save an integer number to begin with.
Answer to question in comment
To strip any and all non-digits from a string:
SELECT regexp_replace(em_code, E'\\D','','g')
FROM employees;
\D is the regular expression class-shorthand for "non-digits".
'g' as 4th parameter is the "globally" switch to apply the replacement to every occurrence in the string, not just the first.
After replacing every non-digit with the empty string, only digits remain.
This always comes up in questions and in my own development and I finally tired of tricky ways of doing this. I finally broke down and implemented it as a PostgreSQL extension:
https://github.com/Bjond/pg_natural_sort_order
It's free to use, MIT license.
Basically it just normalizes the numerics (zero pre-pending numerics) within strings such that you can create an index column for full-speed sorting au naturel. The readme explains.
The advantage is you can have a trigger do the work and not your application code. It will be calculated at machine-speed on the PostgreSQL server and migrations adding columns become simple and fast.
you can use just this line
"ORDER BY length(substring(em_code FROM '[0-9]+')), em_code"
I wrote about this in detail in this related question:
Humanized or natural number sorting of mixed word-and-number strings
(I'm posting this answer as a useful cross-reference only, so it's community wiki).
I came up with something slightly different.
The basic idea is to create an array of tuples (integer, string) and then order by these. The magic number 2147483647 is int32_max, used so that strings are sorted after numbers.
ORDER BY ARRAY(
SELECT ROW(
CAST(COALESCE(NULLIF(match[1], ''), '2147483647') AS INTEGER),
match[2]
)
FROM REGEXP_MATCHES(col_to_sort_by, '(\d*)|(\D*)', 'g')
AS match
)
I thought about another way of doing this that uses less db storage than padding and saves time than calculating on the fly.
https://stackoverflow.com/a/47522040/935122
I've also put it on GitHub
https://github.com/ccsalway/dbNaturalSort
The following solution is a combination of various ideas presented in another question, as well as some ideas from the classic solution:
create function natsort(s text) returns text immutable language sql as $$
select string_agg(r[1] || E'\x01' || lpad(r[2], 20, '0'), '')
from regexp_matches(s, '(\D*)(\d*)', 'g') r;
$$;
The design goals of this function were simplicity and pure string operations (no custom types and no arrays), so it can easily be used as a drop-in solution, and is trivial to be indexed over.
Note: If you expect numbers with more than 20 digits, you'll have to replace the hard-coded maximum length 20 in the function with a suitable larger length. Note that this will directly affect the length of the resulting strings, so don't make that value larger than needed.
In my table results from column work_time (interval type) display as 200:00:00. Is it possible to cut the seconds part, so it will be displayed as 200:00? Or, even better: 200h00min (I've seen it accepts h unit in insert so why not load it like this?).
Preferably, by altering work_time column, not by changing the select query.
This is not something you should do by altering a column but by changing the select query in some way. If you change the column you are changing storage and functional uses, and that's not good. To change it on output, you need to modify how it is retrieved.
You have two basic options. The first is to modify your select queries directly, using to_char(myintervalcol, 'HH24:MI')
However if your issue is that you have a common format you want to have universal access to in your select query, PostgreSQL has a neat trick I call "table methods." You can attach a function to a table in such a way that you can call it in a similar (but not quite identical) syntax to a new column. In this case you would do something like:
CREATE OR REPLACE FUNCTION myinterval_nosecs(mytable) RETURNS text LANGUAGE SQL
IMMUTABLE AS
$$
SELECT to_char($1.myintervalcol, 'HH24:MI');
$$;
This works on the row input, not on the underlying table. As it always returns the same information for the same input, you can mark it immutable and even index the output (meaning it can be run at plan time and indexed used).
To call this, you'd do something like:
SELECT myinterval_nosecs(m) FROM mytable m;
But you can then use the special syntax above to rewrite that as:
SELECT m.myinterval_nosecs FROM mytable m;
Note that since myinterval_nosecs is a function you cannot omit the m. at the beginning. This is because the query planner will rewrite the query in the former syntax and will not guess as to which relation you mean to run it against.
In postgresql if I want percentages I just write:
select x / sum(x) over() ...
Inside a function it doesn't work since aggregate functions don't behave well.
I tried to find a solution but with no success.
This is a simple version of what I really need, but I believe the solution to this problem would surely point me in the right direction.
Some more details...
If I create this simple table:
create table ttt(v1 numeric, v2 numeric);
insert into ttt values (2,1),(5,2),(10,4);
If I run:
select v1/sum(v1) over() from ttt; --returns relative frequencies
I get:
select v1/sum(v1) over() from ttt;
?column?
------------------------
0.11764705882352941176
0.29411764705882352941
0.58823529411764705882
(3 rows)
Now, if I want to create a function which does the same thing, I would write:
create or replace function rfreq (double precision)
returns double precision
AS
'
select
$1 / sum($1) over()
'
LANGUAGE 'sql';
I get:
select rfreq(v1) from bruto;
rfreq
-------
1
1
1
(3 rows)
Postgresql is not summing up inside a function.
Any suggestions?
Thank you,
Ali.
To debug your function, write the query with arbitrary parameters in a text file, and then use psql to run it:
\i ./myfunc.sql
Content of myfunc.sql would be:
select x / sum(y) over (...) ...
This will allow you to debug the function before wrapping it in a function.
When you're done and happy with the results for a few samples, copy/paste it into your function, and replace the hard-coded test values with parameters where applicable.
As to optimizing it when it has parameters, I'm not aware of any means to run explain analyze within the Postgres function, but you can get a plan which -- best I'm aware -- is the same as the function will use by preparing a statement with the same parameters. So you can explain analyze the latter instead.
Seeing the new details, note that if you prepare the query that you're running in function, you should always get 1 -- bar with zero.
You've an error in there, in the sense that you'd need to keep state from a call to the next first to return the expected result. Per Pavel's suggestion, you actually need a custom aggregate or a custom window function here. See the link he suggested in a comment, as well as:
http://www.postgresql.org/docs/current/static/xaggr.html
I found the solution browsing through the pl/r mailing list.
Percentages (or relative frequencies) can be calculated in postgres using the following code:
CREATE OR REPLACE
FUNCTION rel_freq(float8)
RETURNS float8 AS
$BODY$
var <- as.vector(farg1)
return((var/sum(var))[prownum]
$BODY$
LANGUAGE plr WINDOW;
I would like to know if there is a built-in way to convert integer values into numerals in PostgreSQL?
As an example, is it possible to convert the integer 10 into the string TEN.
Thank You.
There's nothing built-in. For this sort of thing your best bet will be to make use of PostgreSQL's pluggable procedural languages. Use PL/Perl or PL/Python with a suitable Perl or Python library to do the job.
In this case I'd probably use PL/Perl with Lingua::EN::Numbers.
CREATE OR REPLACE FUNCTION num2en(numeric) RETURNS text AS $$
use Lingua::EN::Numbers qw(num2en);
return num2en($_[0]);
$$ LANGUAGE plperlu;
You'll need to install Lingua::EN::Numbers into the Perl being used by PostgreSQL using CPAN or system packages first. In my case (Fedora 19) this was a simple yum install perl-Lingua-EN-Numbers.noarch, then I could:
regress=> SELECT num2en(10);
num2en
--------
ten
(1 row)
regress=# SELECT num2en(NUMERIC '142.5');
num2en
--------------------------------------
one hundred and forty-two point five
(1 row)
By default the function is accessible by normal users so you don't have to issue any extra GRANTs.
Try this query:
SELECT split_part (cash_words (10::VARCHAR::MONEY), 'dollar', 1);
It's a internal function of PostgreSQL.