I need to decode a base64 string and take a chunk of binary.
Is there a SQL function in Postgres to simply convert a bytea into a binary string representation?
(Like "00010001010101010".)
If your Postgres installation runs with the default setting bytea_output = 'hex', there is a very simple hack:
SELECT right(bytea_col::text, -1)::varbit;
Example:
SELECT right((bytea '\xDEADBEEF')::text, -1)::varbit;
Result:
'11011110101011011011111011101111'
right(text, -1) is just the cheapest way to remove the leading backslash from the text representation.
varbit (standard SQL name bit varying) is for bit strings of arbitrary length. Cast the result to text or varchar if you like.
Related, with explanation:
Convert hex in text representation to decimal number
demo:db<>fiddle
You could put the following code into a function:
WITH byte AS ( -- 1
SELECT E'\\xDEADBEEF'::bytea as value
)
SELECT
string_agg( -- 5
get_byte(value, gs)::bit(8)::text -- 4
, ''
)
FROM
byte,
generate_series( -- 3
0,
length(value) - 1 -- 2
) gs
I demonstrated the development of the query within the fiddle.
The WITH clause encapsulates the bytea value for double usage in further code
length() calculates the binary length of the bytea value
generate_series() creates a list from 0 to length - 1 (0 - 3 in my example)
get_byte() takes the bytea value a second time and gives out the byte at position gs (the previous calculated values 0-3). This gives an integer representation of the the byte. After that the cast to bit(8) type converts the result of this function to its binary representation (1 byte = 8 bit)
string_agg() finally aggregates all for binary strings into one. (taking its text representations instead of bit type, with no delimiters)
A function could look like this:
CREATE OR REPLACE FUNCTION to_bit(value bytea) RETURNS SETOF text AS
$$
BEGIN
RETURN QUERY
SELECT
string_agg(get_byte(value, gs)::bit(8)::text, '')
FROM
generate_series(0, length(value) - 1) gs;
END;
$$ LANGUAGE plpgsql;
After that you could call it:
SELECT to_bit(E'\\xDEADBEEF'::bytea)
You could try it using get_bit() instead of get_byte(). This safes you the ::bit(8) cast but of course you need to multiply the length with factor 8 indeed.
The resulting bit string has another bit order but maybe it fits your use case better:
WITH byte AS (
SELECT E'\\xDEADBEEF'::bytea as value
)
SELECT
string_agg(get_bit(value, gs)::text, '')
FROM
byte,
generate_series(0, length(value) * 8 - 1) gs
demo:db<>fiddle
I'm trying to run this query:
select count(uid),
case when duration=0 then '0'
when duration between 0 and 10 then '1-10 sec' end as Duration
from mr_session_log
where event_id=10
group by
case when duration=0 then '0'
when duration between 0 and 10 then '1-10 sec' end
But since the duration is integer Redshift doesn't allow me to enter "1-10 sec" value after the then.
Is there a way to to put different value from the original column value?
SQL is a strongly-typed language. #John explains the basic problem in the query nicely. But I think we need more precision how to handle data types here. I suggest:
SELECT count(uid) AS ct
, CASE WHEN duration = 0 THEN text '0 sec' -- explicit type for string literal
WHEN duration BETWEEN 1 AND 10 THEN '1-10 sec' -- coerced to same type
END AS duration
FROM mr_session_log
WHERE event_id = 10
GROUP BY 2;
You can use '0 sec'::text or cast('0 sec' AS text) as well.
In modern Postgres '0 sec' or even '0' without explicit type does the same because a string literal defaults to type text in most contexts.
You could even use 0::text or cast(0 AS text), but it's pointless to provide a string as numeric literal and make Postgres cast twice (first the numeric literal is automatically cast to int in this case, then int to text).
Read the chapter about constants in the Postgres manual. (I wouldn't know of proper documentation for Redshift.)
GROUP BY 2 is just syntax shorthand referring to the 2nd SELECT item. If that should not be implemented in Redshift, repeat the CASE expression like you did before.
The issue that you are mixing types within a column.
when duration=0 then 0 returns an integer
when duration between 0 and 10 then '1-10 sec' returns a string
They are both populating the same Duration output column (which is not the same as the original duration column)
I recommend you change the first result into a string, eg: when duration=0 then '0 sec'
I am using PostgreSQL via the Ruby gem 'sequel'.
I'm trying to round to two decimal places.
Here's my code:
SELECT ROUND(AVG(some_column),2)
FROM table
I get the following error:
PG::Error: ERROR: function round(double precision, integer) does
not exist (Sequel::DatabaseError)
I get no error when I run the following code:
SELECT ROUND(AVG(some_column))
FROM table
Does anyone know what I am doing wrong?
PostgreSQL does not define round(double precision, integer). For reasons #Mike Sherrill 'Cat Recall' explains in the comments, the version of round that takes a precision is only available for numeric.
regress=> SELECT round( float8 '3.1415927', 2 );
ERROR: function round(double precision, integer) does not exist
regress=> \df *round*
List of functions
Schema | Name | Result data type | Argument data types | Type
------------+--------+------------------+---------------------+--------
pg_catalog | dround | double precision | double precision | normal
pg_catalog | round | double precision | double precision | normal
pg_catalog | round | numeric | numeric | normal
pg_catalog | round | numeric | numeric, integer | normal
(4 rows)
regress=> SELECT round( CAST(float8 '3.1415927' as numeric), 2);
round
-------
3.14
(1 row)
(In the above, note that float8 is just a shorthand alias for double precision. You can see that PostgreSQL is expanding it in the output).
You must cast the value to be rounded to numeric to use the two-argument form of round. Just append ::numeric for the shorthand cast, like round(val::numeric,2).
If you're formatting for display to the user, don't use round. Use to_char (see: data type formatting functions in the manual), which lets you specify a format and gives you a text result that isn't affected by whatever weirdness your client language might do with numeric values. For example:
regress=> SELECT to_char(float8 '3.1415927', 'FM999999999.00');
to_char
---------------
3.14
(1 row)
to_char will round numbers for you as part of formatting. The FM prefix tells to_char that you don't want any padding with leading spaces.
((this is a Wiki! please edit to enhance!))
Try also the old syntax for casting,
SELECT ROUND( AVG(some_column)::numeric, 2 ) FROM table;
works with any version of PostgreSQL. ...But, as definitive solution, you can overload the ROUND function.
Overloading as casting strategy
CREATE FUNCTION ROUND(float,int) RETURNS NUMERIC AS $f$
SELECT ROUND( CAST($1 AS numeric), $2 )
$f$ language SQL IMMUTABLE;
Now your instruction will works fine, try this complete comparison:
SELECT trunc(n,3), round(n,3) n_round, round(f,3) f_round,
pg_typeof(n) n_type, pg_typeof(f) f_type, pg_typeof(round(f,3)) f_round_type
FROM (SELECT 2.0/3.0, 2/3::float) t(n,f);
trunc
n_round
f_round
n_type
f_type
f_round_type
0.666
0.667
0.667
numeric
double precision
numeric
The ROUND(float,int) function is f_round, it returns a (decimal) NUMERIC datatype, that is fine for some applications: problem solved!
In another applications we need a float also as result. An alternative is to use round(f,3)::float or to create a round_tofloat() function.
Other alternative, overloading ROUND function again, and using all range of accuracy-precision of a floating point number, is to return a float when the accuracy is defined (see IanKenney's answer),
CREATE FUNCTION ROUND(
input float, -- the input number
accuracy float -- accuracy, the "counting unit"
) RETURNS float AS $f$
SELECT ROUND($1/accuracy)*accuracy
$f$ language SQL IMMUTABLE;
Try
SELECT round(21.04, 0.05); -- 21.05 float!
SELECT round(21.04, 5::float); -- 20
SELECT round(1/3., 0.0001); -- 0.3333
SELECT round(2.8+1/3., 0.5); -- 3.15
SELECT round(pi(), 0.0001); -- 3.1416
PS: the command \df round, on psql after overloadings, will show something like this table
Schema | Name | Result | Argument
------------+-------+---------+------------------
myschema | round | numeric | float, int
myschema | round | float | float, float
pg_catalog | round | float | float
pg_catalog | round | numeric | numeric
pg_catalog | round | numeric | numeric, int
where float is synonymous of double precision and myschema is public when you not use a schema. The pg_catalog functions are the default ones, see at Guide the build-in math functions.
Rounding and formating
The to_char function apply internally the round procedure, so, when your aim is only to show a final result in the terminal, you can use the FM modifier as a prefix to a numeric format pattern:
SELECT round(x::numeric,2), trunc(x::numeric,2), to_char(x, 'FM99.99')
FROM (SELECT 2.0/3) t(x);
round
trunc
to_char
0.67
0.66
.67
NOTES
Cause of the problem
There are a lack of overloads in some PostgreSQL functions, why (???): I think "it is a lack" (!), but #CraigRinger, #Catcall and the PostgreSQL team agree about "pg's historic rationale".
Note about performance and reuse
The build-in functions, such as ROUND of the pg_catalog, can be overloaded with no performance loss, when compared to direct cast encoding. Two precautions must be taken when implementing user-defined cast functions for high performance:
The IMMUTABLE clause is very important for code snippets like this, because, as said in the Guide: "allows the optimizer to pre-evaluate the function when a query calls it with constant arguments"
PLpgSQL is the preferred language, except for "pure SQL". For JIT optimizations (and sometimes for parallelism) language SQL can obtain better optimizations. Is something like copy/paste small piece of code instead of use a function call.
Conclusion: the above ROUND(float,int) function, after optimizations, is so fast than #CraigRinger's answer; it will compile to (exactly) the same internal representation. So, although it is not standard for PostgreSQL, it can be standard for your projects, by a centralized and reusable "library of snippets", like pg_pubLib.
Round to the nth bit or other numeric representation
Some people argue that it doesn't make sense for PostgreSQL to round a number of float datatype, because float is a binary representation, it requires rounding the number of bits or its hexadecimal representation.
Well, let's solve the problem, adding an exotic suggestion... The aim here is to return a float type in another overloaded function, ROUND(float, text, int) RETURNS float The text is to offer a choice between
'dec' for "decimal representation",
'bin' for "binary" representation and
'hex' for hexadecimal representation.
So, in different representations we have a different interpretation about the number of digits to be rounded. Rounding a number x with an approximate shorter value, with less "fractionary digits" (tham its original d digits), will be shorter when d is couting binary digits instead decimal or hexadecimal.
It is not easy without C++, using "pure SQL", but this code snippets will illustrate and can be used as workaround:
-- Looking for a round_bin() function! this is only a workaround:
CREATE FUNCTION trunc_bin(x bigint, t int) RETURNS bigint AS $f$
SELECT ((x::bit(64) >> t) << t)::bigint;
$f$ language SQL IMMUTABLE;
CREATE FUNCTION ROUND(
x float,
xtype text, -- 'bin', 'dec' or 'hex'
xdigits int DEFAULT 0
)
RETURNS FLOAT AS $f$
SELECT CASE
WHEN xtype NOT IN ('dec','bin','hex') THEN 'NaN'::float
WHEN xdigits=0 THEN ROUND(x)
WHEN xtype='dec' THEN ROUND(x::numeric,xdigits)
ELSE (s1 ||'.'|| s2)::float
END
FROM (
SELECT s1,
lpad(
trunc_bin( s2::bigint, CASE WHEN xd<bin_bits THEN bin_bits - xd ELSE 0 END )::text,
l2,
'0'
) AS s2
FROM (
SELECT *,
(floor( log(2,s2::numeric) ) +1)::int AS bin_bits, -- most significant bit position
CASE WHEN xtype='hex' THEN xdigits*4 ELSE xdigits END AS xd
FROM (
SELECT s[1] AS s1, s[2] AS s2, length(s[2]) AS l2
FROM (SELECT regexp_split_to_array(x::text,'\.')) t1a(s)
) t1b
) t1c
) t2
$f$ language SQL IMMUTABLE;
Try
SELECT round(1/3.,'dec',4); -- 0.3333 float!
SELECT round(2.8+1/3.,'dec',1); -- 3.1 float!
SELECT round(2.8+1/3.,'dec'); -- ERROR, need to cast string
SELECT round(2.8+1/3.,'dec'::text); -- 3 float
SELECT round(2.8+1/3.,'dec',0); -- 3 float
SELECT round(2.8+1/3.,'hex',0); -- 3 float (no change)
SELECT round(2.8+1/3.,'hex',1); -- 3.1266
SELECT round(2.8+1/3.,'hex',3); -- 3.13331578486784
SELECT round(2.8+1/3.,'bin',1); -- 3.1125899906842625
SELECT round(2.8+1/3.,'bin',6); -- 3.1301821767286784
SELECT round(2.8+1/3.,'bin',12); -- 3.13331578486784
And \df round have also:
Schema | Name | Result | Argument
------------+-------+---------+---------------
myschema | round | float | x float, xtype text, xdigits int DEFAULT 0
Try with this:
SELECT to_char (2/3::float, 'FM999999990.00');
-- RESULT: 0.67
Or simply:
SELECT round (2/3::DECIMAL, 2)::TEXT
-- RESULT: 0.67
you can use the function below
SELECT TRUNC(14.568,2);
the result will show :
14.56
you can also cast your variable to the desire type :
SELECT TRUNC(YOUR_VAR::numeric,2)
SELECT ROUND(SUM(amount)::numeric, 2) AS total_amount
FROM transactions
Gives: 200234.08
Try casting your column to a numeric like:
SELECT ROUND(cast(some_column as numeric),2) FROM table
According to Bryan's response you can do this to limit decimals in a query. I convert from km/h to m/s and display it in dygraphs but when I did it in dygraphs it looked weird. Looks fine when doing the calculation in the query instead. This is on postgresql 9.5.1.
select date,(wind_speed/3.6)::numeric(7,1) from readings;
Error:function round(double precision, integer) does not exist
Solution: You need to addtype cast then it will work
Ex: round(extract(second from job_end_time_t)::integer,0)
Assuming the following query:
SELECT
ID,
COUNT(1) AS NumRecords,
SUM(Quantity) AS TotalQty
SUM(Quantity)/COUNT(1) AS Avg
FROM SOME_TABLE
GROUP BY ID
Right now it returns:
ID NumRecords TotalQty Avg
1 15 6 2
I want it to return a decimal value with a Scale of 2 for Avg (i.e. "2.5").
I've tried to CAST the calcluation as a DECIMAL, NUMERIC, FLOAT, and VARCHAR, but it always returns an INTEGER.
You need to cast the inputs to the calculation not the result. Also any reason you aren't using the AVG function?
AVG(CAST(Quantity as decimal(10,2)))
An alternative solution is via implicit Casting. I found this to be much cleaner SQL as well. The precision will be determined by the number of trailing zeros used when multiplying by 1.
AVG(Quantity * 1.00) //x.xx
AVG(Quantity * 1.0000) //x.xxxx