Convert a bytea into a binary string - postgresql

I need to decode a base64 string and take a chunk of binary.
Is there a SQL function in Postgres to simply convert a bytea into a binary string representation?
(Like "00010001010101010".)

If your Postgres installation runs with the default setting bytea_output = 'hex', there is a very simple hack:
SELECT right(bytea_col::text, -1)::varbit;
Example:
SELECT right((bytea '\xDEADBEEF')::text, -1)::varbit;
Result:
'11011110101011011011111011101111'
right(text, -1) is just the cheapest way to remove the leading backslash from the text representation.
varbit (standard SQL name bit varying) is for bit strings of arbitrary length. Cast the result to text or varchar if you like.
Related, with explanation:
Convert hex in text representation to decimal number

demo:db<>fiddle
You could put the following code into a function:
WITH byte AS ( -- 1
SELECT E'\\xDEADBEEF'::bytea as value
)
SELECT
string_agg( -- 5
get_byte(value, gs)::bit(8)::text -- 4
, ''
)
FROM
byte,
generate_series( -- 3
0,
length(value) - 1 -- 2
) gs
I demonstrated the development of the query within the fiddle.
The WITH clause encapsulates the bytea value for double usage in further code
length() calculates the binary length of the bytea value
generate_series() creates a list from 0 to length - 1 (0 - 3 in my example)
get_byte() takes the bytea value a second time and gives out the byte at position gs (the previous calculated values 0-3). This gives an integer representation of the the byte. After that the cast to bit(8) type converts the result of this function to its binary representation (1 byte = 8 bit)
string_agg() finally aggregates all for binary strings into one. (taking its text representations instead of bit type, with no delimiters)
A function could look like this:
CREATE OR REPLACE FUNCTION to_bit(value bytea) RETURNS SETOF text AS
$$
BEGIN
RETURN QUERY
SELECT
string_agg(get_byte(value, gs)::bit(8)::text, '')
FROM
generate_series(0, length(value) - 1) gs;
END;
$$ LANGUAGE plpgsql;
After that you could call it:
SELECT to_bit(E'\\xDEADBEEF'::bytea)
You could try it using get_bit() instead of get_byte(). This safes you the ::bit(8) cast but of course you need to multiply the length with factor 8 indeed.
The resulting bit string has another bit order but maybe it fits your use case better:
WITH byte AS (
SELECT E'\\xDEADBEEF'::bytea as value
)
SELECT
string_agg(get_bit(value, gs)::text, '')
FROM
byte,
generate_series(0, length(value) * 8 - 1) gs
demo:db<>fiddle

Related

How to covert substring to integer?

I have varchar data then i want to convert it to integer so i can using the number to order my data this is my varchar
No.SKF.4-04/2021/CBO-ODSP
No.SKF.5-04/2021/CBO-ODSP
No.SKF.6-04/2021/CBO-ODSP
`
i want to take the number so i can select order the data
SELECT varchar from account_information order by CAST(substring(left("NO_SURAT", "length"("NO_SURAT")-17),8)as integer)
but it show some error
SELECT CAST(substring(left("NO_SURAT", "length"("NO_SURAT")-17),8)as integer) from account_information
ERROR: invalid input syntax for type integer: ""
how do i covert substring result to int?
Avoiding an expensive regular expression, you could use:
SELECT CAST(
substr(
'No.SKF.5-04/2021/CBO-ODSP',
8,
position('-' IN 'No.SKF.5-04/2021/CBO-ODSP') - 8
) AS integer
);
substr
════════
5
(1 row)

Convert a decimal number to a letter in PostgreSQL

I have a function that already translates an integer into a letter, I would like to optimize it to translate decimal numbers into letters.
function : https://dbfiddle.uk/?rdbms=postgres_12&fiddle=22a67dedd54291d7ffb527e2f9d482ca
Example : 2053.21
i tried :
select concat(en_function(2053), ' point', en_function(21))
but it is not good approch
i want to put select en_function(5423.21) and return "five thousand four hundred twenty-three point twenty-one"
how can i do that?
the easiest way would be to write a wrapper function and call the original function twice for real and decimal part of the number inside that wrapper function:
CREATE OR REPLACE FUNCTION en_function2(nombre decimal)
RETURNS text
AS $BODY$
select case when substring(nombre::text from '[.]') is null or replace(substring(nombre::text from '[.][0-9]*'),'.','')::int = 0 then en_function(nombre)
else concat(en_function(replace(substring(nombre::text from '[0-9]*[.]'),'.','')::decimal), ' point ', en_function(replace(substring(nombre::text from '[.][0-9]*'),'.','')::decimal))
end
$BODY$ LANGUAGE sql
select en_function2(2053.21);
db<>fiddle here

How to cast bynary numbers to bytea?

select
'1000000011011110101111100001101101100000011100110011001100111111'::bit(64)
as x;
I need to convert to other representations, like hexadecimal, base64 or used it in a function for base36 or base58 conversion... So need to cast to bytea. But there is no x::bytea, what the simplest way to do it?
Use decode():
with data(val) as (
values (
'1000000011011110101111100001101101100000011100110011001100111111'::bit(64))
)
select decode(to_hex(val::bigint), 'hex')
from data;
decode
--------------------
\x80debe1b6073333f
(1 row)
The function returns bytea, you can encode it as base64:
select encode(decode(to_hex(val::bigint), 'hex'), 'base64')
from data;
encode
--------------
gN6+G2BzMz8=
(1 row)
NOTE: for bigger tham bigint binaries, you need to use GMP external extension and the mpz datatypes... And check if the cast to bytea is working fine, or use a direct conversion with GMP.

How to extract timestamp from mongodb objectid in postgres

In MongoDB you can retrieve the date from an ObjectId using the getTimestamp() function. How can I retrieve the date from a MongoDB ObjectId using Postgresql (e.g., in the case where such an ObjectId is stored in a Postgres database)?
Example input:
507c7f79bcf86cd7994f6c0e
Wanted output:
2012-10-15T21:26:17Z
In Mongodb documentation the Objectid is formed with a timestamp as the first 4 bytes, but this is represented in hexidecimal. Assuming that hexidecimal value is stored as a string in PostgreSQL, then the following query will extract just the first 8 characters of that objectid, convert that to an integer (which is seconds from 1970-01-01) then convert that integer to a timestamp. For example:
SELECT TO_TIMESTAMP(int_val) ts_val
FROM (
SELECT ('x' || lpad(left(objectid,8), 8, '0'))::bit(32)::int AS int_val
FROM (
VALUES ('507c7f79bcf86cd7994f6c0e')
) AS t1(objectid)
) AS t2
;
Converting a hexadecimal string to integer is discussed here:
Convert hex in text representation to decimal number
The first answer is quite excellent. This one expands the answer by making a reusable function out of it.
create function extractMongoTimestamp(text) RETURNS TIMESTAMP WITH TIME ZONE
as
'SELECT TO_TIMESTAMP(int_val) ts_val
FROM (
SELECT (''x'' || lpad(left(objectid,8), 8, ''0''))::bit(32)::int AS int_val
FROM (
VALUES ($1)
) AS t1(objectid)
) AS t2'
language sql
immutable
RETURNS null on null input;
Use it in your query:
select extractMongoTimestamp('507c7f79bcf86cd7994f6c0e');

PostgreSql round() giving Error [duplicate]

I am using PostgreSQL via the Ruby gem 'sequel'.
I'm trying to round to two decimal places.
Here's my code:
SELECT ROUND(AVG(some_column),2)
FROM table
I get the following error:
PG::Error: ERROR: function round(double precision, integer) does
not exist (Sequel::DatabaseError)
I get no error when I run the following code:
SELECT ROUND(AVG(some_column))
FROM table
Does anyone know what I am doing wrong?
PostgreSQL does not define round(double precision, integer). For reasons #Mike Sherrill 'Cat Recall' explains in the comments, the version of round that takes a precision is only available for numeric.
regress=> SELECT round( float8 '3.1415927', 2 );
ERROR: function round(double precision, integer) does not exist
regress=> \df *round*
List of functions
Schema | Name | Result data type | Argument data types | Type
------------+--------+------------------+---------------------+--------
pg_catalog | dround | double precision | double precision | normal
pg_catalog | round | double precision | double precision | normal
pg_catalog | round | numeric | numeric | normal
pg_catalog | round | numeric | numeric, integer | normal
(4 rows)
regress=> SELECT round( CAST(float8 '3.1415927' as numeric), 2);
round
-------
3.14
(1 row)
(In the above, note that float8 is just a shorthand alias for double precision. You can see that PostgreSQL is expanding it in the output).
You must cast the value to be rounded to numeric to use the two-argument form of round. Just append ::numeric for the shorthand cast, like round(val::numeric,2).
If you're formatting for display to the user, don't use round. Use to_char (see: data type formatting functions in the manual), which lets you specify a format and gives you a text result that isn't affected by whatever weirdness your client language might do with numeric values. For example:
regress=> SELECT to_char(float8 '3.1415927', 'FM999999999.00');
to_char
---------------
3.14
(1 row)
to_char will round numbers for you as part of formatting. The FM prefix tells to_char that you don't want any padding with leading spaces.
        ((this is a Wiki! please edit to enhance!))
Try also the old syntax for casting,
SELECT ROUND( AVG(some_column)::numeric, 2 ) FROM table;
works with any version of PostgreSQL. ...But, as definitive solution, you can overload the ROUND function.
Overloading as casting strategy
CREATE FUNCTION ROUND(float,int) RETURNS NUMERIC AS $f$
SELECT ROUND( CAST($1 AS numeric), $2 )
$f$ language SQL IMMUTABLE;
Now your instruction will works fine, try this complete comparison:
SELECT trunc(n,3), round(n,3) n_round, round(f,3) f_round,
pg_typeof(n) n_type, pg_typeof(f) f_type, pg_typeof(round(f,3)) f_round_type
FROM (SELECT 2.0/3.0, 2/3::float) t(n,f);
trunc
n_round
f_round
n_type
f_type
f_round_type
0.666
0.667
0.667
numeric
double precision
numeric
The ROUND(float,int) function is f_round, it returns a (decimal) NUMERIC datatype, that is fine for some applications: problem solved!
In another applications we need a float also as result. An alternative is to use round(f,3)::float or to create a round_tofloat() function.
Other alternative, overloading ROUND function again, and using all range of accuracy-precision of a floating point number, is to return a float when the accuracy is defined (see IanKenney's answer),
CREATE FUNCTION ROUND(
input float, -- the input number
accuracy float -- accuracy, the "counting unit"
) RETURNS float AS $f$
SELECT ROUND($1/accuracy)*accuracy
$f$ language SQL IMMUTABLE;
Try
SELECT round(21.04, 0.05); -- 21.05 float!
SELECT round(21.04, 5::float); -- 20
SELECT round(1/3., 0.0001); -- 0.3333
SELECT round(2.8+1/3., 0.5); -- 3.15
SELECT round(pi(), 0.0001); -- 3.1416
PS: the command \df round, on psql after overloadings, will show something like this table
Schema | Name | Result | Argument
------------+-------+---------+------------------
myschema | round | numeric | float, int
myschema | round | float | float, float
pg_catalog | round | float | float
pg_catalog | round | numeric | numeric
pg_catalog | round | numeric | numeric, int
where float is synonymous of double precision and myschema is public when you not use a schema. The pg_catalog functions are the default ones, see at Guide the build-in math functions.
Rounding and formating
The to_char function apply internally the round procedure, so, when your aim is only to show a final result in the terminal, you can use the FM modifier as a prefix to a numeric format pattern:
SELECT round(x::numeric,2), trunc(x::numeric,2), to_char(x, 'FM99.99')
FROM (SELECT 2.0/3) t(x);
round
trunc
to_char
0.67
0.66
.67
NOTES
Cause of the problem
There are a lack of overloads in some PostgreSQL functions, why (???): I think "it is a lack" (!), but #CraigRinger, #Catcall and the PostgreSQL team agree about "pg's historic rationale".
Note about performance and reuse
The build-in functions, such as ROUND of the pg_catalog, can be overloaded with no performance loss, when compared to direct cast encoding. Two precautions must be taken when implementing user-defined cast functions for high performance:
The IMMUTABLE clause is very important for code snippets like this, because, as said in the Guide: "allows the optimizer to pre-evaluate the function when a query calls it with constant arguments"
PLpgSQL is the preferred language, except for "pure SQL". For JIT optimizations (and sometimes for parallelism) language SQL can obtain better optimizations. Is something like copy/paste small piece of code instead of use a function call.
Conclusion: the above ROUND(float,int) function, after optimizations, is so fast than #CraigRinger's answer; it will compile to (exactly) the same internal representation. So, although it is not standard for PostgreSQL, it can be standard for your projects, by a centralized and reusable "library of snippets", like pg_pubLib.
Round to the nth bit or other numeric representation
Some people argue that it doesn't make sense for PostgreSQL to round a number of float datatype, because float is a binary representation, it requires rounding the number of bits or its hexadecimal representation.
Well, let's solve the problem, adding an exotic suggestion... The aim here is to return a float type in another overloaded function,   ROUND(float, text, int) RETURNS float The text is to offer a choice between
'dec' for "decimal representation",
'bin' for "binary" representation and
'hex' for hexadecimal representation.
So, in different representations we have a different interpretation about the number of digits to be rounded. Rounding a number x with an approximate shorter value, with less "fractionary digits" (tham its original d digits), will be shorter when d is couting binary digits instead decimal or hexadecimal.
It is not easy without C++, using "pure SQL", but this code snippets will illustrate and can be used as workaround:
-- Looking for a round_bin() function! this is only a workaround:
CREATE FUNCTION trunc_bin(x bigint, t int) RETURNS bigint AS $f$
SELECT ((x::bit(64) >> t) << t)::bigint;
$f$ language SQL IMMUTABLE;
CREATE FUNCTION ROUND(
x float,
xtype text, -- 'bin', 'dec' or 'hex'
xdigits int DEFAULT 0
)
RETURNS FLOAT AS $f$
SELECT CASE
WHEN xtype NOT IN ('dec','bin','hex') THEN 'NaN'::float
WHEN xdigits=0 THEN ROUND(x)
WHEN xtype='dec' THEN ROUND(x::numeric,xdigits)
ELSE (s1 ||'.'|| s2)::float
END
FROM (
SELECT s1,
lpad(
trunc_bin( s2::bigint, CASE WHEN xd<bin_bits THEN bin_bits - xd ELSE 0 END )::text,
l2,
'0'
) AS s2
FROM (
SELECT *,
(floor( log(2,s2::numeric) ) +1)::int AS bin_bits, -- most significant bit position
CASE WHEN xtype='hex' THEN xdigits*4 ELSE xdigits END AS xd
FROM (
SELECT s[1] AS s1, s[2] AS s2, length(s[2]) AS l2
FROM (SELECT regexp_split_to_array(x::text,'\.')) t1a(s)
) t1b
) t1c
) t2
$f$ language SQL IMMUTABLE;
Try
SELECT round(1/3.,'dec',4); -- 0.3333 float!
SELECT round(2.8+1/3.,'dec',1); -- 3.1 float!
SELECT round(2.8+1/3.,'dec'); -- ERROR, need to cast string
SELECT round(2.8+1/3.,'dec'::text); -- 3 float
SELECT round(2.8+1/3.,'dec',0); -- 3 float
SELECT round(2.8+1/3.,'hex',0); -- 3 float (no change)
SELECT round(2.8+1/3.,'hex',1); -- 3.1266
SELECT round(2.8+1/3.,'hex',3); -- 3.13331578486784
SELECT round(2.8+1/3.,'bin',1); -- 3.1125899906842625
SELECT round(2.8+1/3.,'bin',6); -- 3.1301821767286784
SELECT round(2.8+1/3.,'bin',12); -- 3.13331578486784
And \df round have also:
Schema | Name | Result | Argument
------------+-------+---------+---------------
myschema | round | float | x float, xtype text, xdigits int DEFAULT 0
Try with this:
SELECT to_char (2/3::float, 'FM999999990.00');
-- RESULT: 0.67
Or simply:
SELECT round (2/3::DECIMAL, 2)::TEXT
-- RESULT: 0.67
you can use the function below
SELECT TRUNC(14.568,2);
the result will show :
14.56
you can also cast your variable to the desire type :
SELECT TRUNC(YOUR_VAR::numeric,2)
SELECT ROUND(SUM(amount)::numeric, 2) AS total_amount
FROM transactions
Gives: 200234.08
Try casting your column to a numeric like:
SELECT ROUND(cast(some_column as numeric),2) FROM table
According to Bryan's response you can do this to limit decimals in a query. I convert from km/h to m/s and display it in dygraphs but when I did it in dygraphs it looked weird. Looks fine when doing the calculation in the query instead. This is on postgresql 9.5.1.
select date,(wind_speed/3.6)::numeric(7,1) from readings;
Error:function round(double precision, integer) does not exist
Solution: You need to addtype cast then it will work
Ex: round(extract(second from job_end_time_t)::integer,0)