I have a PostgreSQL table that I want to alter a column from bigint to bytea byte to hold more data. I am thinking using the following sequence:
alter table mytable add new_column
update mytable set new_column = int8send(old_column)
alter table drop old_column
alter table rename new_column to old_column
The above sequence works, the only problem is that I want the byte sequence in the bytea to be reversed. For example, if a value in old_column is 0x1234567890abcdef, the above sequence would generate \0224Vx\220\253\315\357, but I want it to be
\357\315\253\220xV4\022. Seems like the resulting bytea uses the big-endian order from originating bigint.
Is there an easy way to do that without writing a program? I was looking for a swap64() sort of function in PostgreSQL but failed to find one.
Here's a pure-SQL function I wrote to reverse the byte-order of a bytea-type value:
CREATE OR REPLACE FUNCTION reverse_bytes_iter(bytes bytea, length int, midpoint int, index int)
RETURNS bytea AS
$$
SELECT CASE WHEN index >= midpoint THEN bytes ELSE
reverse_bytes_iter(
set_byte(
set_byte(bytes, index, get_byte(bytes, length-index)),
length-index, get_byte(bytes, index)
),
length, midpoint, index + 1
)
END;
$$ LANGUAGE SQL IMMUTABLE;
CREATE OR REPLACE FUNCTION reverse_bytes(bytes bytea) RETURNS bytea AS
'SELECT reverse_bytes_iter(bytes, octet_length(bytes)-1, octet_length(bytes)/2, 0)'
LANGUAGE SQL IMMUTABLE;
I just wrote it yesterday, so it's not particularly well-tested nor optimized, but it seems to work, at least on byte strings up to 1k in length.
It is possible to byte-swap without plpgsql code using regexp extractions on the hexadecimal representation.
Here's an example to swap a bigint constant, assuming SET standard_conforming_strings to ON (the default with PG 9.1)
select regexp_replace( lpad(to_hex(x'123456789abcd'::bigint),16,'0'),
'(\w\w)(\w\w)(\w\w)(\w\w)(\w\w)(\w\w)(\w\w)(\w\w)',
'\8\7\6\5\4\3\2\1');
It returns cdab896745230100. Then apply decode(value, 'hex') to convert that to a bytea.
The whole type conversion could actually be done in a single SQL statement:
ALTER TABLE mytable ALTER COLUMN old_column TYPE bytea
USING decode(
regexp_replace( lpad(to_hex(old_column), 16,'0'),
'(\w\w)(\w\w)(\w\w)(\w\w)(\w\w)(\w\w)(\w\w)(\w\w)',
'\8\7\6\5\4\3\2\1')
, 'hex');
I am playing with pageinspect module now, and I was also curios how to change byte order of an existing bytea value, which pretty much matches your case.
I've come up with the following function:
CREATE OR REPLACE FUNCTION reverse(bytea) RETURNS bytea AS $reverse$
SELECT string_agg(byte,''::bytea)
FROM (
SELECT substr($1,i,1) byte
FROM generate_series(length($1),1,-1) i) s
$reverse$ LANGUAGE sql;
It's pretty straightforward and works similar to textual reverse() function:
WITH v(val) AS (
VALUES ('\xaabbccdd'::bytea),('\x0123456789abcd'::bytea)
)
SELECT val, reverse(val)
FROM v;
This function, while not exactly what you're looking for, should help you get on your way.
The source code from that function is reproduced verbatim below.
CREATE OR REPLACE FUNCTION utils.int_littleendian(v_number integer)
RETURNS bytea AS
$BODY$
DECLARE
v_textresult bytea;
v_temp int;
v_int int;
v_i int = 0;
BEGIN
v_int = v_number;
v_textresult = '1234';
WHILE(v_i < 4) LOOP
raise notice 'loop %',v_int;
v_temp := v_int%256;
v_int := v_int - v_temp;
v_int := v_int / 256;
SELECT set_byte(v_textresult,v_i,v_temp) INTO v_textresult;
v_i := v_i + 1;
END LOOP;
return v_textresult;
END;
$BODY$
LANGUAGE 'plpgsql' VOLATILE
COST 100;
Related
I have a table with multiple columns in PostgreSQL. I try to make a function returning a table with a few default columns and a variable column. The column name should be passed as function parameter. Example:
SELECT * FROM get_gas('temperature');
This is my code right now:
CREATE OR REPLACE FUNCTION get_gas(gas text)
RETURNS TABLE (id INTEGER, node_id INTEGER,
gas_name DOUBLE PRECISION,
measurement_timestamp timestamp without time zone )
AS
$$
BEGIN
SELECT measurements_lora.id, measurements_lora.node_id, gas, measurements_lora.measurement_timestamp
AS measure
FROM public.measurements_lora;
END
$$ LANGUAGE plpgsql;
When passing, for example, 'temperature' as column name (gas), I want to get a table with these columns from the function call.
id - node_id - temperature - measurement_timestamp
How would I achieve this?
You can use EXECUTE statement.
CREATE OR REPLACE FUNCTION get_gas(gas text) RETURNS TABLE (f1 INTEGER, f2 INTEGER, f3 DOUBLE PRECISION, f4 timestamp without time zone ) AS
$$
DECLARE
sql_to_execute TEXT;
BEGIN
SELECT 'SELECT measurements_lora.id,
measurements_lora.node_id, '
|| gas ||',
measurements_lora.measurement_timestamp AS measure
FROM public.measurements_lora '
INTO sql_to_execute;
RETURN QUERY EXECUTE sql_to_execute;
END
$$ LANGUAGE plpgsql;
This will create a variable sql_to_execute with your field and. QUERY EXECUTE will execute your interpreted query.
EDIT 1: Look at the another answer the concernings about security issues.
If you really need dynamic SQL in a PL/pgSQL function (which you don't), be sure to defend against SQL injection! Like:
CREATE OR REPLACE FUNCTION get_gas(gas text)
RETURNS TABLE (id integer
, node_id integer
, gas_name double precision
, measurement_timestamp timestamp)
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE format(
'SELECT m.id, m.node_id, m.%I, m.measurement_timestamp
FROM public.measurements_lora m'
, gas
);
END
$func$;
The format specifier %I in format() double-quotes identifiers where needed,
See:
SQL injection in Postgres functions vs prepared queries
Insert text with single quotes in PostgreSQL
I am trying to convert SQL Server stored function to PostgreSQL stored function I am getting one syntactical error at declare #table1 table
CREATE OR REPLACE FUNCTION ETL_GetBuildingDetailsByUserID ( p_nInstID numeric=0)
RETURNS Boolean
AS $$
declare #table1 table
(
nbuilding numeric(18, 0) NOT NULL,
sbuild_name varchar(50) NULL,
sclient_build_id varchar(50) NULL,
nbuilding_inst_id numeric(18, 0) NOT NULL,
ntemp_building_id numeric(18,0) NULL,
nno_of_floors numeric(18,0) NULL
)
declare v_strDeptIds text;
v_strSubDeptIds text;
BEGIN
v_strsql := 'SELECT building.*
FROM building
WHERE (building.nbuilding_inst_id = '|| cast(p_nInstID as varchar(1)) ||')
';
print v_strsql
v_strsql1 text;
v_strsql1 := v_strsql
Insert into #table1; execute sp_executesql; v_strsql1
select * from #table1;
Return true;
END;
$$ LANGUAGE plpgsql;
Error
ERROR: syntax error at or near "#"
LINE 4: declare #table1 table
Can any one please tell what I am doing wrong?
It seems your function actually returns the result of a SELECT query, not a boolean value, so returns boolean is wrong to begin with.
To return a result, you need to declare the function as returns table(). But as you seem to simply return rows from the building table you can define it as returns setof building.
Then remove the useless dynamic SQL which seems completely unnecessary.
In PL/pgSQL there are no table variables, and copying the result of a query into one before returning that result from that table seems to be an unnecessary step which only slows down things. In Postgres you simply return the result of the query, there is no need to store it locally.
Additionally: rather than casting a parameter to another type inside the function it's better to declare that parameter with the type you expect.
So the simplified version of that function in PostgreSQL would be:
CREATE OR REPLACE FUNCTION ETL_GetBuildingDetailsByUserID ( p_nInstID text)
RETURNS setof building
AS $$
select building.*
from building
WHERE building.nbuilding_inst_id = p_nInstID
$$ LANGUAGE sql;
You can use it like this:
select *
from ETL_GetBuildingDetailsByUserID ('42');
Unrelated, but: using numeric(18,0) for columns that store values without decimals is overkill. You should define those columns as bigint. Much faster and uses less space than numeric.
Can anyone help me to implement the userdefined types in postgres?
how can i replicate the same functionality of below oracle types in postgres:
CREATE TYPE gr_data AS (
col1 numeric, col2 varchar(30)
);
CREATE OR REPLACE TYPE GT_DATA IS TABLE OF GR_DATA;
CREATE OR REPLACE TYPE GT_TEXT IS TABLE OF VARCHAR2(255) INDEX BY BINARY_INTEGER ;
CREATE OR REPLACE TYPE GN_CPT IS TABLE OF NUMBER INDEX BY VARCHAR2(30);
Postgres code is as below with type GT_CPT(NomTable) and GT_DATA().
can anyone explain why GT_DATA() type is used as a function here?
create or replace function test() returns void as $body$
declare
NomTable text:= 'tab1';
tes text;
--Result text;-- not required
lt_Result GT_DATA := GT_DATA();
begin
GT_CPT(NomTable) := tes;
--Result GT_DATA := GT_DATA(); -- modified
lt_Result.EXTEND;
lt_Result(lt_Result.LAST) := lr_Result;
end $body$
;
is table of is usually done using arrays in Postgres.
The equivalent to INDEX BY VARCHAR2(30) is probably a JSONB value (key/value pair) or a hstore value. In both cases, you lose the type safety that is table of number index by varchar would offer.
It is very unclear to me what you are trying to achieve (especially how you want to use the table of number, but maybe the following will get you started:
create type gr_data as
(
col1 numeric, col2 varchar(30)
);
create or replace function test()
returns void as $body$
declare
NomTable text:= 'tab1';
tes text := 'some value';
result gr_data[];
gt_cpt jsonb := '{}';
begin
-- this assigns 'some value' to the key 'tab1' in the jsonb key/value pair
-- similar to GT_CPT(NomTable) := tes;
gt_cpt := jsonb_set(gt_cpt, array[nomtable], tes);
-- this assigns an empty array to the variable result
-- if I remember correctly that's the same as Oracle's type constructor GT_DATA()
result := array[];
end
$body$
language plpgsql;
I have a function to left pad bit stings in PostgreSQL 9.5:
CREATE OR REPLACE FUNCTION lpad_bits(val bit varying)
RETURNS bit varying as
$BODY$
BEGIN return val::bit(32) >> (32-length(val));
END;
$BODY$
LANGUAGE plpgsql IMMUTABLE;
which works fine:
# select lpad_bits(b'1001100111000');
lpad_bits
----------------------------------
00000000000000000001001100111000
(1 row)
My problem is when I try to add a parameter to change the amount of padding:
CREATE OR REPLACE FUNCTION lpad_bits(val bit varying, sz integer default 1024)
RETURNS bit varying as
$BODY$
BEGIN return val::bit(sz) >> (sz-length(val));
END;
$BODY$
LANGUAGE plpgsql IMMUTABLE;
The function is now broken:
# select lpad_bits(b'1001100111000', 32);
ERROR: invalid input syntax for integer: "sz"
LINE 1: SELECT val::bit(sz) >> (sz-length(val))
^
QUERY: SELECT val::bit(sz) >> (sz-length(val))
CONTEXT: PL/pgSQL function lpad_bits(bit varying,integer) line 2 at RETURN
I have stared at the bitstring documentation and PL/pgSQL function documentation, am simply not seeing what is fundamentally different between these two implementations.
Why?
PL/pgSQL executes SQL queries like prepared statements. The manual about parameter substituion:
Prepared statements can take parameters: values that are substituted
into the statement when it is executed.
Note the term values. Only actual values can be parameterized, but not key words, identifiers or type names. 32 in bit(32) looks like a value, but the modifier of a data type is only a "value" internally and can't be parameterized. SQL demands to know data types at planning stage, it cannot wait for the execution stage.
You could achieve your goal with dynamic SQL and EXECUTE. As proof of concept:
CREATE OR REPLACE FUNCTION lpad_bits(val varbit, sz int = 32, OUT outval varbit) AS
$func$
BEGIN
EXECUTE format('SELECT $1::bit(%s) >> $2', sz) -- literal
USING val, sz - length(val) -- values
INTO outval;
END
$func$ LANGUAGE plpgsql IMMUTABLE;
Call:
SELECT lpad_bits(b'1001100111000', 32);
Note the distinction between sz being used as literal to build the statement and its second occurrence where it's used as value, that can be passed as parameter.
Faster alternatives
A superior solution for this particular task is to just use lpad() like #Abelisto suggested:
CREATE OR REPLACE FUNCTION lpad_bits2(val varbit, sz int = 32)
RETURNS varbit AS
$func$
SELECT lpad(val::text, sz, '0')::varbit;
$func$ LANGUAGE sql IMMUTABLE;
(Simpler as plain SQL function, which also allows function inlining in the context of outer queries.)
Several times faster than the above function. A minor flaw: we have to cast to text and back to varbit. Unfortunately, lpad() is not currently implemented for varbit. The manual:
The following SQL-standard functions work on bit strings as well as
character strings: length, bit_length, octet_length, position, substring, overlay.
overlay() is available, we can have a cheaper function:
CREATE OR REPLACE FUNCTION lpad_bits3(val varbit, base varbit = '00000000000000000000000000000000')
RETURNS varbit AS
$func$
SELECT overlay(base PLACING val FROM bit_length(base) - bit_length(val))
$func$ LANGUAGE sql IMMUTABLE;
Faster if you can work with varbit values to begin with. (The advantage is (partly) voided, if you have to cast text to varbit anyway.)
Call:
SELECT lpad_bits3(b'1001100111000', '00000000000000000000000000000000');
SELECT lpad_bits3(b'1001100111000', repeat('0', 32)::varbit);
We might overlaod the function with a variant taking an integer to generate base itself:
CREATE OR REPLACE FUNCTION lpad_bits3(val varbit, sz int = 32)
RETURNS varbit AS
$func$
SELECT overlay(repeat('0', sz)::varbit PLACING val FROM sz - bit_length(val))
$func$ LANGUAGE sql IMMUTABLE;
Call:
SELECT lpad_bits3(b'1001100111000', 32;
Related:
Postgresql Convert bit varying to integer
Convert hex in text representation to decimal number
The parser does not allow a variable at that place. The alternative is to use a constant and trim it:
select right((val::bit(128) >> (128 -length(val)))::text, sz)::bit(sz)
from (values (b'1001100111000', 32)) s(val,sz)
;
right
----------------------------------
00000000000000000001001100111000
Or the lpad function as suggested in the comments.
I've got a postgresql stored procedure, which is returning an integer.
When I call that function, the result is returned with the function name as column name.
For example the name of the function is: "add-person". The column name, when invoking the function, is "add-person".
Is there a way to make the database return the integer with a self-choosen column name? For example "id"?
I think it is pretty easy, but I currently miss the forests for the trees..
Edit:
What i'd missed to tell, is that the return value is a variable, like so:
CREATE OR REPLACE FUNCTION "scheme"."add-person"(arggivenname character varying, argfamilyname character varying) RETURNS integer AS
$BODY$
DECLARE
varResponse integer;
BEGIN
-- Operations before
INSERT INTO "scheme"."table"
(
given_name,
family_name
)
VALUES
(
arggivenname,
argfamilyname
)
RETURNING
"id"
INTO
varResponse;
-- Operations after
RETURN varResponse;
END;
$BODY$
LANGUAGE plpgsql VOLATILE COST 100;
You can us the AS statement for that. That means:
Select add-person() AS yourcolumnname
To have a named column from a function it is necessary to create a type and return that type from the function
create type mytype as (mycolumn integer);
create or replace function ri()
returns mytype as $$
select 1;
$$ language sql;
select * from ri();
mycolumn
----------
1
Edit
Or much simpler without the type creation as in #pozs comment:
create or replace function ri(out mycolumn integer)
as $$
select 1;
$$ language sql;