Understanding difference between int literal vs int parameter in PL/pgSQL function - postgresql

I have a function to left pad bit stings in PostgreSQL 9.5:
CREATE OR REPLACE FUNCTION lpad_bits(val bit varying)
RETURNS bit varying as
$BODY$
BEGIN return val::bit(32) >> (32-length(val));
END;
$BODY$
LANGUAGE plpgsql IMMUTABLE;
which works fine:
# select lpad_bits(b'1001100111000');
lpad_bits
----------------------------------
00000000000000000001001100111000
(1 row)
My problem is when I try to add a parameter to change the amount of padding:
CREATE OR REPLACE FUNCTION lpad_bits(val bit varying, sz integer default 1024)
RETURNS bit varying as
$BODY$
BEGIN return val::bit(sz) >> (sz-length(val));
END;
$BODY$
LANGUAGE plpgsql IMMUTABLE;
The function is now broken:
# select lpad_bits(b'1001100111000', 32);
ERROR: invalid input syntax for integer: "sz"
LINE 1: SELECT val::bit(sz) >> (sz-length(val))
^
QUERY: SELECT val::bit(sz) >> (sz-length(val))
CONTEXT: PL/pgSQL function lpad_bits(bit varying,integer) line 2 at RETURN
I have stared at the bitstring documentation and PL/pgSQL function documentation, am simply not seeing what is fundamentally different between these two implementations.

Why?
PL/pgSQL executes SQL queries like prepared statements. The manual about parameter substituion:
Prepared statements can take parameters: values that are substituted
into the statement when it is executed.
Note the term values. Only actual values can be parameterized, but not key words, identifiers or type names. 32 in bit(32) looks like a value, but the modifier of a data type is only a "value" internally and can't be parameterized. SQL demands to know data types at planning stage, it cannot wait for the execution stage.
You could achieve your goal with dynamic SQL and EXECUTE. As proof of concept:
CREATE OR REPLACE FUNCTION lpad_bits(val varbit, sz int = 32, OUT outval varbit) AS
$func$
BEGIN
EXECUTE format('SELECT $1::bit(%s) >> $2', sz) -- literal
USING val, sz - length(val) -- values
INTO outval;
END
$func$ LANGUAGE plpgsql IMMUTABLE;
Call:
SELECT lpad_bits(b'1001100111000', 32);
Note the distinction between sz being used as literal to build the statement and its second occurrence where it's used as value, that can be passed as parameter.
Faster alternatives
A superior solution for this particular task is to just use lpad() like #Abelisto suggested:
CREATE OR REPLACE FUNCTION lpad_bits2(val varbit, sz int = 32)
RETURNS varbit AS
$func$
SELECT lpad(val::text, sz, '0')::varbit;
$func$ LANGUAGE sql IMMUTABLE;
(Simpler as plain SQL function, which also allows function inlining in the context of outer queries.)
Several times faster than the above function. A minor flaw: we have to cast to text and back to varbit. Unfortunately, lpad() is not currently implemented for varbit. The manual:
The following SQL-standard functions work on bit strings as well as
character strings: length, bit_length, octet_length, position, substring, overlay.
overlay() is available, we can have a cheaper function:
CREATE OR REPLACE FUNCTION lpad_bits3(val varbit, base varbit = '00000000000000000000000000000000')
RETURNS varbit AS
$func$
SELECT overlay(base PLACING val FROM bit_length(base) - bit_length(val))
$func$ LANGUAGE sql IMMUTABLE;
Faster if you can work with varbit values to begin with. (The advantage is (partly) voided, if you have to cast text to varbit anyway.)
Call:
SELECT lpad_bits3(b'1001100111000', '00000000000000000000000000000000');
SELECT lpad_bits3(b'1001100111000', repeat('0', 32)::varbit);
We might overlaod the function with a variant taking an integer to generate base itself:
CREATE OR REPLACE FUNCTION lpad_bits3(val varbit, sz int = 32)
RETURNS varbit AS
$func$
SELECT overlay(repeat('0', sz)::varbit PLACING val FROM sz - bit_length(val))
$func$ LANGUAGE sql IMMUTABLE;
Call:
SELECT lpad_bits3(b'1001100111000', 32;
Related:
Postgresql Convert bit varying to integer
Convert hex in text representation to decimal number

The parser does not allow a variable at that place. The alternative is to use a constant and trim it:
select right((val::bit(128) >> (128 -length(val)))::text, sz)::bit(sz)
from (values (b'1001100111000', 32)) s(val,sz)
;
right
----------------------------------
00000000000000000001001100111000
Or the lpad function as suggested in the comments.

Related

I have the following string '3,45,543,6,89'. Need output like table thorugh function. Pl help me get output through postgresql function

CREATE OR REPLACE FUNCTION public.SplitToTable(
iv_datalist varchar,iv_Separator varchar)
RETURNS TABLE(out_param varchar)
LANGUAGE 'plpgsql'
COST 100
VOLATILE
ROWS 1000
AS $BODY$
BEGIN
RETURN query select (select regexp_split_to_table(iv_datalist, iv_Separator) as out_param );
END
$BODY$;
When I run
select * from splitToTable('3,34,4,545,35,3',',');
I get this error:
ERROR: structure of query does not match function result type
DETAIL: Returned type text does not match expected type character varying in column 1.
CONTEXT: PL/pgSQL function splittotable1(character varying,character varying) line 4 at RETURN QUERY
SQL state: 42804
You'd have to declare the function as RETURNS text rather than RETURNS varchar. Use text everywhere, as that is the preferred string type in PostgreSQL. That way, you have no problems with type conversions.
The function could be written simpler as
CREATE OR REPLACE FUNCTION public.SplitToTable(
iv_datalist text,
iv_Separator text
) RETURNS TABLE(out_param text)
LANGUAGE sql
IMMUTABLE
AS
$BODY$SELECT * FROM regexp_split_to_table(iv_datalist, iv_Separator);$BODY$;
Essentially, the function is useless; you could just use regexp_split_to_table directly.

Use of custom return types in a FOR loop in plpgsql

The following code that I use returns an integer 1:
CREATE TYPE my_test AS (
foo Integer
);
CREATE FUNCTION foo_out()
RETURNS SETOF Integer AS
$$
BEGIN
RETURN QUERY
SELECT 1 as foo;
END
$$
LANGUAGE plpgsql;
CREATE FUNCTION foo1()
RETURNS SETOF my_test
AS $$
DECLARE
x my_test;
BEGIN
FOR x IN
SELECT foo_out()
LOOP
RETURN NEXT x;
END LOOP;
END;
$$
LANGUAGE 'plpgsql';
select * from foo1();
But why does the same code return:
ERROR: invalid input syntax for integer: (1)
if I change the return type to:
CREATE FUNCTION foo_out()
RETURNS SETOF my_test
Which also should be an integer!? It could be the case that the system differs between the type integer and a custom type that includes an integer. What should be changed that the use of custom types is possible here?
my_test is a composite type, a row type, that contains a single integer field. It's not an integer.
When assigning to a record or row type in a FOR loop all output columns from the query form the row. Your SELECT needs to return a single, unnested integer column, which is then nested into a row type matching your custom type my_test.
You can also assign to a list of scalar variables in a FOR loop, in which case columns from the query are assigned left to right to variables as is - not forming a row.
If the column itself is a row type, you have one level of nesting to many. The text representation of a row containing an integer field is '(1)' (with parentheses!), and that's what you see in the error message.
You can fix that by extracting the integer field from the row with attribute notation:
SELECT (foo_out()).*
Or (more efficiently for multiple columns) by decomposing with:
SELECT * FROM foo_out()
Example Code
CREATE FUNCTION foo_out()
RETURNS SETOF my_test
LANGUAGE sql AS
'SELECT ''(1)''::my_test';
CREATE FUNCTION foo1()
RETURNS SETOF my_test
LANGUAGE plpgsql AS
$func$
DECLARE
x my_test;
BEGIN
FOR x IN
SELECT * FROM foo_out()
LOOP
RETURN NEXT x;
END LOOP;
END
$func$;
db<>fiddle here
Old sqlfiddle
Don't quote the language name plpgsql. It's an identifier.
Remember that looping is rarely needed, since most problems are more efficiently solved with a set-based approach (SQL only).
Related answers by Craig and Pavel:
Passing array of a composite type to stored procedure

Defining global constants in Postgresql stored function/procedures?

I have a set of functions I have created in PostgreSql. I would like to be able to configure some behavior and limits with global constants.
So far, I have implemented functions like these, which my functions call to retrieve the constant value:
CREATE OR REPLACE FUNCTION slice_length()
RETURNS integer AS $$
BEGIN
RETURN 50;
END; $$
LANGUAGE plpgsql IMMUTABLE;
I was wondering, is there a better/smarter way to achieve this?
I had a similar problem: store some sort of configuration and access it from my function.
To solve the problem I created a function which returns a constant JSONB containing my configuration.
The function looks like this:
create or replace function config()
returns jsonb
language plpgsql
immutable
as $BODY$
declare
job_manager_config constant jsonb := '{
"notify": {
"channels": {
"all_available": "all.available",
"all_status": "all.status",
"task_available": "task.%s.available",
"task_status": "task.%s.status",
"job": "job.%s"
},
}
"allowed_pets": {
"dogs": 6,
"cats": 6,
"rabbits": 3,
"lions": 2,
"sweet_piranha": 8
}
}';
begin
return job_manager_config;
end;
$BODY$;
To access configuration elements quickly I defined a second function to query the configuration. This function accepts a path as a list of strings and return the value (or JSON object) found at the path.
Note that the returned value is text but you can easily cast it to the actual type.
create or replace function job_manager.config(
variadic path_array text[])
returns text
language plpgsql
immutable
as $BODY$
begin
-- Return selected object as text (instead of json or jsonb)
return jsonb_extract_path_text(job_manager.config(), variadic path_array);
end;
$BODY$;
This is a usage example:
test=# select job_manager.config('notify', 'channels', 'job');
config
--------------------
job_manager.job.%s
(1 row)
I would create a table for that:
create table constant (c1 int, c2 numeric);
insert into constant (c1, c2) values (100, 33.2);
The function, SQL not PL/pgSQL, would retrieve the single row:
create or replace function get_constants()
returns constant as $$
select *
from constant;
$$ language sql immutable;
And would be called for each constant:
select (get_constants()).c1, (get_constants()).c2;
All data would be in a single place and retrieved with a single function.
If a table is really that bad then place all the values in a single function:
create or replace function get_constants (
c1 out int, c2 out numeric
) returns record as $$
select 100, 33.5;
$$ language sql immutable;
And use it as above.
Take a look at this other answer. It uses the same approach that you do.
Create constant string for entire database
You can declare a table named constants for that purpose as mentioned that answer where each column corresponds to one setting.
To ensure that no more than one row can be added to the table, this answer might be helpful.
As come comments have pointed to the increased I/O when constants are stored like this as opposed to storing constants as functions, this might be a good reeding.

Optional argument in PL/pgSQL function

I am trying to write a PL/pgSQL function with optional arguments. It performs a query based on a filtered set of records (if specified), otherwise performs a query on the entire data set in a table.
For example (PSEUDO CODE):
CREATE OR REPLACE FUNCTION foofunc(param1 integer, param2 date, param2 date, optional_list_of_ids=[]) RETURNS SETOF RECORD AS $$
IF len(optional_list_of_ids) > 0 THEN
RETURN QUERY (SELECT * from foobar where f1=param1 AND f2=param2 AND id in optional_list_of_ids);
ELSE
RETURN QUERY (SELECT * from foobar where f1=param1 AND f2=param2);
ENDIF
$$ LANGUAGE SQL;
What would be the correct way to implement this function?
As an aside, I would like to know how I could call such a function in another outer function. This is how I would do it - is it correct, or is there a better way?
CREATE FUNCTION foofuncwrapper(param1 integer, param2 date, param2 date) RETURNS SETOF RECORD AS $$
BEGIN
CREATE TABLE ids AS SELECT id from foobar where id < 100;
RETURN QUERY (SELECT * FROM foofunc(param1, param2, ids));
END
$$ LANGUAGE SQL
Since PostgreSQL 8.4 (which you seem to be running), there are default values for function parameters. If you put your parameter last and provide a default, you can simply omit it from the call:
CREATE OR REPLACE FUNCTION foofunc(_param1 integer
, _param2 date
, _ids int[] DEFAULT '{}')
RETURNS SETOF foobar -- declare return type!
LANGUAGE plpgsql AS
$func$
BEGIN -- required for plpgsql
IF _ids <> '{}'::int[] THEN -- exclude empty array and NULL
RETURN QUERY
SELECT *
FROM foobar
WHERE f1 = _param1
AND f2 = _param2
AND id = ANY(_ids); -- "IN" is not proper syntax for arrays
ELSE
RETURN QUERY
SELECT *
FROM foobar
WHERE f1 = _param1
AND f2 = _param2;
END IF;
END -- required for plpgsql
$func$;
Major points:
The keyword DEFAULT is used to declare parameter defaults. Short alternative: =.
I removed the redundant param1 from the messy example.
Since you return SELECT * FROM foobar, declare the return type as RETURNS SETOF foobar instead of RETURNS SETOF record. The latter form with anonymous records is very unwieldy, you'd have to provide a column definition list with every call.
I use an array of integer (int[]) as function parameter. Adapted the IF expression and the WHERE clause accordingly.
IF statements are not available in plain SQL. Has to be LANGUAGE plpgsql for that.
Call with or without _ids:
SELECT * FROM foofunc(1, '2012-1-1'::date);
Effectively the same:
SELECT * FROM foofunc(1, '2012-1-1'::date, '{}'::int[]);
You have to make sure the call is unambiguous. If you have another function of the same name and two parameters, Postgres might not know which to pick. Explicit casting (like I demonstrate) narrows it down. Else, untyped string literals work, too, but being explicit never hurts.
Call from within another function:
CREATE FUNCTION foofuncwrapper(_param1 integer, _param2 date)
RETURNS SETOF foobar
LANGUAGE plgpsql AS
$func$
DECLARE
_ids int[] := '{1,2,3}';
BEGIN
-- whatever
RETURN QUERY
SELECT * FROM foofunc(_param1, _param2, _ids);
END
$func$;
Elaborating on Frank's answer on this thread:
The VARIADIC agument doesn't have to be the only argument, only the last one.
You can use VARIADIC for functions that may take zero variadic arguments, it's just a little fiddlier in that it requires a different calling style for zero args. You can provide a wrapper function to hide the ugliness. Given an initial varardic function definition like:
CREATE OR REPLACE FUNCTION foofunc(param1 integer, param2 date, param2 date, optional_list_of_ids VARIADIC integer[]) RETURNS SETOF RECORD AS $$
....
$$ language sql;
For zero args use a wrapper like:
CREATE OR REPLACE FUNCTION foofunc(integer, date, date) RETURNS SETOF RECORD AS $body$
SELECT foofunc($1,$2,$3,VARIADIC ARRAY[]::integer[]);
$body$ LANGUAGE 'sql';
or just call the main func with an empty array like VARIADIC '{}'::integer[] directly. The wrapper is ugly, but it's contained ugliness, so I'd recommend using a wrapper.
Direct calls can be made in variadic form:
SELECT foofunc(1,'2011-01-01','2011-01-01', 1, 2, 3, 4);
... or array call form with array ctor:
SELECT foofunc(1,'2011-01-01','2011-01-01', VARIADIC ARRAY[1,2,3,4]);
... or array text literal form:
SELECT foofunc(1,'2011-01-01','2011-01-01', VARIADIC '{1,2,3,4}'::int[]);
The latter two forms work with empty arrays.
You mean SQL Functions with Variable Numbers of Arguments? If so, use VARIADIC.

Declare variable of composite type in PostgreSQL using %TYPE

Question: How can I declare a variable of the same type a parameter in a stored function?
The simple answer is use %TYPE, this works:
CREATE OR REPLACE FUNCTION test_function_1(param1 text)
RETURNS integer AS
$BODY$
DECLARE
myVariable param1%TYPE;
BEGIN
return 1;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
But the problem is when param1 is a composite type:
CREATE TYPE comp_type as
(
field1 text
)
CREATE OR REPLACE FUNCTION test_function_2(param1 comp_type)
RETURNS integer AS
$BODY$
DECLARE
myVariable param1%TYPE;
BEGIN
return 1;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
This doesn't work:
ERROR: type comp_type does not exist [SQL State=42704]
So how can I do when param1 is a composite type?
(Note: Just myVariable comp_type is not a good option because my function is slightly more complex.)
Edited:
I had a mistake on copy&paste, the real error is:
ERROR: invalid type name "param1%TYPE"
Position: 130 [SQL State=42601]
And using param1%ROWTYPE the error is:
ERROR: relation "param1" does not exist
Where: compilation of PL/pgSQL function "test_function_2" near line 3 [SQL State=42P01]
Use %ROWTYPE in that case.
Simple case
Tests by A.H. and DavidEG have shown this won't work. Interesting problem!
You could try a workaround. As long as your definition is like the example you can simply resort to
CREATE FUNCTION test(param1 comp_type)
RETURNS integer
LANGUAGE plpgsql VOLATILE AS
$func$
DECLARE
myvar comp_type;
BEGIN
RETURN 1;
END
$func$;
But your real problem is probably not as simple?
The real problem
As expected, the real problem turns out to be more complex: a polymorphic input type.
Workaround for that scenario was harder, but should work flawlessly:
CREATE FUNCTION test(param1 anyelement, OUT a integer, OUT myvar anyelement)
RETURNS record
LANGUAGE plpgsql VOLATILE AS
$func$
BEGIN
myvar := $1; -- myvar has the required type now.
-- do stuff with myvar.
myvar := NULL; -- reset if you don't want to output ..
a := 1;
END;
$func$;
Call:
SELECT a FROM test('("foo")'::comp_type); -- just retrieve a, ignore myvar
See full output:
SELECT * FROM test('("foo")'::comp_type);
Note for Postgres 9.0+
There has been a crucial update in Postgres 9.0. The release notes:
Allow input parameters to be assigned values within PL/pgSQL functions
(Steve Prentice)
Formerly, input parameters were treated as being declared CONST, so
the function's code could not change their values. This restriction
has been removed to simplify porting of functions from other DBMSes
that do not impose the equivalent restriction. An input parameter now
acts like a local variable initialized to the passed-in value.
In addition to my workaround, you can (ab)use input variables directly now.
Dynamic field names
See:
How to clone a RECORD in PostgreSQL
How to set value of composite variable field using dynamic SQL