This is a typical wrap function (overloading a non-json foo() function) for use in JSONB context,
-- suppose FUNCTION foo(text[], numeric, boolean) returns int.
CREATE FUNCTION foo(JSONB) RETURNS int AS $f$
SELECT foo(
(SELECT array_agg(x) FROM jsonb_array_elements_text($1->'list') t(x) ),
($1->>'lim')::numeric, -- why cast JSONB number to numeric?
($1->>'osort')::boolean -- idem for boolean
);
$f$ LANGUAGE SQL IMMUTABLE;
Is there an elegant way to cast a JSONB string array into SQL text array? Why we need this ugly numeric and boolean casts if the internal representation says its datatypes?
EDITED after good #IgorRomanchenko discussion:
PHILOSOPHIC NOTES
My preocupation is
Why PostreSQL 9.5 is losing JSONB internal datatype information and internal (number and boolean) representations?
In nowadays the runtime engine use CPU to convert JSONB number to text and text to SQL numeric, or boolean to text them text to boolean... Ideal (optimized) is "to by-pass" JSONB internal representations, or only casts "binary to binary" (not parse to text and cast text to binary).
PostgreSQL will be more optimized and friendly when internally using JSONB datatype (jsonb_typeof)!
PS1: the SELECT array_agg(x) FROM jsonb_array_elements_text() example only amplifies and exposes the same problem. In this case we also need a direct jsonb_array_to_array(x,type) function to by-pass array bynary representation.
PS2: CREATE CAST is not a solution to this problem, because destroys the original JSONB representations.
... How to do it?
Hum... Perhaps creating alternative casting types like ::numericByPass to "bypass JSONB binary number to SQL numeric" and ::booleanByPass "bypass JSONB boolean to SQL boolean", so, preserving internal representation, not parsing it as a text value. Of course, if runtime check see string instead expected datatype, it triggers an error.
Looking at the example we see that the context obligates to use number and boolean, the user not need to say it to the compiler.
The SQL parser can use this implicit "context obligation" to do by it self, in compile time, the cast to ::numericByPass and ::booleanByPass, preserving JSONB internal representations. If the user is really afraid about string inputs, only in this case the user will add explicit ::numeric and ::boolean castings.
You need runtime casts and checks because an input JSON does not define any structure. You can not check that this input JSON has field "list" with text array in it or a field "lim" with an integer.
Take this example JSON:
{
"list":42,
"lim": [3,5],
"osort": "maybe"
}
Current function will throw type cast exceptions because it has runtime casts and checks. What will this function do without any runtime checks?
BTW If you really want postgres to do this for you - you can enable implicit cast from varchar to int and boolean as in this post: Postgresql. CREATE CAST 'character varying' to 'integer'
It is not the best idea, but it will work.
You can create different operators similar to ->> that produce different output types. For example, -># could give you a number and ->| a boolean.
CREATE FUNCTION json_numeric(
j json
, e text
) RETURNS NUMERIC IMMUTABLE STRICT LANGUAGE SQL AS $$
SELECT (j->>e)::numeric
$$;
CREATE OPERATOR -># ( PROCEDURE = json_numeric, LEFTARG = json, RIGHTARG = text );
SELECT ('{"a": 42, "b": "1"}')::json -># 'a';
?column?
----------
42
(1 row)
SELECT ('{"a": 42, "b": "1"}')::json -># 'b';
?column?
----------
1
(1 row)
If you used plpgsql you could throw an error if json_typeof() wasn't correct, but note that will be significantly slower than the SQL version (though in most usage that won't matter).
Related
The following code is not a full setup that you can run to check. It shall just make it a bit clearer what the question is about.
With an example function like this (the variadic example is taken from PostgreSQL inserting list of objects into a stored procedure or PostgreSQL - Passing Array to Stored Function):
CREATE OR REPLACE function get_v(variadic _v text[]) returns table (v varchar(50)) as
$F$
declare
begin
return query
select t.v
from test t
where t.v = any(_v)
end;
$F$
language plpgsql
;
If you copy the one-value output of a select string_agg... query, 'x','y','z', by hand and put it as the argument of the function, the function works:
SELECT v FROM get_v_from_v(
'x','y','z'
);
The 'x','y','z' gets read into the function as variadic _v text[] so that the function can check its values with where t.v = any(_v).
If you instead put the (select string_agg...) query that is behind that 'x','y','z' output in the same place, the function does not work:
select v from get_v_from_v(
(select string_agg(quote_literal(x.v), ',') from (select v from get_v_from_name('something')) as x)
);
That means: the "one-value output field" 'x','y','z' that comes from the (select string_agg...) query is not the same as the text[] list type: 'x','y','z'.
With get_v_from_name('something') as another function that returns a table of one column and the "v" values in the rows, and after running the string_agg() on its output, you get the 'x','y','z' output. I learnt this working function string_agg() at How to make a list of quoted strings from the string values of a column in postgresql?. The full list of such string functions is in the postgreSQL guide at 9.4. String Functions and Operators.
I guess that the format of the select query output is just a string, not a list, so that the input is not seen as a list of quoted strings, but rather like a string as a whole: ''x','y','z''. The get_v_from_v argument does not need just one string of all values, but a list of quoted strings, since the argument for the function is of type text[] - which is a list.
It seems as if this question does not depend on the query that is behind the output. It seems rather just a general thing that the output in a tuple of a table and taken as the argument of a function is not the same as the same output hand-copied as the same argument.
Therefore, the question. What needs to be done to make the output of a select query the same as the hand-copy of its output, so that the output is just the list 'x','y','z', as if it was just copied and pasted?
PS: I guess that this way of making lists of quoted strings from the one-column table output only to pass it to the function is not best practice. For example, in TSQL/SQL Server, you should pass "table valued parameters", so that you pass values as a table that you select from within the function to get the values, see How do I pass a list as a parameter in a stored procedure?. Not sure how this is done in postgreSQL, but it might be what is needed here.
CREATE OR REPLACE function get_v(_v text[]) returns table (v varchar(50)) as
$F$
declare
begin
return query
select t.v
from test t
where t.v = any((select * from unnest(_v)))
end;
$F$
language plpgsql
;
With get_v_from_name('something') as another function that returns a table of one column and the "v" values in the rows (this was said in the question), the following works:
select v from get_v_from_v(
(select array_agg(x.v) from (select v from get_v_from_name('something')) as x)
);
Side remark:
array_agg(quote_literal(x.v), ',') instead of array_agg(x.v) fails, the function does not allow a second argument.
I have query like
_search_text := 'ind'
select * from table where (_search_text ILIKE ANY (addresses))
The st.addresses have value like [india,us,uk,pak,bang]
It should return each item rows where any item of column addresses contains the string _search_text,
Currently it returns only if give full india in _search_text.
What should I make the change
I was also try to thinkin to use unnest, but since it wil be a sub clause of a very long where cluase... so avoid that.
Thanks
I afraid so it is not possible - LIKE, ILIKE supports a array only on right side, and there is searching pattern. Not string. You can write own SQL function:
CREATE OR REPLACE FUNCTION public.array_like(text[], text)
RETURNS boolean
LANGUAGE sql
IMMUTABLE STRICT
AS $function$
select bool_or(v like $2) from unnest($1) v
$function$
and the usage can looks like:
WHERE array_like(addresses, _search_text)
So the readability of this query can be well. But I afraid about performance. There cannot be used any index on this expression. It is result of little bit bad design (the data are not normalized).
ilike ignores cases (difference in upper or lower cases), it doesn't search for string containing your value.
In your case you can use:
_search_text := '%ind%'
select * from table where (_search_text ILIKE ANY (addresses))
ANY will not work here because the arguments are in the wrong order for ILIKE.
You can define a custom operator to make this work. But this will most likely suffer from poor performance:
create table addresses(id integer primary key, states varchar[]);
insert into addresses values (1,'{"belgium","france","united states"}'),
(2,'{"belgium","ireland","canada"}');
CREATE OR REPLACE FUNCTION pg_catalog."rlike"(
leftarg text,
rightarg text)
RETURNS boolean
LANGUAGE 'sql'
COST 100
IMMUTABLE STRICT PARALLEL SAFE SUPPORT pg_catalog.textlike_support
AS $BODY$
SELECT pg_catalog."like"(rightarg,leftarg);
$BODY$;
ALTER FUNCTION pg_catalog."rlike"(text, text)
OWNER TO postgres;
CREATE OPERATOR ~~~ (
leftarg = text,
rightarg = text,
procedure = rlike
);
select * from addresses where 'bel%'::text ~~~ ANY (states);
It should be possible to define this function in C to make it faster.
I'm facing a strange behaviour when using a function to cast to composite type in Postgres 9.6.
I have declared a composite type "vector" as x,y,z - each of double precision as well as a cast as follows:
create type vector as(
x double precision,
y double precision,
z double precision
);
create cast (text as vector)
with function vector_from_text(text) as implicit;
Function vector_from_text looks like this:
create or replace function vector_from_text(text, out result vector) strict immutable language plpgsql as $$
declare
d double precision[];
begin
result := null;
if ($1 is null) then
return;
end if;
begin
with c1 as (
select $1::jsonb src
)
select row((src->>'x')::double precision, (src->>'y')::double precision, (src->>'z')::double precision)::vector
**into result** -- see below
from c1;
exception
when others then
d := translate($1, '()', '{}')::double precision[];
result := row(d[1], d[2], d[3])::vector;
end;
end$$
;
The function returns null on null, or a vector type for both input formats either a json-string like '{"x":0, "y":0, "z":0}' or a constructor expression like '(0,0,0)'.
The Problem:
For json-like inputs the functions returns the error
invalid input syntax for type double precision: "(0,0,0)"
as soon the select statement contains the line into result. If I remove this line or change the output variable from result to something of type text the functions works as expected.
Why is it not possible to assign an already to type vector casted value into a vector? Don't get it!
You do not need to (and in fact should not) create a cast from text. When you create a composite type you can cast a text to the type without any additional steps:
create type my_record as(
x int,
y int,
z int
);
select '(1,2,3)'::my_record;
my_record
-----------
(1,2,3)
(1 row)
If you want to use jsonb, create a cast from jsonb to the type:
create or replace function my_record_from_jsonb(obj jsonb, out result my_record)
strict immutable language plpgsql as $$
begin
select (obj->>'x')::int, (obj->>'y')::int, (obj->>'z')::int
into result;
end
$$;
create cast (jsonb as my_record)
with function my_record_from_jsonb(jsonb);
select '{"x":1, "y":2, "z":3}'::jsonb::my_record;
my_record
-----------
(1,2,3)
(1 row)
Do not try to interpret text literals in two different ways. If you want to use jsonb syntax, use jsonb type. Creating a custom implicit cast from text is particularly unreasonable. Read in the documentation::
It is wise to be conservative about marking casts as implicit. An overabundance of implicit casting paths can cause PostgreSQL to choose surprising interpretations of commands, or to be unable to resolve commands at all because there are multiple possible interpretations. A good rule of thumb is to make a cast implicitly invokable only for information-preserving transformations between types in the same general type category. For example, the cast from int2 to int4 can reasonably be implicit, but the cast from float8 to int4 should probably be assignment-only. Cross-type-category casts, such as text to int4, are best made explicit-only.
I'm writing a PL/pgSQL stored procedure that will return a set of records; each record contains all the fields of an existing table (call it Retailer, which has two fields: retailer_key and retailer_name). This, of course, works:
CREATE FUNCTION proc_Find_retailers
(IN p_Store_key INT)
RETURNS SETOF Retailer
AS $$ ...`
Now I want to update the sp so that it returns an additional two fields to the 'end' of each returned record. I can do something such as:
CREATE FUNCTION proc_Find_store
(IN p_Store_key INT)
RETURNS TABLE (
retailer_key int,
retailer_name varchar(50),
addl_field_1 int,
addl_field_2 double precision)
AS $$ ...
In the real world, my Retailer table has 50 fields (not the two in my example), so enumerating all those fields in the RETURNS TABLE clause is tedious. Is there any shortcut to this, so that I might say something such as (I realize I'm making stuff up here that's syntactically illegal, but I'm doing it to give you the flavor of what I'm looking for):
CREATE FUNCTION proc_Find_store
(IN p_Store_key INT)
RETURNS (SETOF Store,
addl_field_1 int,
addl_field_2 double precision)
AS $$ ...
You could return a whole row as composite type and add some more:
CREATE OR REPLACE FUNCTION f_rowplus()
RETURNS TABLE (rec demo, add_int int, add_txt text) AS
$func$
SELECT d, 5, 'baz'::text FROM demo d;
$func$ LANGUAGE sql;
But then, when you use the simple call:
SELECT * FROM f_rowplus();
You get the row from table demo as separate composite type. You'd have to call:
SELECT (rec).*, add_int, add_txt FROM f_rowplus();
to get all individual columns. Parentheses required.
Postgres is a bit inconsistent here. If you create a function with:
CREATE OR REPLACE FUNCTION f_row2()
RETURNS TABLE (rec demo) AS
...
then the composite type demo is silently converted into individual columns (decomposed). No link to the original composite type remains. You cannot reference the declared output column rec at all, since that has been replaced with the columns of the decomposed type. This call would result in an error message:
SELECT rec FROM f_row2(); -- error!
Same here:
CREATE OR REPLACE FUNCTION f_row3(OUT rec demo)
RETURNS SETOF demo AS
...
However, as soon as you add any more OUT columns, the composite type is preserved as declared (not decomposed) and you can:
SELECT rec FROM f_rowplus();
with the first function.
db<>fiddle here - demonstrating all variants
Old sqlfiddle
Asides
When using a function returning multiple columns in the FROM list (as table function) and decomposing in the SELECT list like this:
SELECT (rec).* FROM f_rowplus();
... the function is still evaluated once only - while calling and decomposing in the SELECT list directly like this:
SELECT (f_rowplus()).*; -- also: different result
... would evaluate once for every column in the return type. See:
How to avoid multiple function evals with the (func()).* syntax in an SQL query?
In Postgres 14 or later, you can also use standard-SQL syntax:
CREATE OR REPLACE FUNCTION f_rowplus_std()
RETURNS TABLE (rec demo, add_int int, add_txt text)
LANGUAGE sql PARALLEL SAFE
BEGIN ATOMIC
SELECT d, 5, 'baz'::text FROM demo d;
END;
See:
What does BEGIN ATOMIC ... END mean in a PostgreSQL SQL function / procedure?
Pointedly what I'm asking below is: What is the actual data type of the #cleartext parameter of this SQL function? >> ENCRYPTBYKEY (..) -
http://msdn.microsoft.com/en-us/library/ms174361.aspx
(If you read below this line you can follow the history and reasoning. I think it's trickier than it first appears.)
The SQL Server documentation states the #cleartext (2nd) parameter of the EncryptByKey(..) function can accept a number of various types:
EncryptByKey (#key_GUID , #cleartext [, #add_authenticator, #authenticator] )
#cleartext Is a variable of type
nvarchar, char, varchar, binary,
varbinary, or nchar that contains data
that is to be encrypted with the key.
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ - - - - - But what is its actual declared data type? ...
If I were to create a custom function (totally separate from the EncryptByKey example given above) what actual data type do I give a custom parameter so it will accept all those same types in the same manner?
Edit 1: I'm actually wrapping the SQL EncryptByKey function in a custom
UDF function and I want to recreate the same
parameter types to pass through to
it. This is the reasoning behind my want to create exact same params by type.
Edit 2: If I try using sql_variant it results in the error
Msg 8116, Level 16, State 1, Procedure
EncryptWrapper, Line 17 Argument data
type sql_variant is invalid for
argument 2 of EncryptByKey function.
Edit 3:
Here's my custom wrapper function - and the direct problem. What should the data type of #cleartext be for direct pass through to EncryptByKey?
ALTER FUNCTION [dbo].[EncryptWrapper]
(
#key_GUID uniqueidentifier,
#cleartext -- ??????????? <<< WHAT TYPE ????????????????
#add_authenticator int = 0,
#authenticator sysname = NULL
)
RETURNS varbinary(8000)
AS
BEGIN
-- //Calling a SQL Server builtin function.
-- //Second param #cleartext is the problem. What data type should it be?
Return EncryptByKey(#key_GUID, #cleartext, #add_authenticator, #authenticator)
END
Note: I shouldn't have to use CAST or CONVERT - I only need to use the proper data type for my #cleartext param.
Edit 4: Discovered the EncryptByKey(..) #cleartext parameter is not the following types:
sql_variant- raises error when passed
varbinary- too restrictive- doesn't allow passing of the text types otherwise accepted by EncryptByKey(..)
sysname, nvarchar, varchar- weird behaviour- tends to take only the first character of the argument text or something
try sql_variant:
CREATE FUNCTION [dbo].[yourFunction]
(
#InputStr sql_variant --can not be varchar(max) or nvarchar(max)
)
returns
varchar(8000)
BEGIN
--can use SQL_VARIANT_PROPERTY(#InputStr,'BaseType') to determine given datatype
--do whatever you want with #inputStr here
RETURN CONVERT(varchar(8000),#InputStr) --key is to convert the sql_varient to something you can use
END
GO
the key is to convert the sql_varient to something you can use within the function. you can use IF statements and check the BaseType and convert the sql_varient back into the native data type
EDIT
here is an example of how to get the original datatype:
CREATE FUNCTION [dbo].[yourFunction]
(
#InputStr sql_variant --can not be varchar(max) or nvarchar(max)
)
returns
varchar(8000)
BEGIN
DECLARE #Value varchar(50)
--can use SQL_VARIANT_PROPERTY(#InputStr,'BaseType') to determine given datatype
--do whatever you want with #inputStr here
IF #InputStr IS NULL
BEGIN
SET #value= 'was null'
END
ELSE IF SQL_VARIANT_PROPERTY(#InputStr,'BaseType')='char'
BEGIN
--your special code here
SET #value= 'char('+CONVERT(varchar(10),SQL_VARIANT_PROPERTY(#InputStr,'MaxLength '))+') - '+CONVERT(varchar(8000),#InputStr)
END
ELSE IF SQL_VARIANT_PROPERTY(#InputStr,'BaseType')='datetime'
BEGIN
--your special code here
SET #value= 'datetime - '+CONVERT(char(23),#InputStr,121)
END
ELSE IF SQL_VARIANT_PROPERTY(#InputStr,'BaseType')='nvarchar'
BEGIN
--your special code here
SET #value= 'nvarchar('+CONVERT(varchar(10),CONVERT(int,SQL_VARIANT_PROPERTY(#InputStr,'MaxLength '))/2)+') - '+CONVERT(varchar(8000),#InputStr)
END
ELSE
BEGIN
--your special code here
set #value= 'unknown!'
END
RETURN #value
END
GO
test it out:
DECLARE #x char(5), #z int, #d datetime, #n nvarchar(27)
SELECT #x='abc',#d=GETDATE(),#n='wow!'
select [dbo].[yourFunction](#x)
select [dbo].[yourFunction](#d)
select [dbo].[yourFunction](#z)
select [dbo].[yourFunction](#n)
test output:
-------------------------------------
char(5) - abc
(1 row(s) affected)
-------------------------------------
datetime - 2010-02-17 15:10:44.017
(1 row(s) affected)
-------------------------------------
was null
(1 row(s) affected)
-------------------------------------
nvarchar(27) - wow!
(1 row(s) affected)
ENCRYPTBYKEY() almost certainly isn't written in vanilla T-SQL. It doesn't need to follow T-SQL data typing rules.
That said, if you want to write a wrapper for it, use SQL_VARIANT for the #cleartext parameter, just as KM suggested.
If ENCRYPTBYKEY() is not sensitive to the max length of #cleartext, you could munge all CHAR/VARCHARs to VARCHAR(8000), and all NCHAR/NVARCHARs to NVACHAR(4000).
Otherwise you may be SOL: any data type conversion that respects maximum length--eg, CHAR(10) vs CHAR(20)--will require dynamic SQL, so you would have to write it as a stored procedure, rather than a function. At that point, it's not really a wrapper anymore.