In my test database, I want to override now() in Postgres, so I can travel to a certain point in time. I'd like to override it like this:
CREATE SCHEMA if not exists override;
CREATE OR REPLACE FUNCTION override.now()
RETURNS timestamp with time zone
AS
$$
BEGIN
RETURN pg_catalog.now() + COALESCE(
NULLIF(current_setting('timecop.offset_in_seconds', true), '')::integer, 0
) * interval '1 second';
END;
$$
LANGUAGE plpgsql STABLE PARALLEL SAFE STRICT;
SET search_path TO DEFAULT;
SELECT set_config('search_path', 'override,' || current_setting('search_path'), false);
To enable it, I call
SET timecop.offset_in_seconds = 3600 -- 1 hour ahead
To disable it, I call
RESET timecop.offset_in_seconds
The problem is, that Postgres somehow doesn't use the function:
app_test=# select now();
now
-------------------------------
2022-12-04 10:22:26.824469+00
(1 row)
app_test=# SET timecop.offset_in_seconds = 3600;
SET
app_test=# select now();
now
-------------------------------
2022-12-04 10:22:34.481502+00
(1 row)
Looking at the now() method itself, I seems like the search path searches in pg_catalog before the override schema:
app_test=# \df+ now
List of functions
Schema | Name | Result data type | Argument data types | Type | Volatility | Parallel | Owner | Security | Access privileges | Language | Source code | Description
------------+------+--------------------------+---------------------+------+------------+----------+----------+----------+-------------------+----------+-------------+--------------------------
pg_catalog | now | timestamp with time zone | | func | stable | safe | postgres | invoker | | internal | now | current transaction time
So, how could I move my overwritten now() BEFORE the pg_catalog?
pg_catalog is always on the search path, but you can opt not to have it in the beginning:
SET search_path = override, pg_catalog;
Related
I have a table with partially consecutive integer ids, i.e. there are blocks such as 1,2,3, 6,7,8, 10, 23,24,25,26.
the gap size is dynamic
the length of the blocks is dynamic
I am breaking my head about a simple solution that selects from the table
and includes a column where the value corresponds to the first id of the respective block.
I.e. something like this
select id, first(id) over <what goes here?> first from table;
The result should look as following
| id | first |
|----|-------|
| 1 | 1 |
| 2 | 1 |
| 3 | 1 |
| 6 | 6 |
| 7 | 6 |
| 8 | 6 |
| 10 | 10 |
| 23 | 23 |
| 24 | 23 |
| 25 | 23 |
| 26 | 23 |
Afterwards i could use this column nicely with the partition by window function clause.
What I came up with so far always looked similar to this and didn't succeed:
WITH foo AS (
SELECT LAG(id) OVER (ORDER BY id) AS previous_id,
id AS id,
id - LAG(id, 1, id) OVER (ORDER BY id) AS first_in_sequence
FROM table)
SELECT *,
FIRST_VALUE(id) OVER (ORDER BY id) AS first
FROM foo
ORDER BY id;
Defining a custom postgres function would also be an acceptable solution.
Thanks for any advice,
Marti
In Postgres you can create a custom aggregate. Example:
create or replace function first_in_series_func(int[], int)
returns int[] language sql immutable
as $$
select case
when $1[2] is distinct from $2- 1 then array[$2, $2]
else array[$1[1], $2] end;
$$;
create or replace function first_in_series_final(int[])
returns int language sql immutable
as $$
select $1[1]
$$;
create aggregate first_in_series(int) (
sfunc = first_in_series_func,
finalfunc = first_in_series_final,
stype = int[]
);
Db<>fiddle.
Read in the docs: User-Defined Aggregates
Here is an idea how this could be done. An implicit cursor is not horribly efficient though.
create or replace function ff()
returns table (r_id integer, r_first integer)
language plpgsql as
$$
declare
running_previous integer;
running_id integer;
running_first integer := null;
begin
for running_id in select id from _table order by id loop
if running_previous is distinct from running_id - 1 then
running_first := running_id;
end if;
r_id := running_id;
r_first := running_first;
running_previous := running_id;
return next;
end loop;
end
$$;
-- test
select * from ff() as t(id, first);
I expected this to be simple, but I've tried multiple combinations and I can't get PostgreSQL to accept them. It claims there are no matching functions, yet the types listed (as seen in the example below) clearly matches my defined function.
List of data types
Schema | Name | Internal name | Size | Elements | Access privileges | Description
--------+--------+---------------+------+----------+-------------------+-------------
public | levels | levels | 4 | debug +| |
| | | | info +| |
| | | | warn +| |
| | | | critical | |
CREATE FUNCTION public."logEvent"(IN in_type text,IN in_priority public.levels,IN in_message text)
RETURNS void
LANGUAGE 'sql'
NOT LEAKPROOF
AS $function$
INSERT INTO public.log (type,priority,message) VALUES (in_type, in_priority, in_message);
$function$;
The failing query:
SELECT 1 FROM public.logEvent('test'::text,'debug'::public.levels,'test from sql prompt'::text);
ERROR: function public.logevent(text, levels, text) does not exist
LINE 1: SELECT 1 FROM public.logEvent('test'::text,'debug'::public.l...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
An alternate function definition: I get the same error when trying to execute this create function call:
CREATE FUNCTION public."logEvent"(IN in_type text, IN in_priority text, IN in_message text)
RETURNS boolean
LANGUAGE 'sql'
NOT LEAKPROOF
AS $function$
SELECT public.logEvent( in_type, CAST(lower(in_priority) AS public.levels), in_message);
$function$;
Are you aware that double-quoted identifiers are case-sensitive?
Are PostgreSQL column names case-sensitive?
CREATE FUNCTION public."logEvent"
But:
SELECT 1 FROM public.logEvent
This should work:
SELECT 1 FROM public."logEvent"('test', 'debug', 'test from sql prompt');
Explicit type casts are only necessary if there can be ambiguity with overloaded function.
Either keep double-quoting "logEvent" for the rest of its existence, or (smarter) use unquoted (effectively lower-case), legal identifiers.
I have a function where I want to create a table for a every year based on the year from bill date which I will be looping.
CREATE OR REPLACE FUNCTION ccdb.ccdb_archival()
RETURNS void AS
$BODY$
DECLARE dpsql text;
DECLARE i smallint;
BEGIN
FOR i IN SELECT DISTINCT EXTRACT(year FROM bill_date) FROM ccdb.bills ORDER BY 1 LOOP
DO $$
BEGIN
CREATE TABLE IF NOT EXISTS ccdb_archival.bills||i (LIKE ccdb.bills INCLUDING ALL);
BEGIN
ALTER TABLE ccdb_archival.bills ADD COLUMN archival_date timestamp;
EXCEPTION
WHEN duplicate_column THEN RAISE NOTICE 'column archival_date already exists in <table_name>.';
END;
END;
$$;
INSERT INTO ccdb_archival.bills
SELECT *, now() AS archival_date
FROM ccdb.bills
WHERE bill_date::date >= current_date - interval '3 years' AND bill_date::date < current_date - interval '8 years';
END LOOP;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
I want to concatenate the year with the actual table name for each year.
I am unable to do the same with the above code. I get an error:
ERROR: syntax error at or near "||"
LINE 3: CREATE TABLE IF NOT EXISTS ccdb_archival.bills||i (LI...
Please suggest how do I achieve my requirement.
you cannot compose strings with metadata. You should utilize execute: http://www.postgresql.org/docs/9.1/static/ecpg-sql-execute-immediate.html
To create N tables with a prefix use this script.
This code uses a for loop and variable to creates 10 table starting with prefix 'sbtest' namely sbtest1, sbtest2 ... sbtest10
create_table.sql
do $$
DECLARE myvar integer;
begin
for myvar in 1..10 loop
EXECUTE format('CREATE TABLE sbtest%s (
id SERIAL NOT NULL,
k INTEGER NOT NULL,
c CHAR(120) NOT NULL,
pad CHAR(60) NOT NULL,
PRIMARY KEY (id))', myvar);
end loop;
end; $$
Run it using psql -U user_name -d database_name -f create_table.sql
Example Table sbtest1 is as
id | k | c | pad
----+---+---+-----
(0 rows)
Table "public.sbtest1"
Column | Type | Collation | Nullable | Default | Storage | Stats
target | Description
--------+----------------+-----------+----------+-------------------------------------+----------+------
--------+-------------
id | integer | | not null | nextval('sbtest1_id_seq'::regclass) | plain |
|
k | integer | | not null | | plain |
|
c | character(120) | | not null | | extended |
|
pad | character(60) | | not null | | extended |
|
Indexes:
"sbtest1_pkey" PRIMARY KEY, btree (id)
Access method: heap
CREATE OR REPLACE FUNCTION getParentLtree(parent_id bigint, tbl_name varchar)
RETURNS ltree AS
$BODY$
DECLARE
parent_ltree ltree;
BEGIN
-- This works fine:
-- select into parent_ltree l_tree from tbl1 where id = parent_id;
EXECUTE format('select into parent_ltree l_tree from %I
where id = %I', tbl_name,parent_id);
RETURN parent_ltree;
END;
$BODY$ LANGUAGE plpgsql;
There are 2 issues in above function:
parent_id is integer but it is replaced with quotes? What is the correct format specifier for int variables?
select into does not work with EXECUTE? How can I make above commented query to use table name passed?
This would be shorter, faster and safer:
CREATE OR REPLACE FUNCTION get_parent_ltree(parent_id bigint, tbl_name regclass
, OUT parent_ltree ltree)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format('SELECT l_tree FROM %s WHERE id = $1', tbl_name)
INTO parent_ltree
USING parent_id;
END
$func$;
Why?
Most importantly, use the USING clause of EXECUTE for parameter values. Don't convert them to text, concatenate and interpret them back. That would be slower and error-prone.
Normally you would use the %I specifier with format() for identifiers like the table name. For existing tables, a regclass object-identifier type may be even better. See:
Table name as a PostgreSQL function parameter
The OUT parameter makes it simpler. Performance is the same.
Don't use unquoted CaMeL case identifiers like getParentLtree in Postgres. Details in the manual.
Use %s for strings. %I is for identifiers:
select format('select into parent_ltree l_tree from %I where id = %s', 'tbl1', 1);
format
---------------------------------------------------------
select into parent_ltree l_tree from tbl1 where id = 1
http://www.postgresql.org/docs/current/static/functions-string.html#FUNCTIONS-STRING-FORMAT
PL/pgSQL's select into is not the same as Postgresql's select into. Use instead create table as:
create table parent_ltree as
select l_tree
from tbl1
where id = 1
http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-SQL-ONEROW
Tip: Note that this interpretation of SELECT with INTO is quite different from PostgreSQL's regular SELECT INTO command, wherein the INTO target is a newly created table. If you want to create a table from a SELECT result inside a PL/pgSQL function, use the syntax CREATE TABLE ... AS SELECT.
To select into a variable from an execute statement:
EXECUTE format('select l_tree from %I where id = %s', tbl_name,parent_id)
into parent_ltree;
http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-EXECUTING-DYN
Following postgres uses a for loop and variable to creates 10 table starting with prefix 'sbtest' namely sbtest1, sbtest2 ... sbtest10
create_table.sql
do $$
DECLARE myvar integer;
begin
for myvar in 1..10 loop
EXECUTE format('CREATE TABLE sbtest%s (
id SERIAL NOT NULL,
k INTEGER NOT NULL,
c CHAR(120) NOT NULL,
pad CHAR(60) NOT NULL,
PRIMARY KEY (id))', myvar);
end loop;
end; $$
Run it using psql -U user_name -d database_name -f create_table.sql
Example Table sbtest1 is as
id | k | c | pad
----+---+---+-----
(0 rows)
Table "public.sbtest1"
Column | Type | Collation | Nullable | Default | Storage | Stats
target | Description
--------+----------------+-----------+----------+-------------------------------------+----------+------
--------+-------------
id | integer | | not null | nextval('sbtest1_id_seq'::regclass) | plain |
|
k | integer | | not null | | plain |
|
c | character(120) | | not null | | extended |
|
pad | character(60) | | not null | | extended |
|
Indexes:
"sbtest1_pkey" PRIMARY KEY, btree (id)
Access method: heap
We want to use pgsql_fdw to select a table of remote postgresql database. When we
select the table in a session it is okay, but when we use the foreign table in a funciton
it turns out "ERROR: cache lookup failed for type 0". Does anybody know why?
1. base information
skytf=> \d ft_test;
Foreign table "skytf.ft_test"
Column | Type | Modifiers
--------+-----------------------+-----------
id | integer |
name | character varying(32) |
Server: pgsql_srv
skytf=> \des+ pgsql_srv
List of foreign servers
Name | Owner | Foreign-data wrapper | Access privileges | Type | Version | Options
-----------+-------+----------------------+-------------------+------+---------+----------------------------------------
pgsql_srv | skytf | pgsql_fdw | | | | {host=127.0.0.1,port=1923,dbname=mydb}
(1 row)
2. destination table
mydb=> \d test
Table "mydb.test"
Column | Type | Modifiers
--------+-----------------------+-----------
id | integer |
name | character varying(32) |
Indexes:
"idx_test_1" btree (id)
3. function
CREATE or replace FUNCTION func_sync_bill() RETURNS INTEGER AS $$
BEGIN
begin
insert into test_tf (id,name) select id,name from ft_test;
return 1;
end;
END;
$$ LANGUAGE 'plpgsql';
4. it works in a session
skytf=> create table test_tf(id integer,name varchar(32));
CREATE TABLE
skytf=> insert into test_tf select * from ft_test;
INSERT 0 1990000
5. function call error
skytf=> truncate table test_tf;
TRUNCATE TABLE
skytf=> select func_sync_bill();
ERROR: cache lookup failed for type 0
CONTEXT: SQL statement "insert into test_tf (id,name) select id,name from ft_test"
PL/pgSQL function "func_sync_bill" line 5 at SQL statement
When I call the function func_sync_bill() which will select a foreign table, it generates the the error.
Is this a bug of pgsql_fdw?
Looks like a bug in the foreign data wrapper or plpgsql or something. Given that the \d commands are working, I would expect that the catalogs are fine, which suggests that the real problem is data corruption although the fact that it works outside the function suggests that maybe that's not the case.
things to do to troubleshoot would be to try to rewrite your function as an SQL language function instead of plpgsql and see if that fixes the problem. if it does, you have a workaround. Also try on more recent minor versions of PostgreSQL. If the problem is fixed, then great. But if not, this is a case where a bug report to the project may result in getting the problem fixed.
Finaly I use dynamic statements avoid the problem.
--dynamic statements
skytf=> CREATE or replace FUNCTION func_sync_bill() RETURNS INTEGER AS $$
skytf$> BEGIN
skytf$> begin
skytf$> EXECUTE 'insert into test_tf (id,name) select id,name from ft_test';
skytf$> return 0;
skytf$> end;
skytf$> END;
skytf$> $$ LANGUAGE 'plpgsql';
CREATE FUNCTION
--try again
skytf=> truncate table test_tf;
TRUNCATE TABLE
skytf=> select func_sync_bill();
func_sync_bill
----------------
0
(1 row)
skytf=> select count(*) from test_tf;
count
---------
1990000
(1 row)