I have ERP application that uses the system date when posting transactions. The database is PostgreSQL. I'm able to use https://www.nirsoft.net/utils/run_as_date.html for backdate the application but I notice that the transactions are still posting as of "today" and I think that maybe because of PostgreSQL using the system date.
Is there any way I can set the date back for PostgreSQL? Or any other way to do this? The process in the ERP application does not have an option to back date.
The easiest would be to add a trigger to the database that would change the date for inserted rows:
create table testpast(
id serial primary key,
time timestamp with time zone not null default now()
);
insert into testpast (time) values (default);
select * from testpast;
id | time
----+-------------------------------
1 | 2018-03-16 00:09:20.219419+01
(1 row)
create function time_20_years_back() returns trigger as $$
begin
NEW.time = now()-'20 years'::interval;
return NEW;
end;
$$ language plpgsql;
create trigger testpast_time_20_years_back
before insert on testpast
for each row
execute procedure time_20_years_back();
insert into testpast (time) values (default);
select * from testpast;
id | time
----+-------------------------------
1 | 2018-03-16 00:09:20.219419+01
2 | 1998-03-16 00:09:55.741345+01
(2 rows)
Though I have no idea what would be the purpose of such a hack.
Related
There is some option to store only hours and minutes in the database (No seconds)?
I know that I can use the function to_char in my query, but this is not the solution for me.
I am using/editing a system written in PHP that I cannot edit (I mean PHP), it is something like WYSIWYG, and it injects the values directly into the forms (Talking about editing existing), and the form should be in HH:MM and as long as the injection does not cause an application error , saving (Even without editing) causes an error due to validation because it incject HH:MM:SS because that is the column type.
So far, to work around the problem, I have created text columns, but it creates other complications and I was wondering if it could be done in a different way
Assuming that
You can't modify your PHP application.
You wish to keep a column of type time for other purposes.
There are no select * anywhere that expect the column layout of that table to remain unchanged.
You only need this one piece of the application to select, insert and update that table as if HH:MM was its default output format.
The application won't mind that it'll read text instead of time type data, as long as it's HH:MM.
You can do a column swap and handle incoming data in a trigger, also maintaining integrity between the actual time-type column and its text representation column. That way anything that doesn't rely on the data being type `time will get HH:MM format from that particular table.
drop table if exists test_70092210;
create table test_70092210 as select now()::time time_column;
select time_column from test_70092210;
-- time_column
-------------------
-- 12:06:23.890971
--(1 row)
alter table test_70092210 rename column time_column to time_column_raw;--column swap
alter table test_70092210 add column time_column text;--column swap
update test_70092210 set time_column=to_char(time_column_raw,'HH24:MI');
select time_column from test_70092210;
-- time_column
---------------
-- 12:06
--(1 row)
While updates and inserts can be handled using triggers:
CREATE or replace FUNCTION test_70092210_time_column_insert_handler() RETURNS trigger AS $test_70092210_time_column_insert_handler$
BEGIN
NEW.time_column_raw=coalesce( NEW.time_column_raw::time,
NEW.time_column::time);
NEW.time_column=coalesce( to_char(NEW.time_column_raw::time,'HH24:MI'),
to_char(NEW.time_column::time,'HH24:MI'));
RETURN NEW;
END;
$test_70092210_time_column_insert_handler$ LANGUAGE plpgsql;
create or replace trigger test_70092210_time_column_insert_handler_trigger before insert on test_70092210
for each row execute function test_70092210_time_column_insert_handler();
CREATE or replace FUNCTION test_70092210_time_column_update_handler() RETURNS trigger AS $test_70092210_time_column_update_handler$
BEGIN
NEW.time_column_raw=case
when NEW.time_column_raw<>OLD.time_column_raw
then NEW.time_column_raw::time
else
NEW.time_column::time
end;
NEW.time_column=case
when NEW.time_column_raw<>OLD.time_column_raw
then to_char(NEW.time_column_raw::time,'HH24:MI')
else
to_char(NEW.time_column::time,'HH24:MI')
end;
RETURN NEW;
END;
$test_70092210_time_column_update_handler$ LANGUAGE plpgsql;
create or replace trigger test_70092210_time_column_update_handler_trigger before update on test_70092210
for each row execute function test_70092210_time_column_update_handler();
insert into test_70092210 select now()-'01:30'::time;
select time_column from test_70092210;
-- time_column
---------------
-- 12:06
-- 10:37
--(2 rows)
update test_70092210 set time_column='16:23' where ctid in (select min(ctid) from test_70092210);
select time_column from test_70092210;
-- time_column
---------------
-- 10:37
-- 16:23
--(2 rows)
Having the cookie and eating it too:
select * from test_70092210;
-- time_column_raw | time_column
-------------------+-------------
-- 10:37:19.91891 | 10:37
-- 16:23:00 | 16:23
--(2 rows)
Beside of unique actions we need recurrent actions in our database. We wan't the user to be able to define a periodicity (all 1,2,3,.. years) and a period (e.g. from 2018 - to 2020) in a form. This data should be used to insert appropriat datasets for a defined action.
If the user chooses an annual periodicity starting from 2018 3 datasets (2018, 2019 and 2020) should be inserted in the actions table.
If the user chooses an biannual periodicity starting from 2018 only 2 datasets (2018 and 2020) should be inserted in the actions table.
The simplified table actions looks like this:
id serial not null
id_action integer
action_year integer
periodicity integer
from_ integer
to_ integer
I need a starting point for the sql statement.
You should use generate_series(start, stop, step)
Annual:
=> select generate_series(2018,2020,1);
generate_series
-----------------
2018
2019
2020
(3 rows)
Biannual:
=> select generate_series(2018,2020,2);
generate_series
-----------------
2018
2020
(2 rows)
I didn't knew the function generate_series() until now. Thanks to point me in that direction.
To get things running like I intended I need to use the generate_series() inside an Trigger function that is fired AFTER INSERT. After first running into troubles with recursive Trigger inserts I now have the problem, that my Trigger produces to many duplicate inserts (increasing with the choosen periodicity).
My table actions looks like this:
id serial not null
id_action integer
action_year integer
periodicity integer
from_ integer
to_ integer
My Trigger on the table:
CREATE TRIGGER tr_actions_recurrent
AFTER INSERT
ON actions
FOR EACH ROW
WHEN ((pg_trigger_depth() = 0))
EXECUTE PROCEDURE actions_recurrent();
Here my trigger function:
CREATE OR REPLACE FUNCTION actions_recurrent()
RETURNS trigger AS
$BODY$
BEGIN
IF NEW.periodicity >0 AND NEW.action_year <= NEW.to_-NEW.periodicity THEN
INSERT into actions(id_action, action_year,periodicity, from_, to_)
SELECT NEW.id_action, y, NEW.periodicity, NEW.from_, NEW.to_
FROM actions, generate_series(NEW.from_+NEW.periodicity,NEW.to_,NEW.periodicity) AS y;
END IF;
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
When I'm doing an insert
INSERT INTO actions (id_action, action_year,periodicity,from_, to_)
VALUES (50,2018,4,2018,2028);
I get one row for action_year 2018, but 13 rows 2022 and 2026??
In my understanding, the IF-clause in the trigger-function should avoid such a repetitive execution.
I have created a function that creates a temporary table and inserts into the table. The problem is that I need this to work in a read-only instance as well, but in a read-only instance I can't create a table and/or insert into it. Is there any other way of doing this? Maybe by creating a table variable in a way similar to other SQL languages?
I did some research and there doesn't seem to be a table variable, but maybe an array of records? Any ideas?
UPDATE:
To answer people's questions, I am trying to create a function that returns a list of dates from now until x intervals ago in intervals of y.
So for instance, select * from getDates('3 days', 1 day') returns:
startdate | enddate
------------+------------
2016-07-20 | 2016-07-21
2016-07-19 | 2016-07-20
2016-07-18 | 2016-07-19
And select * from getDates('3 months', '1 month'); returns:
startdate | enddate
------------+------------
2016-07-01 | 2016-08-01
2016-06-01 | 2016-07-01
2016-05-01 | 2016-06-01
I currently do this by using a while loop and going back per interval until I hit the time given by the first parameter. I then insert those values into a temp table, and when finished I select everything from the table. I can include the code if necessary.
You can create a permanent named Composite Type representing the structure of your temporary table, and then use an array variable to manipulate a set of rows inside a function:
-- Define columns outside function
CREATE TYPE t_foo AS
(
id int,
bar text
);
CREATE OR REPLACE FUNCTION test()
RETURNS SETOF t_foo AS
$BODY$
DECLARE
-- Create an empty array of records of the appropriate type
v_foo t_foo[] = ARRAY[]::foo[];
BEGIN
-- Add some rows to the array
v_foo := v_foo || ( 42, 'test' )::t_foo;
v_foo := v_foo || ( -1, 'nothing' )::t_foo;
-- Convert the array to a resultset as though it was a table
RETURN QUERY SELECT * FROM unnest(v_foo);
END;
$BODY$
LANGUAGE plpgsql;
SELECT * FROM test();
The crucial part here is the variable of type t_foo[] - that is, an array of records of the pre-defined type t_foo.
This is not as easy to work with as a temporary table or table variable, because you need to use array functions to get data in and out, but may be useful.
It's worth considering though whether you really need the complex local state, or whether your problem can be re-framed to use a different approach, e.g. sub-queries, CTEs, or a set-returning function with RETURN NEXT.
Maybe the best way to approach it is to get your administrator to GRANT TEMPORARY ON DATABASE database_name TO the user account performing your actions. You still will only have read-only access to the database.
Declare the function as retuning table
create function f()
returns table (
a int,
b text
) as $$
select x, y from t;
$$ language sql;
Use it as:
select *
from f()
My idea is to implement a basic «vector clock», where a timestamps are clock-based, always go forward and are guaranteed to be unique.
For example, in a simple table:
CREATE TABLE IF NOT EXISTS timestamps (
last_modified TIMESTAMP UNIQUE
);
I use a trigger to set the timestamp value before insertion. It basically just goes into the future when two inserts arrive at the same time:
CREATE OR REPLACE FUNCTION bump_timestamp()
RETURNS trigger AS $$
DECLARE
previous TIMESTAMP;
current TIMESTAMP;
BEGIN
previous := NULL;
SELECT last_modified INTO previous
FROM timestamps
ORDER BY last_modified DESC LIMIT 1;
current := clock_timestamp();
IF previous IS NOT NULL AND previous >= current THEN
current := previous + INTERVAL '1 milliseconds';
END IF;
NEW.last_modified := current;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS tgr_timestamps_last_modified ON timestamps;
CREATE TRIGGER tgr_timestamps_last_modified
BEFORE INSERT OR UPDATE ON timestamps
FOR EACH ROW EXECUTE PROCEDURE bump_timestamp();
I then run a massive amount of insertions in two separate clients:
DO
$$
BEGIN
FOR i IN 1..100000 LOOP
INSERT INTO timestamps DEFAULT VALUES;
END LOOP;
END;
$$;
As expected, I get collisions:
ERROR: duplicate key value violates unique constraint "timestamps_last_modified_key"
État SQL :23505
Détail :Key (last_modified)=(2016-01-15 18:35:22.550367) already exists.
Contexte : SQL statement "INSERT INTO timestamps DEFAULT VALUES"
PL/pgSQL function inline_code_block line 4 at SQL statement
#rach suggested to mix current_clock() with a SEQUENCE object, but it would probably imply getting rid of the TIMESTAMP type. Even though I can't really figure out how it'd solve the isolation problem...
Is there a common pattern to avoid this?
Thank you for your insights :)
If you have only one Postgres server as you said, I think that using timestamp + sequence can solve the problem because sequence are non transactional and respect the insert order.
If you have db shard then it will be much more complex but maybe the distributed sequence of 2ndquadrant in BDR could help but I don't think that ordinality will be respected. I added some code below if you have setup to test it.
CREATE SEQUENCE "timestamps_seq";
-- Let's test first, how to generate id.
SELECT extract(epoch from now())::bigint::text || LPAD(nextval('timestamps_seq')::text, 20, '0') as unique_id ;
unique_id
--------------------------------
145288519200000000000000000010
(1 row)
CREATE TABLE IF NOT EXISTS timestamps (
unique_id TEXT UNIQUE NOT NULL DEFAULT extract(epoch from now())::bigint::text || LPAD(nextval('timestamps_seq')::text, 20, '0')
);
INSERT INTO timestamps DEFAULT VALUES;
INSERT INTO timestamps DEFAULT VALUES;
INSERT INTO timestamps DEFAULT VALUES;
select * from timestamps;
unique_id
--------------------------------
145288556900000000000000000001
145288557000000000000000000002
145288557100000000000000000003
(3 rows)
Let me know if that works. I'm not a DBA so maybe it will be good to ask on dba.stackexchange.com too about the potential side effect.
My two cents (Inspired from http://tapoueh.org/blog/2013/03/15-batch-update).
try adding the following before massive amount of insertions:
LOCK TABLE timestamps IN SHARE MODE;
Official documentation is here: http://www.postgresql.org/docs/current/static/sql-lock.html
I am using Postgresql 9.0.5 and I have a cron job that periodically reads newly created rows from a table and accumulate its value into a summary table that has hourly data.
I need to get the latest ID (serial) that is committed and all rows before it are committed.
The currval function will not give a correct value in this case, because the transaction inserting currval may commit earlier than others. Using SELECT statement at a moment, I can see Id column is not continuous because some rows are still not committed.
Here is some sample code I have used to test:
--test race condition
create table mydata(id serial,val int);
--run in thread 1
create or replace function insert_delay() returns void as $$
begin
insert into mydata(val) values (1);
perform pg_sleep(60);
end;
$$ language 'plpgsql';
--run in thread 2
create or replace function insert_ok() returns void as $$
begin
insert into mydata(val) values (2);
end;
$$ language 'plpgsql';
--run in thread 3
mytest=# select * from mydata; --SHOULD HAVE SEEN id = 1 and 2;
id | val
----+-----
2 | 2
(1 row)
I even tried some statement like the one below;
select max(id) from mydata age(xmin) >= age(txid_snapshot_xmin(txid_current_snapshot())::text::xid);
But in production line (running high volume transactions), the returned max(id) will not move forwards (even all the busy transaction are finished). So this does not work either.
There isn't a really good way to do this directly. I think the best option really is to create a temporary table which truncates on transaction commit, and a trigger that inserts such into that table. Then you can look up the values from the temp table.