In the official documentation, both functions have the same description:
Current date and time (start of current transaction)
Is there a difference between the two functions, and if not, why do both exist? Thanks.
now and transaction_timestamp are equivalent to the SQL standard current_timestamp. All report the start time of the transaction.
In terms of transactions, there are two timestamps to think of. the start of the transaction & the time each individual statement is executed. Conceptually, there is also the end time of the transaction which one can get by running a select statement_timestamp() trx_end_timestamp at the very end, just before the commit / rollback.
If you run the following in psql [copy the whole line & paste into psql shell]
BEGIN; SELECT PG_SLEEP(5); SELECT NOW(), CURRENT_TIMESTAMP, TRANSACTION_TIMESTAMP(), STATEMENT_TIMESTAMP(); COMMIT;
I got this output:
now | current_timestamp | transaction_timestamp | statement_timestamp
-------------------------------+-------------------------------+-------------------------------+-------------------------------
2019-04-23 11:15:18.676855-04 | 2019-04-23 11:15:18.676855-04 | 2019-04-23 11:15:18.676855-04 | 2019-04-23 11:15:23.713275-04
(1 row)
You can see clearly that NOW, CURRENT_TIMESTAMP, TRANSACTION_TIMESTAMP are equivalent, and STATEMENT_TIMESTAMP has a 5 second offset because we slept for 5 seconds.
BTW, CURRENT_TIMESTAMP is in the sql standard. The others are postgresql specific, though other databases may also implement them
The answer is in the doc your mention:
now() is a traditional PostgreSQL equivalent to
transaction_timestamp().
So, they are the same, and they are here for historical / backward compatibility, and some could argue for the simplicity of the function name.
Related
I have a postgres timescale database running in docker. For the purposes of api testing I want SELECT NOW() to return lets say 2010-12-01 23:00:44.851242 +00:00. Basically every time I start up the container I want it to think the current date is some time in December 2010.
How can I achieve this? I cant seem to find any command to set current time in postgres. Do I need to change the system time in the docker container before the database shows up? Is that even something I can do?
You can achieve this by creating a custom now() function in a separate schema and then adjusting the search_path to prefer that function over the builtin now function:
CREATE SCHEMA test;
CREATE OR REPLACE FUNCTION test.now() RETURNS timestamptz LANGUAGE SQL AS $$ SELECT '2000-01-01 0:00'::timestamptz; $$;
SET search_path TO test,pg_catalog,public;
-- make search_path change permanent for a specific user
ALTER USER <testuser> SET search_path TO test,pg_catalog,public;
SELECT now();
now
------------------------
2000-01-01 00:00:00+01
(1 row)
Time: 1.826 ms
I've got a Postgres 12.3 question: Can I rely on CLOCK_TIMESTAMP() in a trigger to stamp an updated_dts timestamp in exactly the same order as changes are committed to the permanent data?
On the face of it, this might sound like kind of an silly question, but I just spent two tracking down a super rare race condition in a non-Postgres system that hinged on exactly this behavior. (Lagging commits made their 'last value seen' tracking data unreliable.) Now I'm trying to figure out if it's possible for CLOCK_TIMESTAMP() to not match the order of changes recorded in the WAL perfectly.
It's simple to see how this could occur with NOW/TRANSACTION_TIMESTAMP/CURRENT_TIMESTAMP as they're returning the transaction start time, not the completion time. It's pretty easy, in that case, to record a timestamp sequence where the stamps and log order don't agree. But I can't figure out if there's any chance for commits to be saved in a different order to the BEFORE trigger CLOCK_TIMESTAMP() values.
For background, we need a 100% reliable timeline for an external search to use. As I understand it, I can create one using logical replication, and a replication-target side trigger to stamp changes as they're replayed from the log. What I'm unclear on, is if it's possible to get the same fidelity from CLOCK_TIMESTAMP() on a single server.
I haven't got the chops to get deep into the Postgres internals, and see how requests are interleaved, nor how granular execution is, and am hoping that someone here knows definitively. If this is more of a question for one of the PG mailing lists, please let me know.
-- Thanks
Below is a bit of sample code for how I'm looking at building the timestamps. It works fine, but doesn't prove anything about behavior with lots of concurrent processes.
---------------------------------------------
-- Create the trigger function
---------------------------------------------
DROP FUNCTION IF EXISTS api.set_updated CASCADE;
CREATE OR REPLACE FUNCTION api.set_updated()
RETURNS TRIGGER
AS $BODY$
BEGIN
NEW.updated_dts = CLOCK_TIMESTAMP();
RETURN NEW;
END;
$BODY$
language plpgsql;
COMMENT ON FUNCTION api.set_updated() IS 'Sets updated_dts field to CLOCK_TIMESTAMP(), if the record has changed..';
---------------------------------------------
-- Create the table
---------------------------------------------
DROP TABLE IF EXISTS api.numbers;
CREATE TABLE api.numbers (
id uuid NOT NULL DEFAULT extensions.gen_random_uuid (),
number integer NOT NULL DEFAULT NULL,
updated_dts timestamptz NOT NULL DEFAULT 'epoch'::timestamptz
);
---------------------------------------------
-- Define the triggers (binding)
---------------------------------------------
-- NOTE: I'm guessing that in production that I can use DEFAULT CLOCK_TIMESTAMP() instead of a BEFORE INSERT trigger,
-- I'm using a distinct DEFAULT value, as I want it to pop out if I'm not getting the trigger to fire.
CREATE TRIGGER trigger_api_number_before_insert
BEFORE INSERT ON api.numbers
FOR EACH ROW
EXECUTE PROCEDURE set_updated();
CREATE TRIGGER trigger_api_number_before_update
BEFORE UPDATE ON api.numbers
FOR EACH ROW
WHEN (OLD.* IS DISTINCT FROM NEW.*)
EXECUTE PROCEDURE set_updated();
---------------------------------------------
-- INSERT some data
---------------------------------------------
INSERT INTO numbers (number) values (1),(2),(3);
---------------------------------------------
-- Take a look
---------------------------------------------
SELECT * from numbers ORDER BY updated_dts ASC; -- The values should be listed as 1, 2, 3 as oldest to newest.
---------------------------------------------
-- UPDATE a row
---------------------------------------------
UPDATE numbers SET number = 11 where number = 1;
---------------------------------------------
-- Take a look
---------------------------------------------
SELECT * from numbers ORDER BY updated_dts ASC; -- The values should be listed as 2, 3, 11 as oldest to newest.
No, you cannot depend on clock_timestamp() order during trigger execution (or while evaluating a DEFAULT clause) being the same as commit order.
Commit will always happen later than the function call, and you cannot control how long it takes between them.
But I am surprised that that is a problem for you. Typically, the commit time is not visible or relevant. Why don't you simply accept the clock_timestamp() as the measure of things?
On development server I'd like to remove unused databases. To realize that I need to know if database is still used by someone or not.
Is there a way to get last access or modification date of given database, schema or table?
You can do it via checking last modification time of table's file.
In postgresql,every table correspond one or more os files,like this:
select relfilenode from pg_class where relname = 'test';
the relfilenode is the file name of table "test".Then you could find the file in the database's directory.
in my test environment:
cd /data/pgdata/base/18976
ls -l -t | head
the last command means listing all files ordered by last modification time.
There is no built-in way to do this - and all the approaches that check the file mtime described in other answers here are wrong. The only reliable option is to add triggers to every table that record a change to a single change-history table, which is horribly inefficient and can't be done retroactively.
If you only care about "database used" vs "database not used" you can potentially collect this information from the CSV-format database log files. Detecting "modified" vs "not modified" is a lot harder; consider SELECT writes_to_some_table(...).
If you don't need to detect old activity, you can use pg_stat_database, which records activity since the last stats reset. e.g.:
-[ RECORD 6 ]--+------------------------------
datid | 51160
datname | regress
numbackends | 0
xact_commit | 54224
xact_rollback | 157
blks_read | 2591
blks_hit | 1592931
tup_returned | 26658392
tup_fetched | 327541
tup_inserted | 1664
tup_updated | 1371
tup_deleted | 246
conflicts | 0
temp_files | 0
temp_bytes | 0
deadlocks | 0
blk_read_time | 0
blk_write_time | 0
stats_reset | 2013-12-13 18:51:26.650521+08
so I can see that there has been activity on this DB since the last stats reset. However, I don't know anything about what happened before the stats reset, so if I had a DB showing zero activity since a stats reset half an hour ago, I'd know nothing useful.
PostgreSQL 9.5 let us to track last modified commit.
Check track commit is on or off using the following query
show track_commit_timestamp;
If it return "ON" go to step 3 else modify postgresql.conf
cd /etc/postgresql/9.5/main/
vi postgresql.conf
Change
track_commit_timestamp = off
to
track_commit_timestamp = on
Restart the postgres / system
Repeat step 1.
Use the following query to track last commit
SELECT pg_xact_commit_timestamp(xmin), * FROM YOUR_TABLE_NAME;
SELECT pg_xact_commit_timestamp(xmin), * FROM YOUR_TABLE_NAME where COLUMN_NAME=VALUE;
My way to get the modification date of my tables:
Python Function
CREATE OR REPLACE FUNCTION py_get_file_modification_timestamp(afilename text)
RETURNS timestamp without time zone AS
$BODY$
import os
import datetime
return datetime.datetime.fromtimestamp(os.path.getmtime(afilename))
$BODY$
LANGUAGE plpythonu VOLATILE
COST 100;
SQL Query
SELECT
schemaname,
tablename,
py_get_file_modification_timestamp('*postgresql_data_dir*/*tablespace_folder*/'||relfilenode)
FROM
pg_class
INNER JOIN
pg_catalog.pg_tables ON (tablename = relname)
WHERE
schemaname = 'public'
I'm not sure if things like vacuum can mess this aproach, but in my tests it's a pretty acurrate way to get tables that are no longer used, at least, on INSERT/UPDATE operations.
I guess you should activate some log options. You can get information about logging on postgreSQL here.
I have an old web app, the relevant current stack is: Java 8, Tomcat 7, Apache Commons DBCP 2.1, Spring 2.5 (for transactions), iBatis, PostgreSQL 9.2 with postgresql-9.4.1208.jar
Part of the code inserts new records in incidents table, where the field begin_date (timestamp(3) with time zone) is a creation timestamp, filled with now:
insert into incidents
(...., begin_date, )
values
(..., 'now' ....)
All this is executed via iBatis, transactions managed programatically via Spring, connections acquired via DBCP pool. The webapp (actually a pair, clientside and backoffice, who share most of the code and jars) has been working since years.
Lately, perhaps after some libraries updates and reorganization (nothing important, it seemed), I've been experiencing (intermitent, hard to reproduce) some nasty problem: now seems to freeze, and it begins returning the same "old" value. Then, many records appear with the same creation timestamp, hours or days ago:
db=# select 'now'::timestamptz;
timestamp
-------------------------
2016-06-10 21:59:03.637+00
db=# select rid,begin_date from incidents order by rid desc limit 6;
rid | begin_date
-------+----------------------------
85059 | 2016-06-08 00:11:06.503+00
85058 | 2016-06-08 00:11:06.503+00
85057 | 2016-06-08 00:11:06.503+00
85056 | 2016-06-08 00:11:06.503+00
85055 | 2016-06-08 00:11:06.503+00
85054 | 2016-06-08 00:11:06.503+00
(All the above records were actually created minutes before 2016-06-10 21:50)
How can this happen? It might be some problem related with transactions and/or connection pooling, but I can't figure out what.
I know that 'now()' is an alias of transaction_timestamp(), it returns the time at the start of the transaction. This would suggest that a transaction was not properly closed, and the records inserts above were written (unintentionally) in a single long transaction. But this looks rather incredible to me.
First, I can insert a new record (via the webapp) and, using a psql console, I see that it has been written with the same begin_date (if the transaction were uncommited, I should not see the new record, I have the default serialization level).
Furthermore, the pg_stat_activity view only shows idle connections.
Any cues?
There's the constant (special timestamp value) 'now'.
And there's the function now().
The fact that you are mixing them freely suggests that you are unaware of the all-important difference. The manual:
Special Values
PostgreSQL supports several special date/time input values for
convenience, as shown in Table 8-13. The values infinity and -infinity
are specially represented inside the system and will be displayed
unchanged; but the others are simply notational shorthands that will
be converted to ordinary date/time values when read. (In particular,
now and related strings are converted to a specific time value as soon
as they are read.) All of these values need to be enclosed in single
quotes when used as constants in SQL commands.
Bold emphasis mine.
And (like you mentioned yourself already), but quoting the manual:
now() is a traditional PostgreSQL equivalent to transaction_timestamp().
And:
transaction_timestamp() is equivalent to CURRENT_TIMESTAMP
There is more, read the whole chapter.
Now (no pun intended), since you are using the special value instead of the function, you get a different (unexpected for you) behavior with prepared statements.
Consider this demo:
test=# BEGIN;
BEGIN
test=# PREPARE foo AS
test-# SELECT timestamptz 'now' AS now_constant, now() AS now_function;
PREPARE
test=# EXECUTE foo;
now_constant | now_function
-------------------------------+-------------------------------
2016-06-11 03:09:05.622783+02 | 2016-06-11 03:09:05.622783+02 -- identical
(1 row)
test=# commit;
COMMIT
test=# EXECUTE foo;
now_constant | now_function
-------------------------------+------------------------------
2016-06-11 03:09:05.622783+02 | 2016-06-11 03:10:00.92488+02 -- different!
(1 row)
While you run both in the same transaction, 'now' and now() produce the same value. But a prepared statement is designed to last for the duration of your session (possibly across many transactions). Next time you execute the prepared statement, you'll see the difference.
In other words: 'now' implements "early binding", while now() implements "late binding".
You may have introduced prepared statements and / or connection pooling (which can preserve prepared statements for a longer period of time) - both generally good ideas. But the hidden problem in your INSERT now starts kicking.
The "idle connections" you see indicate as much: connections stay open, preserving prepared statements.
In short: Use now().
Alternatively, set the column default of begin_date to now() (not 'now'!) and don't mention the column in the INSERT. Your "creation timestamp" is saved automatically.
I think the function you are looking for is now(), not 'now'...
insert into incidents
(..., begin_date, ...)
values
(..., now(), ...)
or at least that works from a psql shell.
robert.kuhar=# select now();
now
-------------------------------
2016-06-10 18:10:05.953661-07
(1 row)
I have created my tables with a column (type: timestamp with timezone) and set its default value to now() (current_timestamp()).
I run a series of inserts in separate statements in a single function and I noticed all the timestamps are equal down to the (ms), is the function value somehow cached and shared for the entire function call or transaction?
That is expected and documented behaviour:
From the manual:
Since these functions return the start time of the current transaction, their values do not change during the transaction. This is considered a feature: the intent is to allow a single transaction to have a consistent notion of the "current" time, so that multiple modifications within the same transaction bear the same time stamp.
If you want something that changes each time you run a statement, you need to use statement_timestamp() or even clock_timestamp() (again see the description in the manual)
now() and current_timestamp (the latter without parentheses - odd SQL standard) are STABLE functions returning the point in time when the transaction started as timestamptz.
Consider one of the other options PostgreSQL offers, in particular statement_timestamp(). The manual:
statement_timestamp() returns the start time of the current statement (more specifically, the time of receipt of the latest command message from the client)
Related:
Difference between now() and current_timestamp