Postgresql extracting 'epoch' from timestamp cuts off last date in date range - postgresql

My table has the column event_ts with column type numeric.
Here is my query:
select
min(to_timestamp(event_ts)), max(to_timestamp(event_ts))
from
table1
where
event_ts >= extract('epoch' from '2021-07-01'::timestamp) and
event_ts <= extract('epoch' from '2021-07-31'::timestamp)
However, the results are
min: 2021-06-30 20:00:00.000 -0400
max: 2021-07-30 20:00:00.000 -0400
I would think the where clause would include data from 2021-07-01 to 2021-07-31.
There is data for July 31st, 2021.
Why does this query start at 2021-06-30 and end 2021-07-30?

show timezone;
TimeZone
------------
US/Pacific
select extract('epoch' from '2021-07-01'::timestamp);
extract
-------------------
1625097600.000000
select to_timestamp(1625097600);;
to_timestamp
-------------------------
06/30/2021 17:00:00 PDT
select extract('epoch' from '2021-07-01'::timestamptz);
extract
-------------------
1625122800.000000
(1 row)
test(5432)=# select to_timestamp(1625122800);
to_timestamp
-------------------------
07/01/2021 00:00:00 PDT
So by using timestamp you are creating a local time offset by the timezone offset. Using timestamptz will return a timestamp at 0:00:00.
This is because from here:
https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT
epoch
For timestamp with time zone values, the number of seconds since 1970-01-01 00:00:00 UTC (negative for timestamps before that); for date and timestamp values, the nominal number of seconds since 1970-01-01 00:00:00, without regard to timezone or daylight-savings rules; for interval values, the total number of seconds in the interval
Epoch is based on UTC timezone.
Not sure why you are using epoch anyway?
Why not?:
...
where
event_ts between '2021-07-01'::timestamptz and '2021-07-31'::timestamptz

Related

Select a date in my time zone when database is in UTC with PostgreSQL

I want to select a day in my timezone for example:
from the 2021-09-09 00:00 +02 to 2021-09-10 00:00 +02. But the following code:
set timezone TO 'Europe/Berlin';
SELECT TIMESTAMP::timestamptz,sensor_id,value
FROM my_table
WHERE sensor_id IN (1,2,3)
AND TIMESTAMP > '2021-09-09' AND TIMESTAMP < '2021-09-10'
ORDER BY TIMESTAMP
gives me the right values but with wrong timezone for example first rows are:
timestamp,sensor_id,value
2021-09-09 02:00 +02,1,21
2021-09-09 02:00 +02,2,34
2021-09-09 02:00 +02,3,54
but should be 2021-09-09 00:00 +02 or 2021-09-08 22:00 +00
The problem is bigger when the difference with utc changes between winter and summer
Anybony can help me?
PostgreSQL v.12
SELECT TIMESTAMP::timestamptz AT TIME ZONE 'CETDST',sensor_id,value
FROM my_table
WHERE sensor_id IN (1,2,3)
AND TIMESTAMP AT TIME ZONE 'CETDST' > '2021-09-09'
AND TIMESTAMP AT TIME ZONE 'CETDST' < '2021-09-10'
ORDER BY TIMESTAMP

PostgreSQL grouping by day with timezone and DST

I'm working on an existing database which date+time values are stored in BIGINT column (milliseconds since EPOCH). For each entry I can get the corresponding time zone from another table. But to make things easier to understand I will explain my problem by simulating the problem.
The timestamp 1609534800000 is equal to 2021-01-01 21:00:00 at GMT (00-00)
Now if I run the following queries (with DBeaver)
set time zone 'America/Asuncion'; -- (UTC-3)
select
to_timestamp(1609534800000 / 1000) as "1"
, to_timestamp(1609534800000 / 1000) at TIME zone 'America/New_York' as "2"
, date_trunc('day', (to_timestamp(1609534800000 / 1000) at TIME zone 'America/New_York')) as "3"
, to_timestamp(1609534800000 / 1000) at TIME zone 'Pacific/Wake' as "4"
, date_trunc('day', (to_timestamp(1609534800000 / 1000) at TIME zone 'Pacific/Wake')) as "5"
, date_part('epoch', (to_timestamp(1609534800000 / 1000) at TIME zone 'Pacific/Wake')) as "6"
, date_part('epoch', (date_trunc('day', (to_timestamp(1609534800000 / 1000) at TIME zone 'Pacific/Wake') ) ) ) as "7"
I'm getting
1 |2 |3 |4 |5 |6 |7 |
-------------------|-------------------|-------------------|-------------------|-------------------|----------|----------|
2021-01-01 16:00:00|2021-01-01 16:00:00|2021-01-01 00:00:00|2021-01-02 09:00:00|2021-01-02 00:00:00|1609578000|1609545600|
I don't understand the result at all. According to the documentation, the function to_timestamp is supposed to return a timestamp with time zone ? In this case the time zone applied should be the one in my session America/Asuncion (UTC-3). If at GMT the time is 2021-01-01 21:00:00, I should get 2021-01-01 18:00:00. Because (21:00 - 3h = 18:00). Why 16h ?
From my understanding this result is OK as 2021-01-01 21:00:00 at UTC -5h for timezone America/New_York is equal to 2021-01-01 16:00.
Here I'm asking the same thing as the #2 but I want to discard the time of the day. So 2021-01-01 16:00:00 is 2021-01-01 00:00:00. The result is OK.
This result is still OK as 2021-01-01 21:00:00 at UTC + 12h for timezone Pacific/Wake is equal to 2021-01-02 09:00:00.
5 I'm asking the same thing as the #4 but I want to discard the time of the day. So 2021-01-02 09:00:00 is 2021-01-02 00:00:00. The result is OK.
I want to extract the unix EPOCH time in seconds of this timestamp. If I well understand, the timestamp pass to the date_part function is now a timestamp without time zone. Now if I use an online converter to convert the resulting value 1609578000 to GMT time then I'm getting 2021-01-02 9:00:00. Which is OK for me.
This is the same operation as the #6 but I want the unix epoch from the beginning of the day of that local time. The resulting value 1609545600 correspond to the GMT time 2021-01-02 00:00:00. Which is NOT correct as I should get 2021-01-02 12:00:00 as 'Pacific/Wake' is 12h past GMT.
(UPDATED)
Also why Montreal locale time is not correct here ? I'm supposed to have 2021-01-01 00:00:00-05
select ((to_timestamp(1609477200000 /1000) at time zone 'America/Asuncion') at time zone 'America/Asuncion') as asuncion
, ((to_timestamp(1609477200000 /1000) at time zone 'America/Montreal') at time zone 'America/Montreal') as montreal
asuncion | montreal
------------------------+------------------------
2021-01-01 02:00:00-03 | 2021-01-01 02:00:00-03
How could I get
asuncion | montreal
------------------------+------------------------
2021-01-01 02:00:00-03 | 2021-01-01 00:00:00-05
Is there a way to see the time WITHOUT the configured session timeszone ?
PS : My Windows OS timezone is set at America/New_York and I'm using PostgreSQL 10.
Best regards,
I can't replicate 1). I get:
set time zone 'America/Asuncion';
select to_timestamp(1609534800000 / 1000);
to_timestamp
------------------------
2021-01-01 18:00:00-03
As to 6), you did not account for the SET timezone:
select date_part('epoch', date_trunc('day', to_timestamp(1609534800000 / 1000) at TIME zone 'Pacific/Wake') at TIME zone 'Pacific/Wake');
date_part
------------
1609502400
select to_timestamp(1609502400);
to_timestamp
------------------------
2021-01-01 09:00:00-03
--What happened
select to_timestamp(1609534800000 / 1000) at TIME zone 'Pacific/Wake';
timezone
---------------------
2021-01-02 09:00:00
(1 row)
--- Note you lopped off 9 hours and the returned timestamp has no time zone offset
--- so it is now local time 'America/Asuncion'
select date_trunc('day', to_timestamp(1609534800000 / 1000) at TIME zone 'Pacific/Wake');
date_trunc
---------------------
2021-01-02 00:00:00
--- This gives it back a timezone offset.
--- Running the date_part on this then gets you the proper value.
select date_trunc('day', to_timestamp(1609534800000 / 1000) at TIME zone 'Pacific/Wake') at TIME zone 'Pacific/Wake';
timezone
------------------------
2021-01-01 09:00:00-03

How to do JOIN query on column having epoch value ignoring the time part in postgres

I have two tables lets say tableA and tableB , both the tables have a columnn creation_date having epoch value in it. I want to join these two tables on creation_date ignoring the time value in it.
Lets say if the epoch value is 1603385466134 which translates to Thu Oct 22 2020 22:21:06. Now the join should happen as Thu Oct 22 2020 00:00:00
I Tried this but not working
Select t.lr_transaction_id, t.unique_customer_id, t.transaction_id
from boidcrewardz.transaction_temp t
join boidcrewardz.transaction_dump d
on t.first_6_digit_card = d.first_6_digit_card
and t.transaction_amount = d.transaction_amount
and date_trunc('day',t.transaction_date) = date_trunc('day',d.transaction_date)
order by t.creation_date desc
Postgres 9 and later supports to_timestamp():
to_timestamp ( double precision ) → timestamp with time zone
Convert Unix epoch (seconds since 1970-01-01 00:00:00+00) to timestamp
with time zone
to_timestamp(1284352323) → 2010-09-13 04:32:03+00
Postgres 8 supports SELECT TIMESTAMP WITH TIME ZONE 'epoch' + 1603385466.134 * INTERVAL '1 second'
You just need to divide your epoch value by 1000.00 as your epoch is counting milliseconds while Postgres expects seconds.
One of the following should work:
select date_trunc('day',to_timestamp(1603385466134/1000.00));
SELECT date_trunc('day',TIMESTAMP WITH TIME ZONE 'epoch' + 1603385466134/1000.00 * INTERVAL '1 second');

FROM_TZ equivalent in PostgreSQL

How can I convert this to work in PostgreSQL?
TO_CHAR(CAST(FROM_TZ(CAST(columnname AS TIMESTAMP), 'GMT') AT TIME ZONE 'US/Eastern' AS DATE),'MM/DD/YY HH:MI AM') AS dt
testdb=# select TO_CHAR(CAST('2020-02-28T18:43' AS TIMESTAMP) AT TIME ZONE 'UTC' AT TIME ZONE 'US/Eastern','MM/DD/YY HH:MI AM') as dt;
dt
-------------------
02/28/20 01:43 PM
(1 row)
To make it clear what's going on, we'll start with the cast to TIMESTAMP, show that adding the first AT TIME ZONE makes it a tz-aware timestamp, and then how the 2nd does the timezone conversion.
testdb=# select CAST('2020-02-28T18:43' AS TIMESTAMP),
testdb-# CAST('2020-02-28T18:43' AS TIMESTAMP) AT TIME ZONE 'GMT',
testdb-# CAST('2020-02-28T18:43' AS TIMESTAMP) AT TIME ZONE 'GMT' AT TIME ZONE 'US/Eastern';
timestamp | timezone | timezone
---------------------+------------------------+---------------------
2020-02-28 18:43:00 | 2020-02-28 18:43:00+00 | 2020-02-28 13:43:00
(1 row)
See the timezone conversion docs for more details.

PostgreSQL UTC to CET/CEST

I have some big trouble with timezones and Benjamin Franklin.
I have a table with an UTC timestamp field, no timezone is stored.
My goal is to group the rows of this table by day of week or by slices like n hours.
This id how I proceeded until now :
-- CET => UTC/GMT + 1 => Winter
-- CEST => UTC/GMT + 2 => Summer
--
-- Hour changes 2014 : march 30th, 2h CET => 3h CEST
--
--
-- Output for "d" Wanted output for "d"
-- "2014-03-28 23:00:00+00" "2014-03-28 23:00:00+00" (23h UTC because CET)
-- "2014-03-29 23:00:00+00" "2014-03-29 23:00:00+00" (23h UTC because CET)
-- "2014-03-30 23:00:00+00" "2014-03-30 22:00:00+00" (22h UTC because CEST : fail because start timestamp is CET)
SELECT dates.d AS d, archives_d, range_begin_date, sequentialid
FROM generate_series('2014-03-28T23:00:00+00:00'::timestamp, -- 23h UTC because CET -- 2014-03-28T22:00:00+00:00
'2014-03-31T21:59:59+00:00'::timestamp, -- 22h UTC because CEST -- 2014-03-31T21:59:59+00:00
'86400 seconds') AS dates(d)
LEFT JOIN (
-- Select archive date floored by a step (here 86400 seconds -> 24 hours)
SELECT *, (to_timestamp(((floor(extract(epoch from (archives.range_begin_date - '2014-03-28T23:00:00+00:00'::timestamp)) / 86400)) * 86400) + extract(epoch from '2014-03-28T23:00:00+00:00'::timestamp))) AS archives_d
FROM archives
) AS archives
ON dates.d = archives.archives_d
ORDER BY dates.d
Do you have any idea of how get the wanted output (at least for the generate series).
Note that my step for the generate series is not fixed at one day but an arbitrary interval.
Thanks
There is a complete list of timezone names and offsets in pg_timezone_names.
I'm guessing you want a city-based name rather than a zone-based name. That way you get daylight-saving adjustments thrown in.
=> SELECT now() AT TIME ZONE 'America/New_York' AS ny, now() AT TIME ZONE 'Europe/London' AS lon;
ny | lon
---------------------------+---------------------------
2014-08-04 09:01:06.08988 | 2014-08-04 14:01:06.08988
(1 row)
The above was posted at 13:01 UTC. I'm guessing New York is what you want, but geography isn't my strong point :-)