I am trying to query PostgreSQL database for rows where interval has elapsed from the last run. Main columns for this purpose are processed_at as timestamptz and frequency (in minutes) as integer.
I am failing with operators, since not many of them can operate together timestamp & integer.
Can someone please propose a query that would solve this? Thank you very much for help
From here Date/time operators:
timestamp + interval → timestamp
Add an interval to a timestamp
timestamp '2001-09-28 01:00' + interval '23 hours' → 2001-09-29 00:00:00
select now() + (10::varchar || ' min')::interval;
?column?
-------------------------------
2021-10-15 09:05:37.927163-07
--Or in your case. If I'm following you are adding the interval.
select processed_at + (frequency::varchar || ' min')::interval;
The query takes the integer value of minutes and converts it to an interval of minutes that can be added to the timestamp.
Further explanation, || is the Postgres concatenation operator and ::varchar, ::interval are casting shorthand.
UPDATE
I keep forgetting about the make_*() functions for date/time/interval
--A shorter version
select processed_at + make_interval(mins => frequency);
Saves all the casting.
Related
I am trying to translate following sql query into knex:
select count(*) as finished_on_time from task_history
where date = 20160303
and store_id = 2
and (schedule_start_time at time zone 'Australia/sydney' + interval '1' minute * floor (group_duration) )::time >= (finish_time at time zone 'Australia/sydney')::time
date field has in yyyymmdd format
Here is what I have been trying on knex:
db.table('task_history')
.count('*')
.where({date: request.params.storeid, store_id: request.params.storeid })
??????
As you can guess, I am not sure which clause to use to handle sql syntax [at time zone Australia/sydney].
I have been trying to find any similar soloutions on the internet, but ended up here.
http://knexjs.org/#Builder-whereRaw
db.table('task_history')
.count('*')
.where({date: request.params.storeid, store_id: request.params.storeid })
.whereRaw("(schedule_start_time at time zone 'Australia/sydney' + interval '1' minute * floor (group_duration) )::time >= (finish_time at time zone 'Australia/sydney')::time")
I have a column which is of bigint datatype(in seconds) which should be added to a date, so i need to convert this column into dateformat.
The arithmetic must be done against a timestamp data type in Teradata. The date data type does not have a time element associated with it. The following SQL should help point you in the right direction:
SELECT CAST(CAST(1234 AS BIGINT) AS INTERVAL SECOND(4)) AS Seconds_
, CURRENT_TIMESTAMP(0) AS CurrentTimestamp_
, CURRENT_TIMESTAMP + Seconds_ AS NewTimeStamp
If the number of seconds is less than 864000000 you can simply use interval arithmetic:
CAST(col AS TIMESTAMP) + (bigintcol * INTERVAL '0000 00:00:01' DAY TO SECOND)
Based on your other question your input is a Unixtime, those are two functions for converting them from/to Teradata timestamps:
/**********
Converting Unix/POSIX time to a Timestamp
Unix time: Number of seconds since 1970-01-01 00:00:00 UTC not counting leap seconds (currently 24 in 2011)
Also working for negative numbers.
The maximum range of Timestamps is based on the range of INTEGERs:
1901-12-13 20:45:52 (-2147483648) to 2038-01-19 03:14:07 (2147483647)
Can be changed to use BIGINT instead of INTEGER
20101211 initial version - Dieter Noeth
**********/
REPLACE FUNCTION UnixTime_to_TimeStamp (UnixTime INT)
RETURNS TimeStamp(0)
LANGUAGE SQL
CONTAINS SQL
DETERMINISTIC
SQL SECURITY DEFINER
COLLATION INVOKER
INLINE TYPE 1
RETURN
CAST(DATE '1970-01-01' + (UnixTime / 86400) AS TIMESTAMP(0))
+ ((UnixTime MOD 86400) * INTERVAL '00:00:01' HOUR TO SECOND)
;
SELECT
UnixTime_to_TimeStamp(-2147483648)
,UnixTime_to_TimeStamp(0)
,UnixTime_to_TimeStamp(2147483647)
;
/**********
Converting a Timestamp to Unix/POSIX time
Unix time: Number of seconds since 1970-01-01 00:00:00 UTC not counting leap seconds (currently 24 in 2011)
The maximum range of Timestamps is based on the range of INTEGERs:
1901-12-13 20:45:52 (-2147483648) to 2038-01-19 03:14:07 (2147483647)
Can be changed to use BIGINT instead of INTEGER
20101211 initial version - Dieter Noeth
**********/
REPLACE FUNCTION TimeStamp_to_UnixTime (ts TimeStamp(6))
RETURNS INTEGER
LANGUAGE SQL
CONTAINS SQL
DETERMINISTIC
SQL SECURITY DEFINER
COLLATION INVOKER
INLINE TYPE 1
RETURN
(CAST(ts AS DATE) - DATE '1970-01-01') * 86400
+ (EXTRACT(HOUR FROM ts) * 3600)
+ (EXTRACT(MINUTE FROM ts) * 60)
+ (EXTRACT(SECOND FROM ts))
;
SELECT
TimeStamp_to_UnixTime(TIMESTAMP '1901-12-13 20:45:52')
,TimeStamp_to_UnixTime(CURRENT_TIMESTAMP)
,TimeStamp_to_UnixTime(TIMESTAMP '2038-01-19 03:14:07')
;
I'm trying to store in timestamp with timezone field my value. It is in milliseconds from 1970.
select TO_CHAR(TO_TIMESTAMP(1401432881230), 'DD/MM/YYYY HH24:MI:SS.MS')
Expected 30/5/2014 11:29:42 10:54:41.230, but get
22/08/46379 23:27:02.000
Unix timestamps measures time with seconds, and not milliseconds (almost everywhere, in PostgreSQL too).
Therefore you need to call
SELECT TO_TIMESTAMP(1401432881230 / 1000);
If you want to preserve milliseconds, call with double precision:
SELECT TO_TIMESTAMP(1401432881230::double precision / 1000);
This is how I convert ms to timestamp and keep ms instead seconds.The accepted answer will drop ms.
WITH ts AS (SELECT 1401432881230 AS ts)
SELECT to_timestamp(ts / 1000) + ((ts % 1000 ) || ' milliseconds') :: INTERVAL
FROM ts;
-- FOR ALTER COLUMN
ALTER TABLE my_info
ALTER COLUMN tstmp TYPE TIMESTAMP USING to_timestamp(tstmp / 1000) + ((tstmp % 1000) || ' milliseconds') :: INTERVAL;
Okay, I understood. My INSERT should looks like:
INSERT INTO events (timestamp) VALUES (to_timestamp(TO_CHAR(TO_TIMESTAMP(1401432881222 / 1000), 'YYYY-MM-DD HH24:MI:SS') || '.' || (1401432881222%1000), 'YYYY-MM-DD HH24:MI:SS.MS'))
I'm converting bigint-timestamp with milliseconds to text with required format ('YYYY-MM-DD HH24:MI:SS.MS') and pass it to to_timestamp function.
I have a sql query that is very slow:
select number1 from mytable
where symbol = 25
and timeframe = 1
and date::date = '2008-02-05'
and date::time='10:40:00' + INTERVAL '30 minutes'
The goal is to return one value, and postgresql takes 1.7 seconds to return the desired value(always a single value). I need to execute hundreds of those queries for one task, so this gets extremely slow.
Executing the same query, but pointing to the time directly without using interval and ::date, ::time takes only 17ms:
select number1 from mytable
where symbol = 25
and timeframe = 1
and date = '2008-02-05 11:10:00'
I thought it would be faster if I would not use ::date and ::time, but when I execute a query like:
select number1 from mytable
where symbol = 25
and timeframe = 1
and date = '2008-02-05 10:40:00' + interval '30 minutes'
I get a sql error (22007). I've experimented with different variations but I couldn't get interval to work without using ::date and ::time. Date/Time Functions on postgresql.org didn't help me out.
The table got a multi column index on symbol, timeframe, date.
Is there a fast way to execute the query with adding time, or a working syntax with interval where I do not have to use ::date and ::time? Or do I need to have a special index when using queries like these?
Postgresql version is 9.2.
Edit:
The format of the table is:
date = timestamp with time zone,
symbol, timeframe = numeric.
Edit 2:
Using
select open from ohlc_dukascopy_bid
where symbol = 25
and timeframe = 1
and date = timestamp '2008-02-05 10:40:00' + interval '30' minute
Explain shows:
"Index Scan using mcbidindex on mytable (cost=0.00..116.03 rows=1 width=7)"
" Index Cond: ((symbol = 25) AND (timeframe = 1) AND (date = '2008-02-05 11:10:00'::timestamp without time zone))"
Time is now considerably faster: 86ms on first run.
The first version will not use a (regular) index on the column named date.
You didn't provide much information, but assuming the column named date has the datatype timestamp (and not date), then the following should work:
and date = timestamp '2008-02-05 10:40:00' + interval '30 minutes'
this should use an index on the column named date (but only if it is in fact a timestamp not a date). It is essentially the same as yours, the only difference is the explicit timestamp literal (although Postgres should understand '2008-02-05 10:40:00' as a timestamp literal as well).
You will need to run an explain to find out if it's using an index.
And please: change the name of that column. It's bad practise to use a reserved word as an identifier, and it's a really horrible name, which doesn't say anything about what kind of information is stored in the column. Is it the "start date", the "end date", the "due date", ...?
I have a relation that maintains monthly historical data. This data is added to the table on the last day of each month. A service I am writing can then be called specifying a month and a number of months prior for which to retrieve the historical data. I am doing this by creating startDate and endDate variables, and then returning data between the two. The problem I am having is that startDate is a variable number of months before endDate, and I cannot figure out how to use a variable period in an interval.
Here is what I have:
DECLARE
endDate TIMESTAMP := (DATE_TRUNC('MONTH',$2) + INTERVAL '1 MONTH') - INTERVAL '1 DAY';
startDate TIMESTAMP := endDate - INTERVAL $3 'MONTH';
I know that the line for startDate is not correct. How is this properly done?
Use this line:
startDate TIMESTAMP := endDate - ($3 || ' MONTH')::INTERVAL;
and note the space before MONTH.
Basically: You construct a string with like 4 MONTH and cast it with ::type into a proper interval.
Edit: I' have found another solution: You can calculate with interval like this:
startDate TIMESTAMP := endDate - $3 * INTERVAL '1 MONTH';
This looks a little bit nicer to me.
This code has nothing directly to do with your situation, but it does illustrate how to use variables in INTERVAL arithmetic. My table's name is "calendar".
CREATE OR REPLACE FUNCTION test_param(num_months integer)
RETURNS SETOF calendar AS
$BODY$
select * from calendar
where cal_date <= '2008-12-31 00:00:00'
and cal_date > date '2008-12-31' - ($1 || ' month')::interval;
$BODY$
LANGUAGE sql VOLATILE
COST 100
ROWS 1000;
The most readable way I have found to pass a variable time period to Postgres is similar to A.H.'s answer: by multiplying by an integer. But this can be done without a cast.
Python example (with sqlalchemy and pandas):
import pandas as pd
import sqlalchemy as sa
connection = sa.create_engine(connection_string)
df = pd.read_sql(
sa.text('''
select * from events
where
event_date between now() - (interval '1 day' * :ndays) and now()
limit 100;
'''),
connection,
params={'ndays': 100}
)
The number of days (ndays) is passed as an integer from within Python - so unintended consequences are less likely.
My approach is like this.. It gives me option to set specific date or a relative range.
create or replace function search_data(_time_from timestamptz default null, _last_interval text default null)
returns setof journal
language plpgsql as
$$
begin
return query
select *
from journal
where created >= case
when _time_from is not null
then _time_from
else now() - _last_interval::interval end;
end;
$$;
While the above accepted answer is fine, it's a little bit antiquated - requiring a bit more mental energy to read than needed if you're running on Postgres 9.4+.
Old Way (Postgres Versions < 9.4)
startDate TIMESTAMP := endDate - $3 * INTERVAL '1 MONTH';
New Way (Postgres 9.4+)
startDate TIMESTAMP := endDate - MAKE_INTERVAL(MONTHS => $3);
If you are on Postgres 9.4+, the new MAKE_INTERVAL() function seems much more readable - probably why they created it.
If you want something you can run in your editor, here are a couple of
examples (I substituted the original variable binding $3 with the number 2 for an example of 2-months prior to the current date).
SELECT CURRENT_DATE - 2 * INTERVAL '1 MONTH';
SELECT CURRENT_DATE - MAKE_INTERVAL(MONTHS => 2);