I'm a bit of newbie when it comes to postgres, so bear with me a wee bit and i'll see if i can put up enough information.
i insert weather data into a table every 10 mins, i have a time column that is stamped with an epoch date.
I Have a column of the last hrs rain fall, and every hr that number changes of course with the running total (for that hour).
What i would like to do is skim through the rows to the end of each hour, and get that row, but do it over the last 4 hours, so i would only be returning 4 rows say.
Is this possible in 1 query? Or should i do multiple queries?
I would like to do this in 1 query but not fussed...
Thanks
Thanks guys for your answers, i was/am a bit confused by yours gavin - sorry:) comes from not knowing this terribly well.
I'm still a bit unsure about this, so i'll try and explain it a bit better..
I have a c program that inserts data into the database every 10 mins, it reads the data fom a device that keeps the last hrs rain fall, so every 10 mins it could go up by x amount.
So i guess i have 6 rows / hr of data.
My plan was to go back (in my php page) every 7, which would be the last entry for every hour, and just grab that value. Hence why i would only ever need 4 rows.. just spaced out a bit!
My table (readings) has data like this
index | time (text) | last hrs rain fall (text)
1 | 1316069402 | 1.2
All ears to better ways of storing it too :) I very much appreciate your help too guys thanks.
You should be able to do it in one query...
Would something along the lines of:
SELECT various_columns,
the_hour,
SUM ( column_to_be_summed )
FROM ( SELECT various_columns,
column_to_be_summed,
extract ( hour FROM TIME ) AS the_hour
FROM readings
WHERE TIME > ( NOW() - INTERVAL '4 hour' ) ) a
GROUP BY various_columns,
the_hour ;
do what you need?
SELECT SUM(rainfall) FROM weatherdata WHERE time > (NOW() - INTERVAL '4 hour' );
I don't know column names but that should do it the ones in caps are pgsql types. Is that what you are after?
I am not sure if this is exactly what you are looking for but perhaps it may serve as a basis for adaptation.
I often have a requirment for producing summary data over time periods though I don't use epoch time so there may be better ways of manipulating the values than I have come up with.
create and populate test table
create table epoch_t(etime numeric);
insert into epoch_t
select extract(epoch from generate_series(now(),now() - interval '6 hours',interval '-10 minutes'));
To divide up time into period buckets:
select generate_series(to_char(now(),'yyyy-mm-dd hh24:00:00')::timestamptz,
to_char(now(),'yyyy-mm-dd hh24:00:00')::timestamptz - interval '4 hours',
interval '-1 hour');
Convert epoch time to postgres timestamp:
select timestamptz 'epoch' + etime * '1 second'::interval from epoch_t;
then truncate to hour :
select to_char(timestamptz 'epoch' + etime * '1 second'::interval,
'yyyy-mm-dd hh24:00:00')::timestamptz from epoch_t
To provide summary information by hour :
select to_char(timestamptz 'epoch' + etime * '1 second'::interval,
'yyyy-mm-dd hh24:00:00')::timestamptz,
count(*)
from epoch_t
group by 1
order by 1 desc;
If you might have gaps in the data but need to report zero results use a generate_series to create period buckets and left join to data table.
In this case I create sample hour buckets back prior to the data population above - 9 hours instead of 6 and join on the conversion of epoch time to timestamp truncated to hour.
select per.sample_hour,
sum(case etime is null when true then 0 else 1 end) as etcount
from (select generate_series(to_char(now(),
'yyyy-mm-dd hh24:00:00')::timestamptz,
to_char(now(),'yyyy-mm-dd hh24:00:00')::timestamptz - interval '9 hours',
interval '-1 hour') as sample_hour) as per
left join epoch_t on to_char(timestamptz 'epoch' + etime * '1 second'::interval,
'yyyy-mm-dd hh24:00:00')::timestamptz = per.sample_hour
group by per.sample_hour
order by per.sample_hour desc;
Related
I'm querying the database (RedShift) and I have a piece of information stored in epoch MS format. Envision a table along the lines of:
Purchase, date
1, 1620140227019
2, 1620140227045
3, 1620140226573
I need to convert the timestamp to a readable date but I can't make it work with to_timestamp() or extract(). The problem is first with the size of the value (13 digits are not supported).
The closest solution I have is
select to_timestamp(1620140226573/1000, 'SS')
But the result is 0051-05-04 14:57:06. In other words month, date and seconds are correct but the year is wrong.
You can run this query
select to_timestamp(round(1620140227254/1000))
The solution was in the documentation: https://docs.aws.amazon.com/redshift/latest/dg/r_Dateparts_for_datetime_functions.html
SELECT timestamp with time zone 'epoch' + 1620140227019/1000 * interval '1 second' AS converted_timestamp
or
select '1970-01-01'::date + 1620140227019/1000 * interval '1 second'
Suppose I have data as shown in image
I want to create a third column that will give me names of different types of Alarm names occurred in 20 minutes' time from the Alarm name column, So I can understand which Alarms are related.
I am not sure I full understand the output you want, but you can collect the error messages using array_agg() and a window function that uses a window that is -10 minutes and +10 minutes around the "current" timestamp.
Something along the lines:
select created_at,
error_message,
array_agg(error_message) over (order by created_at range between interval '10 minute' preceding and '10 minute' following) as nearby_errors
from error_log
order by created_at;
I would like to get the total number of hours,minute, and seconds difference between two timestamp fields based on starting and ending time in postgres
Table supermarket has fields opening_time and closing_time
Table orders have the fields order_id, arrived_date, and picked_date
I want to calculate the total time spend to pick an order. (The orders can be picked only the supermarket is open but the orders will get placed and queued anytime in the day)
Condition: The total picking time should be calculated considering the opening and closing time of the store.
Example
Consider opening_time is 09:00:00 and closing_time is 22:00:00
Case 1: if an order arrives on 2020-09-08 10:00:00 and is picked at 2020-09-08 12:00:00, then the total picking time should be 02 hours
Case 2: if an order arrives on 2020-09-08 06:00:00 and is picked at 2020-09-08 12:00:00, then the total picking time should be 03 hours, not 06 hours considering the opening time
Case 3: if an order arrives on 2020-09-08 23:00:00 and is picked next day at 2020-09-09 10:00:00, then the total picking time should be 01 hour considering closing and opening time
It is a bit more complicated that it would seem to, especially if the ranges may expand over more than 24 hours. The safest solution approach might be a brute-force approach that generates all hours in the range, then filtering and aggregation:
select s.*, x.*
from supermarket s
cross join lateral (
select count(*) no_hours
from generate_series(s.opening_time, s.closing_time, '1 hour') x(x_time)
where x_time::time >= '09:00:00'::time and x_time::time < '22:00:00'::time
) x
This assumes that the opening and closing dates are truncated to the hour, as shown in your examples. If you want to handle minutes, then:
select s.*, x.*
from supermarket s
cross join lateral (
select count(*) no_minutes
from generate_series(s.opening_time, s.closing_time, '1 minute') x(x_time)
where x_time::time >= '09:00:00'::time and x_time::time < '22:00:00'::time
) x
Timestamps can simply be subtracted resulting in an interval.
test=# select '2020-09-08 12:00:00'::timestamp - '2020-09-08 10:00:00'::timestamp;
?column?
----------
02:00:00
(1 row)
That interval can then be formatted with to_char.
test=# select to_char('2020-09-08 12:00:00'::timestamp - '2020-09-08 10:00:00'::timestamp, 'HH24 hours MI "minutes"');
to_char
---------------------
02 hours 00 minutes
(1 row)
To get the correct interval considering opening and closing time, you need a little bit of logic to figure out the actual picking start time.
select
case
when arrived_date::time < opening_time
-- It arrived before you're open. Start when you open that day.
arrived_date::date + opening_time
when arrived_date::time > closing_time
-- It arrived after you closed. Start when you open tomorrow.
arrived_date::date + '1 day'::interval + opening_time
else
-- It arrived while you're open. Start when it arrives.
arrived_date
end as picking_start
The trick is to cast arrived_date to time, truncating the date part, to compare with the opening and closing times. Similarly, we can cast arrived_date to date and use just the date part, then add the opening time. This assumes that arrived_date is a timestamp and that opening_time and closing_time are time columns.
This could be condensed into a function for easy use.
with partial as(
select
date_part('week', activated_at) as weekly,
count(*) as count
from vendors
where activated_at notnull
group by weekly
)
This is the query counts number of vendors activating per week. I need to change the start day of week from Monday to Saturday. Similar posts like how to change the first day of the week in PostgreSQL or Making Postgres date_trunc() use a Sunday based week but non explain how to embed it in date_part function. I would like to know how to use this function in my query and start day from Saturday.
Thanks in advance.
maybe a little bit overkill for that, you can use some ctes and window functions, so first generate your intervals, start with your first saturday, you want e.g. 2018-01-06 00:00 and the last day you want 2018-12-31, then select your data, join it , sum it and as benefit you also get weeks with zero activations:
with temp_days as (
SELECT a as a ,
a + '7 days'::interval as e
FROM generate_series('2018-01-06 00:00'::timestamp,
'2018-12-31 00:00', '7 day') as a
),
temp_data as (
select
1 as counter,
vendors.activated_at
from vendors
where activated_at notnull
),
temp_order as
(
select *
from temp_days
left join temp_data on temp_data.activated_at between (temp_days.a) and (temp_days.e)
)
select
distinct on (temp_order.a)
temp_order.a,
temp_order.e,
coalesce(sum(temp_order.counter) over (partition by temp_order.a),0) as result
from temp_order
I'm using EXPIRY column as bigint storing epoch time. I need to find all those records which are going to expire in next 48 hours.
Something like this:
select * from bottles where current_epoch_time - EXPIRY < 48 hours
I also need to order the records so that the bottle which will expire first should be the first record and the bottle that will expire last should be the last record.
I hope I was able to make my question clear.
Thanks in advance
Here is one method:
select b.*
from bottles b
where EXPIRY >= extract(epoch from now()) and
EXPIRY < extract(epoch from (now() + interval '48 hours') )
Notice that all the functions and calculations are not on EXPIRY. This makes it easier for Postgres to use an index on that column -- speeding up the query.