I have a dataframe like:
id state city time
123.04 ny 1 01-10-2021 12:30
123.05 ny 2 01-10-2021 12:30
I want the the id that that is associated with the most recent time by state. So I do:
select id, state
from data a
join (select state, max(time) as most_recent
from data group by 1) b on a on a.state = b.state and a.time = b.most_recent)
However, I am running into issues where the timestamp is the same. I know that I can do another query to then get the max ID but I would ideally like to just go by timestamp. I know that the ID is assigned in sequential order so if I am able to get the seconds or milliseconds then I will be able to actually get the most recent ID.
Is there way to get seconds/milliseconds or do I have to do another query?
Redshift timestamps have a 1 microsecond resolution - this is part of the data type definition. If you are not seeing the seconds and fractional seconds it is because of how your bench (/ connection to RS) is presenting the data to you. The easiest way to see the fractional seconds is to format the timestamp to a string when viewing and then there will not be any reformatting. For example:
select to_char(sysdate, 'HH24:MI:SS.US');
Related
Say you've got a table ordered by the date that captures the speed of vehicles with a device in them. And imagine you get 30 updates per day for the speed. It's not always 30 per vehicle. The data will have the vehicle, the timestamp, and the speed.
What I want to do is be able to count how many days have passed since the vehicle last went over 10 mph in order to find inactive vehicles. Is something like that possible in postgresql?
*Or is there a way to get back the row number of the table if it's sorted where the speed goes past 10, and then select the date in that row number to subtract the current date from the date listed?
SELECT DISTINCT ON (vessel) vessel, now() - date
FROM your_table
WHERE speed > 10
ORDER BY vessel, date DESC
This will tell you, for every vehicle, how long ago its speed field was last over 10.
SELECT vessel, now() - max(date)
WHERE speed > 10
FROM your_table
GROUP BY vessel;
I am trying to subtract 1 millisecond from the update time (existing timestamp in dd-MMM-yy hh.mm.ss.MS format but I am getting null when I write -1. Please help. Below is my query in which I need to subtract 1 from the result I get by the lead of update time.
I tried this
nvl( lead(updatetime) over (partition by id order by updatetime asc)-1, now()) DW_END_DATE_TIME
Does not work. I am new to hive, I have no idea why this is not working.
I have a database with price and timestamps in nanoseconds measurement in InfluxDB. When I do a select grouped by time like this one:
select first(price),last(price) from priceseries where time>=1496815212834974866 and time<=1496865599580302882 group by time(1s)
I received a time column in which the timestamps is aligned to the second beginning the group. For example, the timestamp will be 08:00:00 and the next timestamps will be 08:00:01
How to
apply aggregation function on the record timestamps itself like last(time) or first(time) so that to have the real first and last timestamps of the group (I can have many prices within my group) ?
and how the time column in the response could be the closing second and not the opening second, that is if the group goes from 08:00:00 to 08:00:01, I want to see 08:00:01 in my time column instead of 08:00:00 which I see now ?
Not when using an aggregation function, which implies use of group by.
select first(price), last(price) where time >= <..> and time <= <..> will give you the first and last price within that time window.
When the query has a group by, the aggregation applies only to values within the intervals. The values themselves are the real values that fall in the 08:00:00 - 08:00:01 interval, it's just that the timestamp shown is for the interval itself, not the actual values.
Meaning that the query for between 08:00:00 and 08:00:01 without a group by and the query with a group by time(1s) for same period will give same result. Only difference is query without group by will have the value's actual timestamp and the group by query will have the interval's timestamp instead.
The timestamp when using group by indicates the starting time of the interval. From that, you can calculate end time is start time + interval. What timestamp to show is not configurable in the query language.
We're collecting lots of sensor data and logging them to a postgres DB.
Basic schema - cut down:
id | BIGINT PK
sensor-id| INT FK
location-id | INT FK
sensor-value | NUMERIC(0,2)
last-updated | TIMESTAMP_WITH_TIMEZONE
I'm trying to get the biggest change in sensor data in the last day. By that I mean, out of all the sensors, sensor ids 4,5,6,7 changed the biggest compared to the previous day. Before that, I'm trying to get a SQL query to figure out the delta between last reading and latest reading.
I thought maybe the lead and lag functions would help, but my query doesn't quite give me the result I was after:
SELECT
srd.last_updated,
spi.title,
lead(srd.value) OVER (ORDER BY srd.sensor_id DESC) as prev,
lag(srd.value) OVER (ORDER BY srd.sensor_id DESC) as next
FROM
sensor_rt_data srd
join sensor_prod_info spi on srd.sensor_id = spi.id
where srd.last_updated >= NOW() - '1 day'::INTERVAL -- current_date - 1
ORDER BY
srd.last_updated DESC
Simple dataset - making this up now because i can't login to the DB right now:
id|sensor,location,value,updated
1|1,1,24,'2017-04-28 19:30'
2|1,1,22,'2017-04-27 19:30'
3|2,1,35,'2017-04-28 19:30'
4|2,1,33,'2017-04-28 08:30'
5|2,1,31,'2017-04-27 19:30'
6|1,1,25,'2017-04-26 19:30'
Forgetting the join (that's for the user-friendly sensor tag name field staff need and the location), how do I workout which sensor has reported the biggest change in temperature over a time-series when they're grouped by sensor-id?
I'd be expecting:
updated,sensor,prev,next
'2017-04-28 19:30',1,24,22
'2017-04-28 19:30',2,33,31
(then from that, I can subtract and order to workout the top 10 sensors that have changed)
I noticed that Postgres 9.6 has some other functions too but want to try get Lead/Lag working first.
Window function aren't a best fit for this kind of task. Try this:
select sensor, max(value)-min(value) as value_change
from sensordata
where updated>=?-'1 day'::interval
group by sensor
order by value_change desc
limit 1;
Not much use for indexes besides updated for this kind of query. It would be probably possible to use a specially crafted index if you would be looking the largest change for a calendar day instead of last 24 hours.
I want to get data from one date only, example: 2014-06-16
in CMIS reference I know that we can use = (equal) operator that I think the time must be precised.
The alternative that i thought is to do like below :
First:
SELECT * FROM cmis:document WHERE cmis:creationDate >= TIMESTAMP '2014-06-16T00:00:00.000Z' AND cmis:creationDate< TIMESTAMP '2014-06-17T00:00:00.000Z'
Second:
SELECT P.tsi:DATENUM as date_traitement, L.tsi:type as type, P.tsi:statut as statut
FROM tsi:lot AS L JOIN tsi:pli AS P ON L.cmis:name = P.tsi:lot
WHERE
(P.tsi:DATENUM >= TIMESTAMP '2014-06-16T00:00:00.000Z' AND P.tsi:DATENUM < TIMESTAMP '2014-06-17T00:00:00.000Z')
The first one is running perfectly, I've got data from the 16 june BUT in the seconde I don't know WHY but I still got data from 2014-06-17
Note: tsi:DATENUM type is datetime
So could you say what's wrong OR how to get data from ONE date only?
The second one should work. The timestamps you are using are in GMT. If your timestamps are stored with a time zone offset it could be the reason why you are seeing times from 6/17 when you expect to only see times from 6/16.