I am converting an Unix script with a SQL transact command to a PostgreSQL command.
I have a table with records that have a field last_update_time(xtime) and I want to select every record in the table that has been updated within a selected period.
Say, the current time it 05/01/2012 10:00:00 and the selected time is 04/01/2012 23:55:00. How do I select all the records from a table that have been updated between these dates. I have converted the 2 times to seconds in the Unix script prior to issuing the psql command, and have calculated the interval in seconds between the 2 periods.
I thought something like
SELECT A,B,C FROM table
WHERE xtime BETWEEN now() - interval '$selectedtimeParm(in secs)' AND now();
I am having trouble evaluating the Parm for the selectedtimeParm - it doesn't resolve properly.
Editor's note: I did not change the inaccurate use of the terms period, time frame, time and date for the datetime type timestamp because I discuss that in my answer.
What's wrong with:
SELECT a,b,c
FROM table
WHERE xtime BETWEEN '2012-04-01 23:55:00'::timestamp
AND now()::timestamp;
If you want to operate with a count of seconds as interval:
...
WHERE xtime BETWEEN now()::timestamp - (interval '1s') * $selectedtimeParm
AND now()::timestamp;
Note the standard ISO 8601 date format YYYY-MM-DD h24:mi:ss which is unambiguous with any locale or DateStyle setting.
The first value for the BETWEEN construct must be the smaller one. If you don't know which value is smaller use BETWEEN SYMMETRIC instead.
In your question you refer to the datetime type timestamp as "date", "time" and "period". In the title you used the term "time frames", which I changed to "timestamps". All of these terms are wrong. Freely interchanging them makes the question even harder to understand.
That, and the fact that you only tagged the question psql (the problem hardly concerns the command line terminal) might help to explain why nobody answered for days. Normally, it's a matter of minutes around here. I had a hard time understanding your question.
Understand the data types date, interval, time and timestamp - with or without time zone. Start by reading the chapter "Date/Time Types" in the manual.
Error message would have gone a long way, too.
For anyone who is looking for the fix to this. You need to remove timestamp from the where clause and use BETWEEN!
TABLENAME.COL-NAME-FOR-TIMESTAMP BETWEEN '2020-01-29 04:18:00-06' AND CURRENT_TIMESTAMP
Related
I want to get truncked data over the last month. My time is in unix timestamps and I need to get data from last 30 days for each specific day.
The data is in the following form:
{
"id":"648637",
"exchange_name":"BYBIT",
"exchange_icon_url":"https://cryptowisdom.com.au/wp-content/uploads/2022/01/Bybit-colored-logo.png",
"trade_time":"1675262081986",
"price_in_quote_asset":23057.5,
"price_in_usd":1,
"trade_value":60180.075,
"base_asset_icon":"https://assets.coingecko.com/coins/images/1/large/bitcoin.png?1547033579",
"qty":2.61,
"quoteqty":60180.075,
"is_buyer_maker":true,
"pair":"BTCUSDT",
"base_asset_trade":"BTC",
"quote_asset_trade":"USDT"
}
I need to truncate data based on trade_time
How do I write the query?
The secret sauce is the date_trunc function, which takes a timestamp with time zone and truncates it to a specific precision (hour, day, week, etc). You can then group based on this value.
In your case we need to convert these unix timestamps javascript style timestamps to timestamp with time zone first, which we can do with to_timestamp, but it's still a fairly simple query.
SELECT
date_trunc('day', to_timestamp(trade_time / 1000.0)),
COUNT(1)
FROM pings_raw
GROUP BY date_trunc('day', to_timestamp(trade_time / 1000.0))
Another approach would be to leave everything as numbers, which might be marginally faster, though I find it less readable
SELECT
(trade_time/(1000*60*60*24))::int * (1000*60*60*24),
COUNT(1)
FROM pings_raw
GROUP BY (trade_time/(1000*60*60*24))::int
For the query:
SELECT '2018-03-01'::TIMESTAMP - '2018-09-01'::TIMESTAMP,
'2018-03-01'::TIMESTAMPTZ - '2018-09-01'::TIMESTAMPTZ;
the values -184 days for TIMESTAMP and -183 days -23:00:00 for TIMESTAMPTZ is returned.
To me, -184 days "seems" correct but I was always told to use TIMESTAMPTZ. Which is the correct answer?
Answer posted as complement to comment discussion under OP's question.
Data type timestamptz is actually timestamp with time zone, while timestamp is timestamp without time zone.
The result that you get from substracting these specific dates like below is correct with 1 hour difference
SELECT '2018-03-01'::TIMESTAMPTZ - '2018-09-01'::TIMESTAMPTZ;
this is because 2018-03-01::timestamptz has +01 whereas 2018-09-01 has +02.
See it for yourself with this query
SELECT '2018-03-01'::TIMESTAMPTZ, '2018-09-01'::TIMESTAMPTZ
1 hour difference most likely comes from daylight savings and changing timezone over time.
In general I would always choose to include time zones within my system to properly handle time differences across the world and avoid issues that may arise from not having this. Client and server may have different timezones which can lead to problems.
I have dates in a postgres database. The problem is they are stored in a string field and have values similar to: "1187222400000" (which would correspond to 07.08.2007).
I would like to convert them into readable dates usind some SQL to_date() expression or something similar but can't come up with the correct syntax to make it work.
There really isn't enough information here for a conclusion, so I propose this 'scientific-wild-ass-guess' to resolve your puzzle. :)
It appears this number is UNIX 'epoch time' in milliseconds. I'll show this example as if your string field had the arbitrary name, 'epoch_milli'. In postgresql you can convert it to a time stamp using this statement:
SELECT TIMESTAMP WITH TIME ZONE 'epoch' + epoch_milli * INTERVAL '1 millisecond';
or using this built-in postgresql function:
SELECT to_timestamp(epoch_milli / 1000)
either of which, for the example '1187222400000', produces the result
"2007-08-15 17:00:00-07"
You can do some of your own sleuthing with quite a few values selected similarly to this:
SELECT to_timestamp(epoch_milli/1000)::DATE
FROM (VALUES (1187222400000),(1194122400000)) AS val(epoch_milli);
"Well, bollocks, man. I just want the date." Point taken.
Simply cast the timestamp to a date to discard the excess bits:
SELECT to_timestamp(epoch_milli / 1000)::DATE
Of course its possible that this value is a conversion or is relative to some other value, hence the request for a second example data point.
I'm interested in knowing the different possibilities to operate with datetimes in SQLite and understand its pros and cons. I did not find anywhere a detailed explanation of all the alternatives.
So far I have learned that
SQLite doesn't actually have a native storage class for timestamps /
dates. It stores these values as NUMERIC or TEXT typed values
depending on input format. Date manipulation is done using the builtin
date/time functions, which know how to convert inputs from the other
formats.
(quoted from here)
When any operation between datetimes is needed, I have seen two different approaches:
julianday function
SELECT julianday(OneDatetime) - julianday(AnotherDatetime) FROM MyTable;
Number of days is returned, but this can be fractional.
Therefore, you can also get some other measures of time with some extra operations. For instance, to get minutes:
SELECT CAST ((
julianday(OneDatetime) - julianday(AnotherDatetime)
) * 24 * 60 AS INTEGER)
Apparently julianday could cause some problems:
Bear in mind that julianday returns the (fractional) number of 'days'
- i.e. 24hour periods, since noon UTC on the origin date. That's usually not what you need, unless you happen to live 12 hours west of
Greenwich. E.g. if you live in London, this morning is on the same
julianday as yesterday afternoon.
More information in this post.
strftime function
SELECT strftime("%s", OneDatetime)-strftime("%s", AnotherDatetime) FROM MyTable;
Number of seconds is returned. Similarly, you can also get some other measures of time with some extra operations. For instance, to get minutes:
SELECT (strftime("%s", OneDatetime)-strftime("%s", AnotherDatetime))/60 FROM MyTable;
More information here.
My conclusion so far is: julianday seems easier to use, but can cause some problems. strftime seems more verbose, but also safer. Both of them provide only as results a single unit (either days or hours or minutes or seconds), but not a combination of many.
Question
1) Is there any other possibility to operate with datetimes?
2) What would be the best way to get directly the difference of two datetimes in time format (or date or datetime), where datetime would be formatted as 'YYYY-mm-dd HH:MM:SS', and the result would be something in the same format?
I would have imagined that something like the following would work, but it does not:
SELECT DATETIME('2016-11-04 08:05:00') - DATETIME('2016-11-04 07:00:00') FROM MyTable;
> 01:05:00
Julian day numbers are perfectly safe when computing differences.
The only problem would be if you tried to convert them into a date by truncating any fractional digits; this would result in noon, not midnight. (The same could happen if you tried to store them in integer variables.) But that is not what you're doing here.
SQLite has no built-in function to compute date/time differences; you have to convert date/time values into some number first. Whether you use (Julian) days or seconds does not really matter from a technical point of view; use whatever is easier in your program.
If you started with a different format, you might want to convert the resulting difference back into that format, e.g.:
time(difference_value, 'unixepoch') -- from seconds to hh:mm:ss
time(0.5 + difference_value) -- from Julian days to hh:mm:ss
I have two data base in two differents machines with the same schemas, tables and data. I launch this query:
select mydate from mytable where date = '2013-10-03 14:25:00-07'::timestamp::date
the first machine return the correct rows and the second one doesn´t, both machines has the same prostgres version (9.2)
the only different between the machines is that first one works on windows and the second one on Linux (Centos)
Any suggestion?
'2013-10-03' can be interpreted as Oct, 3rd or March, 10th, depending on your datestyle setting. #Chris has more on that.
In addition to that, your query is generally incorrect. This expression is misleading:
'2013-10-03 14:25:00-07'::timestamp
timestamp in Postgres defaults to timestamp without time zone, which doesn't recognize time zone offsets. Therefore, the time zone offset -07 is discarded.
Use instead:
'2013-10-03 14:25:00-07'::timestamptz
Match the point in time:
SELECT * FROM mytable
WHERE mydate = '2013-10-03 14:25:00-07'::timestamptz
Does not depend on your local time zone setting, since the data type of the column is timestamp with time zone as you clarified in a later comment.
Match the day:
...
WHERE mydate::date = '2013-10-03 14:25:00-07'::timestamptz::date
Depends on your local time zone setting, which defines lower and upper borders of the "day".
Detailed explanation in this related answer:
Ignoring timezones altogether in Rails and PostgreSQL
The cleanest solution would be to, at the beginning of the session, just issue the following command:
SET datestyle = "ISO, YMD";
This will ensure properly handling the timestamp according to your input format.