what will be data type to store date time in cassandra
CREATE TABLE testTable (
dateValue date,
time timestamp
)
n my insert staements would be like this ,
insert into caliper.log_per_day ( timeStampValue,dateValue ) values ('2015-12-30 16:10:31','2015-12-30');
i wanted to store date & time both in one column like this '2015-12-30 16:10:31'.
but if i used timestamp it would be store like this '2015-12-30 04:10:31+0530'
Notes : primary key n other things are skip here... ignore it.
Cqlsh will display timestamps in the following format by default:
yyyy-mm-dd HH:mm:ssZ
The Z in these formats refers to an RFC-822 4-digit time zone,
If no time zone is supplied, the current time zone for the Cassandra
server node will be used.
so if you don't want to store in this way you can store it as varchar.
Related
We are storing date time in "2022-11-30T17:30:00.000Z" format in var char column i know its totally wrong but I want to rectify the mistake and use the column as timestamp with time zone.
I Tried using column::timestamp AT TIME ZONE 'Asia/Calcutta' but still date and time is wrong as it does not considers +5:30 in my time zone.please suggest how can i transform the column or if i could use the same column.
select dtm from dashboard where (select dtm::timestamp AT TIME ZONE 'Asia/Calcutta')::date between '2022-11-30' and '2022-11-30' order by dtm;
I have been struggling since few days to identify correct format for sample input date:
2020-01-30 14:39:25.022000 +00:00:00
Background :
I am working on data migration with postgres. And, have identified that in DB, the datetime related fields like 'modifiedAt' have datatype as varchar, which is bit weird. These records have values in above mentioned format.
I have fixed it in new postgres DB instance with datatype as timestamp. After migrating data, these fields are showing datetime in following format:
2020-01-30 14:39:25
How can I format above date to give output in this dateformat 2020-01-30 14:39:25.022000 +00:00:00?
Any help will be appreciated.
First you want to use timestamptz not timestamp. timestamptz does not actually store the time zone, it just makes the value time zone aware. For more information on this see:
Time Stamps 8.5.1.3. Time Stamps.
Second it looks like you did something like timestamp(0) which reduced the precision to whole seconds. If you leave the precision alone you get:
select '2020-01-30 14:39:25.022000 +00:00:00'::timestamptz;
timestamptz
-----------------------------
01/30/2020 06:39:25.022 PST
-- I am in PST so the value gets rotated to that time zone for display.
-- If you want something closer to you desired output, then:
select to_char('2020-01-30 14:39:25.022000 +00:00:00'::timestamptz AT TIME ZONE 'UTC', 'YYYY-MM-DD HH24:MI:SS.USOF');
to_char
-------------------------------
2020-01-30 14:39:25.022000+00
While your timestamp strings do appear to have a time zone component, it is +00, which is the default for timestamps. So, you might be able to just use TO_TIMESTAMP here with an appropriate format mask covering microseconds:
WITH yourTable AS (
SELECT '2020-01-30 14:39:25.022123 +00:00:00'::text AS ts
)
SELECT TO_TIMESTAMP(LEFT(ts, 26), 'YYYY-MM-DD HH24:MI:SS.US')
FROM yourTable;
Demo
The output value from the demo above is 2020-01-30 14:39:25.022123+00
Using psycopg to connect to a Redshift DB I am trying to group by date (in a specific time zone, given at query time). All date_times are stored without time zone (and correspond to UTC timestamps).
I have been trying the following:
SELECT DISTINCT DATE(TIME_ZONE(%(time_zone)s, date_time_dim.timestamp)) AS date,
SUM(meals.sugar_in_g) AS total_sugar
FROM meals
INNER JOIN date_time_dim
ON meals.created_at = date_time_dim.timestamp
WHERE meals.patient_id=%(patient_id)s
AND date_time_dim.timestamp >= %(utc_start_date_time)s
AND date_time_dim.timestamp <= %(utc_end_date_time)s
GROUP BY date
ORDER BY date ASC;
with the following query dictionary:
utc_start_date_time UTC-converted date time
utc_end_date_time UTC-converted date time
patient_id an integer
time_zone, a string, ex: 'US/Hawaii'
The goal being to have one entry for each date (in the given time zone).
What I tried gives me:
function time_zone("unknown", timestamp without time zone) does not exist
What am I missing ?
Edit: This is the same with CONVERT_TIME_ZONE, with and without source time_zone, with and without type casting time_zone to VARCHAR.
The function in Redshift to return a timestamp in a different timezone is TIMEZONE() not TIME_ZONE() - see https://docs.aws.amazon.com/redshift/latest/dg/Date_functions_header.html
We have a table that is filled with data from a legacy report of another system. The columns of that table reflect the same structure of the report.
Here are a abbreviated structure of the table:
CREATE TABLE IF NOT EXISTS LEGACY_TABLE (
REPORT_DATE DATE NOT NULL,
EVENT_ID BIGINT PRIMARY KEY NOT NULL,
START_HOUR TIMESTAMP WITHOUT TIME ZONE,
END_HOUR TIME WITHOUT TIME ZONE,
EXPECTED_HOUR TIME WITHOUT TIME ZONE
);
We are refactoring this table to deal with different time zones of different clients. The new structure would be something like:
CREATE TABLE IF NOT EXISTS LEGACY_TABLE (
REPORT_DATE DATE NOT NULL,
EVENT_ID BIGINT PRIMARY KEY NOT NULL,
START_HOUR TIMESTAMP WITH TIME ZONE,
END_HOUR TIME WITH TIME ZONE,
EXPECTED_HOUR TIME WITH TIME ZONE
);
These hour fields represents a specific point in time during the day represented by the REPORT_DATE column. What I mean by that is that every TIME column represents a moment during the day specified in REPORT_DATE.
Some other points to consider:
We don't know why the START_HOUR is in TIMESTAMP format in the report we receive from the legacy system. But we import the data the way it comes to us.
The fields in the report are formatted according to the timezone of the client, so to refactor this table we need to combine the timezone of the client (we have this info) to properly insert the timestamps/times in UTC.
But now to the problem. The value of these columns are used to compute another values multiple times in our system, something like the following:
START_HOUR - END_HOUR (the result of this operation is currently being casted to TIME WITHOUT TIME ZONE)
START_HOUR < END_HOUR
START_HOUR + EXPECTED_HOUR
EXPECTED_HOUR - END_HOUR
EXPECTED_HOUR < '05:00'
After some research I found that is not recommended to use the type TIME WITH TIME ZONE (Postgres time with time zone equality) and now I'm a bit confused about what is the best way to refactor this table to deal with different time zones and handle the different column operations that we need to.
Besides that, I already know that is safe to subtract two columns of type TIMESTAMP WITH TIME ZONE. This subtraction operation is taking into account DST changes (Subtracting two columns of type timestamp with time zone) but how about the others? And the one subtracting a TIME from a TIMESTAMP?.
And about the table refactoring, should we use TIME WITH TIME ZONE anyways? Should we continue using TIME WITHOUT TIME ZONE? Or is better to forget the type TIME altogether and combine the DATE with the TIME and change the columns to TIMESTAMP WITH TIME ZONE?
I think these questions are related because the new column types we choose to use, will define how we operate with the columns.
You asserted that:
every TIME column represents a moment during the day specified in REPORT_DATE.
So you never cross the a dateline within the same row. I suggest to save 1x date 3x time and the time zone (as text or FK column):
CREATE TABLE legacy_table (
event_id bigint PRIMARY KEY NOT NULL
, report_date date NOT NULL
, start_hour time
, end_hour time
, expected_hour time
, tz text -- time zone
);
Like you already found, timetz (time with time zone) should generally be avoided. It cannot deal with DST rules properly (daylight saving time).
So basically what you already had. Just drop the date component from start_hour, that's dead freight. Cast timestamp to time to cut off the date. Like: (timestamp '2018-03-25 1:00:00')::time
tz can be any string accepted by the AT TIME ZONE construct, but to deal with different time zones reliably, it's best to use time zone names exclusively. Any name you find in the system catalog pg_timezone_names.
To optimize storage, you could collect allowed time zone names in a small lookup table and replace tz text with tz_id int REFERENCES my_tz_table.
Two example rows with and without DST:
INSERT INTO legacy_table VALUES
(1, '2018-03-25', '1:00', '3:00', '2:00', 'Europe/Vienna') -- sadly, with DST
, (2, '2018-03-25', '1:00', '3:00', '2:00', 'Europe/Moscow'); -- Russians got rid of DST
For representation purposes or calculations you can do things like:
SELECT (report_date + start_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS start_utc
, (report_date + end_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS end_utc
, (report_date + expected_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS expected_utc
-- START_HOUR - END_HOUR
, (report_date + start_hour) AT TIME ZONE tz
- (report_date + end_hour) AT TIME ZONE tz AS start_minus_end
FROM legacy_table;
You might create one or more views to readily display strings as needed. The table is for storing the information you need.
Note the parentheses! Else the operator + would bind before AT TIME ZONE due to operator precedence.
And behold the results:
db<>fiddle here
Since the time is manipulated in Vienna (like any place where silly DST rules apply), you get "surprising" results.
Related:
Accounting for DST in Postgres, when selecting scheduled items
Ignoring time zones altogether in Rails and PostgreSQL
When inserting now() in a table, the written value only contains the date (e.g. 2017-12-20), but not the date and time as specified in the documentation.
See this SQLfiddle.
create table timetest (
id serial primary key,
mydate date
);
insert into timetest (mydate) values (
now());
Is there some specific command that should be passed to either write or retrieve also the time information?
You created mydate date as a date type column. date only represents the date fraction (unsurprisingly).
If you need both date and time use timestamp type instead.
References:
https://www.postgresql.org/docs/current/static/datatype-datetime.html