I am new to hive.
I have column in one of my seed tables, say seed_timestamp.
example:
seed.timestamp = '28/04/2020 12:30:54', from this time stamp I want to create a new timestamp such that new_timestamp = '28/04/2020 00:00:00'.
I want to use these timestamps in my where clause of the query such that I check the data from midnight to the seed_timestamp.
In Hive, you can use to_date() to truncate the time part of a timestamp, so just:
to_date(seed_timestamp)
From the documentation:
to_date(string timestamp): returns the date part of a timestamp string
Concatenate date with ' 00:00:00.0':
concat(to_date(seed_timestamp),' 00:00:00.0')
It will produce string compatible with timestamp and you can compare. You can also cast it as timestamp:
cast(concat(to_date(seed_timestamp),' 00:00:00.0') as timestamp)
but it should work without it.
Related
I am using below condition to truncate date in postgres
to_date(to_char(trunc(appointment_date),'YYYYMMDD')||appointment_end_time,''YYYYMMDDHH24:MI:SS')AS tq
How I can use this in postgres ?
Strange data typing, sometimes requires strange, looking at least, queries. Try (see fiddle)
date_trunc('day',appointment_date)
+ substr(appoinment_end,12)::interval
As your to_char() call uses the format 'HH24:MI:SS' for the "time" column, you can cast that column directly to a time value, e.g. using the :: operator: appointment_end_time::time.
To build a new timestamp from the date part of the appointment_date and the time value, just add them:
appointment_date::date + appointment_end_time::time
So first the timestamp is converted to a date (that does not have a time), and then the time value is added to that, which yields a timestamp.
Note that to_date() returns a date so your code would remove the just added time part again. You would need to use to_timestamp() if you really want a timestamp as the result.
To answer the question's title "how to truncate date in Postgres?" (which in reality refers to a timestamp not a date): you can either cast it to a date (see above) or you can use date_trunc() (not trunc()) with a unit to which it should be truncated. However, date_trunc returns a timestamp not a date value, so you couldn't add a time to the result.
I have a column in Timestamp format that includes milliseconds.
I would like to reformat my timestamp column so that it does not include milliseconds. For example if my Timestamp column has values like 2019-11-20T12:23:13.324+0000, I would like my reformatted Timestamp column to have values of 2019-11-20T12:23:13
Is there a straight forward way to perform this operation in spark-scala? I have found lots of posts on converting string to timestamp but not for changing the format of a timestamp.
You can try trunc.
See more examples: https://sparkbyexamples.com/spark/spark-date-functions-truncate-date-time/
I have a timestamp with timezone column in one of my tables. I need to extract both hours and minutes from the timestamp with timezone column using extract function but i am unable too.
I tried like this,
extract(hour_minute from immi_referral_user_tb.completed_time) >= '06:30'
but I am getting a syntax error.
I am using extract function in where clause
immi_referral_user_tb.completed_time = timestamp with timezone column
Is there any other way too accomplish this?
You can cast the column to a time data type and compare that to a time value:
immi_referral_user_tb.completed_time::time >= time '06:30'
I'm in a tricky situation. A client's database timezone was configured for America/Chicago instead of UTC.
From the app, we ask customers to enter useful dates, and sadly those dates are stored 'as-is', so if they entered '2001-01-01 00:00:00' in the text input, that same value will be stored in the DB, and we are ignoring the customer's timezone. We save that info separately.
The table column is of type TIMEZONETZ. So Postgresql will add the America/Chicago timezone offset at the end: Eg '2001-01-01 00:00:00-02'.
Naturally, most of the customers are not in Chicago.
The difficult part, is that, even knowing the customer's timezone, it's really hard to run calculations on the DB given that the datetime was not correctly pre-processed before storing it into the DB.
My attempted solution, is finding a way to extract the datetime string from the column value, and re-convert it to a date with the right timezone. Eg (pseudo-code):
// This is psuedo code
SELECT DATETIME((SELECT date_string(mycolumn) FROM mytable),
TIMEZONE('America/Managua'));
Which would be equivalent in PHP:
$customerInput = '2016-01-01 00:00:00';
$format = 'Y-m-d H:i:s';
$wrongDateStoredInDb = DateTime::createFromFormat($format, $customerInput, new DateTimezone('America/Chicago'));
// In order to fix that date, I'd extract the dateString and create a new DateTime but passing the correct timezone info.
$customerTimezone = new Timezone('America/Bogota');
$customerInput = $wrongDateStoredInDb->format($format); // Assuming we didn't have it already.
$actualDateTime = DateTime::createFromFormat($format, $customerInput, $customerTimezone);
With that kind of information, I'd be able to run calculations on date ranges, with the correct values, eg:
// Pseudo-code
SELECT * FROM myTable WHERE fix_date_time(columnWithInvalidDate, `correctTimezone`)::timestamp > `sometimestamp`;
I've read the Postgresql docs, and I've tried everything I could, but nothing seems to work.
Any suggestion is more than welcomed!
So you say that you have a timestamptz column. That is not stored as a string, but as an "instant" in microseconds since the epoch. But when you do an INSERT and give a string, Postgres converts your string into a time value automatically, before storing it. It's been assuming the strings you give it are in Chicago time, since that's the default timezone.
Now you want to reinterpret those times as being in the user's time zone instead. To do that, you can put them back into strings (in Chicago time), and then parse them again but with a different time zone.
Suppose you have data like this:
CREATE TABLE t (id int primary key, ts timestamptz, tz text);
SET TIMEZONE='America/Chicago';
INSERT INTO t
VALUES
(1, '2015-01-01 12:00:00', 'America/Managua'),
(2, '2015-01-01 12:00:00', 'America/Los_Angeles')
;
Then this will give you new times that are what the user really meant:
SET TIMEZONE='America/Chicago';
SELECT ts::text::timestamp AT TIME ZONE tz FROM t;
To break it down: ts::text stringifies the value into Chicago time, then we re-parse it, but into a bare timestamp with no timezone information yet. Then we attach a time zone---not Chicago time, but the user's own timezone.
From here you should be able to handle fixing the bad rows (and not the new ones, if you've already changed the server's default timezone).
One caveat is that if a user entered a time and then changed their timezone, there is no way to recover that, so this will interpret that old time incorrectly.
Please pardon my ignorance if I have missed any documentation/solution for the same. But I searched the web and could not find an answer.
I have a simple question. In the DB2 table,I have a column of type date and the with data of format 04/25/2013 12:00:00AM . When I query the DB2 database, I want to obtain just the date and not the timestamp i.e to obtain "04/25/2013" and not "04/25/2013 12:00:00AM". I tried DATE(column name) and just gave back the complete value including the time stamp.
This looks like a TIMESTAMP and not a DATE column. If it is indeed a TIMESTAMP column try this:
select varchar_format(current timestamp, 'MM/DD/YYYY') from sysibm.sysdummy1 ;
Just replace the current timestamp in the above example with your column and sysibm.sysdummy1 with your table.
The good thing about varchar_format is that it lets you easily format the timestamp. Just change the 'MM/DD/YYYY' part to 'YYYY.MM.DD' to get a format like '2017.08.18'.