I have a weird dataset that I need to access in hive. Using traditional date/time functions (such as dateadd, etc) have proven difficult/ineffective.
There is a column in my dataset that is a string with the date in YYYY-MM-DD format.
I am wondering if it's possible to get the current date in YYYY-MM-DD format, and somehow cast it as a string?
I've done considerable amounts of research and have tried just about everything in the documentation (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions)
Any help here will be greatly appreciated, as I've been grinding away at this problem for a considerable amount of time :)
Thanks!
I didn't get how the date will be displayed for format YYYY-MM-DD, but you can try below.
hive> select cast(from_unixtime(unix_timestamp(cast(current_date() as string), 'yyyy-MM-dd'),'YYYY-MM-DD') as string);
OK
2018-09-271
Time taken: 0.114 seconds, Fetched: 1 row(s)
hive>
from_unixtime accepts string format and you can get all the options here - https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html
hope this helps.
VIJAY
Related
I have a time field which is storing numbers in '49235062'. Can these be converted to an actual readable time?
This does not look anything like a time stamp.
Thanks
Just a guess, but is this milliseconds from midnight?
Example
Select dateadd(MILLISECOND,49235062,0)
Returns
1900-01-01 13:40:35.063 -- 1:40 PM
If so, then it is a small matter to convert to time or format as time
Assuming that this is a UNIX timestamp (number of seconds since 1/1/1970), try the following:
DECLARE #timeStamp varchar(10) = '49235062'
SELECT #timeStamp, CONVERT(TIME, dateadd(S, CAST(#timeStamp AS int), '1970-01-01'))
Produces the following
49235062 20:24:22.0000000
So your time is 20:24:22 which is the answer that #pac0 suggested.
I m having some issues with converting a regular timestamp in cassandra with Apache Nifi.
My use case is following:
I have a csv file with a date in it looking like this ('2015010109') and I want to put it in cassandra by converting this string ('2015010109') to an proper format: 2015-01-01 09:00 -> yyyy/MM/dd HH:mm (I dont exactly need the minutes, but I guess it is more useful for later usage)
So far I got this propertie in my UpdateAttribute processor when trying to convert this string to a timestamp:
date : ${csvfiledate:toDate("yyyyMMddHH","GMT"):format("yyyy-MM-dd-HH")}
but then there is an error occuring in my PutCassandraQL processor: Unable to coerce '2015-01-01-09' to a formatted date (long).
I tried something along
date : ${csvfiledate:toDate("yyyyMMddHH","GMT"):format("yyyy-MM-dd HH-mmZ")} aswell, but the same error is occuring.
It seems like you need to have a specific timestamp type for cassandra as you can see here:
http://docs.datastax.com/en/archived/cql/3.0/cql/cql_reference/timestamp_type_r.html
But it isnt working so far, maybe you got some tipps.
Thanks in advance.
You've got it backwards..
toDate parameters are used to describe how to parse the date. format function is used to describe how the date output should be. So the expression should be:
${csvfiledate:toDate('yyyy-MM-dd-HH','GMT'):format('yyyyMMddHH')}
I have dates in a postgres database. The problem is they are stored in a string field and have values similar to: "1187222400000" (which would correspond to 07.08.2007).
I would like to convert them into readable dates usind some SQL to_date() expression or something similar but can't come up with the correct syntax to make it work.
There really isn't enough information here for a conclusion, so I propose this 'scientific-wild-ass-guess' to resolve your puzzle. :)
It appears this number is UNIX 'epoch time' in milliseconds. I'll show this example as if your string field had the arbitrary name, 'epoch_milli'. In postgresql you can convert it to a time stamp using this statement:
SELECT TIMESTAMP WITH TIME ZONE 'epoch' + epoch_milli * INTERVAL '1 millisecond';
or using this built-in postgresql function:
SELECT to_timestamp(epoch_milli / 1000)
either of which, for the example '1187222400000', produces the result
"2007-08-15 17:00:00-07"
You can do some of your own sleuthing with quite a few values selected similarly to this:
SELECT to_timestamp(epoch_milli/1000)::DATE
FROM (VALUES (1187222400000),(1194122400000)) AS val(epoch_milli);
"Well, bollocks, man. I just want the date." Point taken.
Simply cast the timestamp to a date to discard the excess bits:
SELECT to_timestamp(epoch_milli / 1000)::DATE
Of course its possible that this value is a conversion or is relative to some other value, hence the request for a second example data point.
I am selecting the data from a table using a date string. I would like to select all rows that have a update time stamp greater than or equal to today.
The simplest way that I can think of is to put today's date in the string, and it works fine.
WHERE UPDATE_DTM >'29NOV2016:12:00'DT;
However, if I want to put something like today's date or system date, what should I put?
I used today(), but it returned all rows in the table. I am not sure if it's because today() in SAS refers to the date 1/1/1960? I also tried &sysdate, but it returned an error message seems like it requires a date conversion.
WHERE UPDATE_DTM > TODAY();
Any ideas? Your thoughts are greatly appreciated!
DATETIME() is the datetime equivalent of TODAY() (but includes the current time). You could also use dhms(TODAY(),0,0,0) if you want effectively midnight (or, for your example above, dhms(TODAY(),12,0,0) to get noon today).
I have a date field set with table.date('day'); in knex schema. When I insert it with knex('table_name').insert({ someOtherData, day: '2016-08-14'}) and then use knex.select('day').from('table_name') I get [Date: 2016-08-13T22:00:00.000Z]. It seems as if it saves it as '2016-08-14T00:00:00.000Z' and then subtracts 2 hours to conver it into UTC.
This issue may be because of time zone conversion. Have you tried using timestamp?
table.timestamp('response_deadline')
It will convert date datatype to timestamp with time zone.
the docs on schema building seems vague however try to deliver this date string to a js date constructor, i'm pretty sure it will deliver you the correct date.
it tries to represent every date as the specs recommends, that's why you're seeing the date this way.