I need to calculate mean sales for sunday. Values for the column salesdate(timestamp) are:
2012-01-01 09:00:00
2012-01-01 09:00:00
2012-01-01 09:00:00
...........
I have extracted the date part using to_date().Now how to get weekday(like sunday) from this date in hive?
Please guide.
You can use a combination of unix_timestamp, and from_unixtime UDFs.
from_unixtime(unix_timestamp(col), 'EEEE')
If you check the documentation for SimpleDateFormat, which from_unixtime uses, you can see that "EEEE" is the code for the full name of the day of the week. "EEE" gets you the abbreviated version, i.e. "Sun" or "Mon".
There is no OOTB feature to achieve this as of now. A ticket is open though.
You need to write a UDF for this. Or, you could also try the patch available with the above mentioned ticket.
HTH
In Hive you can also use the below method to solve this problem in very elegant way and its performance is very good.
from_unixtime accepts the 1st argument in int format:
date_format(from_unixtime(col(timestampinseconds),'yyyy-MM-dd'),'EEEE')
You can also test it like this:
select date_format(from_unixtime(1531372789,'yyyy-MM-dd'),'EEEE');
Output:
Thursday
I hope it serves your purpose.
Just a suggestion.. you can take a low date (lower than the minimum date in your data), which is a Sunday, in 'yyyy-mm-dd' format. Use DATEDIFF() function to find the difference between the date value in your data (in 'yyyy-mm-dd' format) and this low date. Calculate modulo 7 of the datediff output. This will be 0 for Sunday, 1 for Monday, and so on..
select extract(dayofweek from from_unixtime(unix_timestamp));
Related
Is there any function to convert Normal Date to Julian Date.
I have used JulianDayFromDate function in transformer but i am not getting expected output .
Sample Input :
Date -- 2013-02-02
Output Should be:
Julian Date-- 113033
( In Database we can do the query as below )
select to_date(1900000+113033,'YYYYDDD') from dual
But how to convert in Datastage ... ?
Maybe your expectations are wrong -
2456326 is the julian day for 2013-02-02 - the DataStage functions works.
Check out the Wikipedia documentation for defintions and calculations
Not sure what your 113033 is but it is not the Julian date or Julian day for the date shown.
To achieve what you want you have to do the calculation by your own.
Besides the year calcuation you could use YeardayFromDate to get the daynumer in the year.
So finally it would be something like
YearFromDate('2013-02-02') * 1000 - 1900000 + YeardayFromDate('2013-02-02')
Databases use different "day zero" values for Julian dates. Doesn't matter. The point is the ability to do date (day) arithmetic with them.
To convert back, use DateFromJulianDay() function.
I have been looking for it during days but could not find how to do..
It is like:
select to_number(to_char('2015-06-24 00:00:00','J')) on Oracle.
I need to find the Julian Numeric Day value, not to be confused with the ordinal date of the year..
Conversion templates indicate that 'J' is exactly what you want.
I think the issue you have is the to_number() function, not the to_char() function. Use casts instead.
SYSTEM(ADMIN)=> select to_char('2015-06-24 00:00:00'::timestamp,'J')::int;
?COLUMN?
----------
2457198
(1 row)
You need to use the DDD (day of year) date identifier in TO_CHAR.
Reference: date and time constants.
I'm trying to find the best way to subtract 1 year and also one month from the current date in a Hive query. Using the following, I don't believe it will take into account leap years or if the fact that months have different amounts of days so eventually the code will break. Any help would be greatly appreciated!
set my_date = from_unixtime(unix_timestamp()-365*60*60*24, 'yyyy-MM-dd');
set my_date = from_unixtime(unix_timestamp()-30*60*60*24, 'yyyy-MM-dd');
Thank!
-Rebecca
If you have date format like yyyy-MM-dd hh:mm:ss in Hive, it is easy to implement using following functions
concat((year(date_field)-1),'-', (month(date_field)-1), '-', day(date_field))
Use IF and CASE functions to implement your logic to find whether it is a leap year or not(by dividing year by 4)
I have a Date field (CHAR Datatype) which has values in this format: YYYYMMDD.
For example 20140729.
I want to convert it into a Weeknumber in format YYYYWKNO
For example, the result would be 201432.
How this can be done in IBM DB2?
Luckily for you, DB2 has some pretty good date formatting functions. Although the links for the documentation are for DB2 for Linux/Unix/Windows, it will also work on DB2 for z/OS.
You can use TIMESTAMP_FORMAT() to convert your CHAR field to an actual date, which you can then use VARCHAR_FORMAT() to format it in the way you wish:
SELECT
VARCHAR_FORMAT(
TIMESTAMP_FORMAT(
'20140801'
,'YYYYMMDD'
)
,'YYYYWW'
)
FROM SYSIBM.SYSDUMMY1
There are two different formats for "week", one is WW, which will give the week based on a week beginning with January 1 and ending January 7, and IW, which will give the ISO Week.. Please see the documentation page for VARCHAR_FORMAT for the other formats available.
The following gets week 31 instead of 32. But overall I think it is a good solution for this problem:
SELECT TO_DATE('20140801', 'YYYYMMDD') AS MYDATE
, YEAR(TO_DATE('20140801', 'YYYYMMDD')) * 100 + WEEK(TO_DATE('20140801', 'YYYYMMDD')) AS YYYYWKNO
FROM SYSIBM.SYSDUMMY1
In MySql I can use YEARWEEK() to receive the week and the related year of this week in one string. (E.g. SELECT YEARWEEK('1987-01-01'); which leads to "198653").
Is there anything like that in Oracle10g?
I only know about the TO_CHAR function. But if I use TO_CHAR(sysdate, 'YYYYIW'); I receive 198753 and not 198653. So, how I am able to calculate this correctly?
Does using IYYYIW format with TO_CHAR() make any difference? Note the "I" in the beginning instead of first "Y", it is for 4-digit year based on the ISO standard.
I can't reproduce your example that Oracle returns 198753.
select TO_CHAR(DATE '1987-01-01', 'YYYYIW') from dual returns 198701 for me which is correct according to the ISO definition of week numbers.
Oracle has another format mask WW (instead of IW) that uses the week where the first day of the year is in as week #1 - which again would return week number 1 for the January 1st.
Have a look here: http://en.wikipedia.org/wiki/Week_number#Week_numbering
I find MySQL's week number a bit strange actually, because no week numbering scheme I know would return week 53 for January 1st, 1987 (but that doesn't mean very much though...)