Mulesoft add excel date serial number to date - date

I'm reading an excel file data, one of the column is 'DateTime'(i.e. 01/01/1990 00:00:00) but Anypoint read it as string type 'excel date serial number'(i.e. 33257.415972222225).
How do I transform in DataWeave the date correctly?
ex:
date : "01/01/1990 00:00:00" as :date {format: "MM/dd/yyyy HH:mm:ss"} + "P$(33257.415972222225/12)Y"

There is an existing answer for converting Excel dates to Unix dates (epoch time) that can be easily adapted to DataWeave. Note that Excel in Windows and Mac use a different starting date so the formula varies accordingly.
script
%dw 1.0
%output application/json
---
{
dateExcelWindows: (((payload as :number) - 25569 ) * 86400 ) as :datetime,
dateExcelMac: (((payload as :number) - 24107 ) * 86400 ) as :datetime
}
input (this is June 9th, 2011 10:30 AM in Windows Excel):
"40703.4375"`
output:
{
"dateExcelWindows": "2011-06-09T10:30:00Z",
"dateExcelMac": "2015-06-10T10:30:00Z"
}

Related

Wrong day when using day()-formula with format - PowerBI

I'm trying to find out the weekday i.e Mon, Tue, Wed etc. from a date-range formatted as yyyy mm dd
I tried to use the formula format(day(Date Table),"ddd"), but the weekday is wrong. In my example, the output of 2020.01.01 gives Sunday, but it should be Wednesday.
I think your formula is wrong:
Instead of
format(day(Date Table),"ddd")
Use
format(<Target Table>[<date column>],"ddd")
I.e. Omit the DAX DAY call. This is resulting in the day of the month (1..31) being passed to the format function.
When you use the DAY function in DAX, it returns the day of the month (1 through 31).
Thus DAY ( DATE ( 2020, 1, 1) ) = 1 which means you're trying to format the number 1 as a date. Integers are interpreted as days since 1899/12/30 when treated as a date, so 1 corresponds to 1899/12/31, which happened to be a Sunday. Thus FORMAT(1, "ddd") = "Sun".
There's no reason to get DAY involved here. You can simply write
Day = FORMAT ( 'Calendar'[Date], "ddd" )

pyspark : Convert string to date format without minute, decod and hour

Hello I would like to convert string date to date format:
for example from 190424 to 2019-01-24
I try with this code :
tx_wd_df = tx_wd_df.select(
'dateTransmission',
from_unixtime(unix_timestamp('dateTransmission', 'yymmdd')).alias('dateTransmissionDATE')
)
But I got this format : 2019-01-24 00:04:00
I would like only 2019-01-24
Any idea please?
Thanks
tx_wd_df.show(truncate=False)
You can simply use to_date(). This will discard the rest of the date, and pick up only the format that matches the input date format string.
import pyspark.sql.functions as F
date_column = "dateTransmission"
# MM because mm in Java Simple Date Format is minutes, and MM is months
date_format = "yyMMdd"
df = df.withColumn(date_column, F.to_date(F.col(date_column), date_format))

Sas changing of date format

I have three columns with date formatted differently in SAS:
12 june 2017 00:15 - full date
2016 - only year
12 - only month
I Need to change the format of date and subtract after the dates to get results in the number of months.
for instance, "12 June 2017 00:15" - December 2016 = 7
how to do it?
As you have probably already found, there isn't a ready-made SAS date informat that will correctly handle your full date field, so you'll need to write a bit of custom logic to convert it before doing your calculation. date9. is the closest matching format I could find:
data example;
fulldate = '12 june 2017 00:15';
year = 2016;
month = 12;
/* Convert string to date9 format and input */
fulldate_num = input(
cats(
scan(fulldate,1),
substr(scan(fulldate,2,' '),1,3),
scan(fulldate,3)
), date9.
);
/* Calculate difference in months */
monthdiff = intck('month', mdy(month,1,year), fulldate_num);
run;
Convert the "full date" field to a SAS date value.
Convert the combo of year and month to a SAS date value, too.
Use the INTCK function to find the difference in months.
For example:
data dates ;
input dt $18. yy mm ;
mm_diff = intck ("mon", input (cats (yy, mm), yymmn6.), input (dt, anydtdte12.)) ;
put mm_diff= ;
cards ;
12 june 2017 00:15 2016 12
11 june 2018 00:15 2017 3
;
run ;
The log will print:
mm_diff=6
mm_diff=15
As a side note, the statement "there isn't a ready-made SAS date informat that will correctly handle your full date field" made elsewhere in this thread is incorrect. As the program snippet above shows, the ANYDTDTEw. informat handles it with aplomb. It's just incumbent upon the programmer to supply a sufficient informat width W. Above, it is selected as W=12. If you're reluctant to guess and/or count, just use ANYDTDTE32.
Regards,
Paul Dorfman
Assuming that you have three numeric variables and the first one contains valid SAS datetime values you should first convert both to valid SAS date values. You can then use the INTCK() function to count months.
nmonths = intck('month',datepart(VAR1),mdy(VAR3,1,VAR2));

teradata yymmdd date format

I have a field that should be 6 digit character but it is numeric. I am using the following code to add the leading zero:
select CAST(CAST(CHD_OPEN_DATE AS FORMAT '9(6)') AS CHAR(9))
I'm using the following code to format this as a date:
cast(cast(lpad(to_char(CHD_OPEN_DATE),6,'0') as date format 'YYMMDD') as date format 'YYYY-MM-DD')
When using this date format 1990 comes up as 2090. Is there a work-around for this?
If your number has a YYMMDD format you can use the following to cast to a date without the need to cast to an intermediate string. Assuming a date range between 1930 and 2029:
SELECT 900331 AS CHD_OPEN_DATE,
Cast(CASE WHEN CHD_OPEN_DATE < 300000
THEN CHD_OPEN_DATE + 1000000
ELSE CHD_OPEN_DATE
END AS DATE)

What is the precise definition of JDE's Julian Date format?

I am writing code to convert from a Gregorian date to a JDE (J.D.Edwards) Julian date.
Note: a JDE Julian date is different from the normal usage of the term Julian date.
As far as I can work out from Googling, the definition of a JDE Julian date is:
1000*(year-1900) + dayofyear
where year is the 4-digit year (e.g. 2009), and dayofyear is 1 for 1st January, and counts up all year to either 365 or 366 for 31st December (depending whether this is a leap year).
My question is this: are years before 1900 supported? If so, does the above formula still hold, or should it be this:
1000*(year-1900) - dayofyear
(note minus instead of plus.)
or something else?
Does anyone have a link to the official documentation for this date format?
The JDE Julian date consists of CYYDDD which is Century, Year, Day of year.
Century is zero for 20th e.g. 19XX and one for 21st e.g. 20XX.
The year is two digits.
So 101001 is 1 January 2001
As you can see this will not support dates before 1900.
See this Oracle page for a simple and official explanation: About the Julian Date Format
The "JDE Julian Date Converter" does return a negative value for:
1809/07/23 : -90635
As opposed to the classical Julian Date:
The Julian date for CE 1809 July 23 00:00:00.0 UT is
JD 2381986.50000
Here is a example of JD EDWARDS (AS/400 software) Julian Date, but that is not an "official" documentation and it does not seems to support dates before 1900...
Note: this "ACC: How to Convert Julian Days to Dates in Access and Back" does not support date before 1900 either... as it speaks about an "informal" Julian day, commonly used by government agencies and contractors.
The informal Julian day format used in this article is the ordinal day of a year (for example, Julian day 032 represents February 1st, or the 32nd day of the year).
Variations on informal Julian day formats include using a preceding two-digit year (for example 96032 for 2/1/96) and separating the year with a dash (for example 96-032).
Another, less popular, Julian day format uses a one digit year (for example 6-032). These additional formats do not uniquely identify the century or decade. You should carefully consider the consequences when using these formats; for example, the Julian day 00061 can be interpreted as 3/1/2000 or 3/2/1900.
Update: Sorry, JDE is probably something else. But for reference:
The JDE I know is different. From page 59 in the book
"Astronomical algorithms" (Jean Meeus, ISBN 0-943396-35-2):
"If the JD corresponds to an instant
measured in the scale of Dynamical
Time (or Ephemeris Time), the
expression Julian Ephemeris Day
(JDE) is generally used. (Not JED as
it is sometimes written. The 'E' is a
sort of index appended to 'JD')"
JD and JDE (for the same point in time) are close in value
as the difference UT and ET is on the order of minutes. E.g. ET-UT was 56.86 seconds in 1990 and -2.72 seconds in 1900.
There is also MJD (Modified Julian Day):
MJD = JD - 2400000.5
Zero point for MJD is 1858-11-17, 0h UT.
Note that JD as Julian date is a misnomer. It is
Julian day. The JD has nothing to do with the Julian
calendar. (This is in disagreement with the Wikipedia article, this
is from the author of the book mentioned above, Jean Meeus - a Belgian astronomer specializing in celestial mechanics.)
Maybe off from the question, you can convert in Excel using the following formula:
Convert Julian to Date in Excel
In Cell A2 place a Julian date, like 102324
in Cell B2 place this formula: (copy it in)
=DATE(YEAR("01/01/"&TEXT(1900+INT(A2/1000),0)),MONTH("01/01/"&TEXT(1900+INT(A2/1000),0)),DAY("01/01/"&TEXT(1900+INT(A2/1000),0)))+MOD(A2,1000)-1
The date 11/20/02 date will appear in cell B2
Convert Date to Julian in Excel
In Cell C2 copy this formula:
=(YEAR(B2)-2000+100)*1000+B2-DATE(YEAR(B2),"01","01")+1
This will convert B2 back to 102324
Save the below source code in a source member called JDEDATES. Use the runsqlstm on the first line to create the functions. You can then do things like
select jde2date(A1UPMJ), f.* from f00095 f
and see a real date.
Source:
--RUNSQLSTM SRCFILE(qtxtsrc) SRCMBR(JDEDATES) COMMIT(*NONE) NAMING(*SQL)
-- jde 2 date
create function QGPL/jde2date ( d decimal(7,0))
returns date
language sql
deterministic
contains sql
SET OPTION DATFMT=*ISO
BEGIN
if d=0 then return null;
else
return date(digits(decimal(d+1900000,7,0)));
end if;
end; -- date 2 jde
create function QGPL/date2jde ( d date)
returns decimal(7,0)
language sql
deterministic
contains sql
SET OPTION DATFMT=*ISO
BEGIN
if d is null then return 0;
else
return (YEAR(D)-1900)*1000+DAYOFYEAR(D);
end if;
end ;
Several years late to the party, but for other folks like me that find yourselves working with legacy systems like this, I hope some of my java snippets can help. I'm leveraging the fact that you can convert this CYYDDD format into yyyyDDD format and parse based on that.
import java.util.Date;
import java.util.GregorianCalendar;
import java.util.Calendar;
import java.util.SimpleDateFormat;
String jdeJulianDate = "099365"; //Testing with December 31, 1999
// Compile what the year number is
int centIndex = Integer.parseInt(jdeJulianDate.substring(0,1));
int yearIndex = Integer.parseInt(jdeJulianDate.substring(1,3));
int yearNumber = 1900 + (100 * centIndex) + yearIndex;
// Put the year number together with date ordinal to get yyyyDDD format
String fullDate = String.valueOf(yearNumber) + jdeJulianDate.substring(3,6);
// Date parsing, so need to wrap in try/catch block
try {
Date dt = new SimpleDateFormat("yyyyDDD").parse(fullDate);
// Validate it parses to a date in the same year...
Calendar cal = new GregorianCalendar();
cal.setTime(dt);
if (cal.get(Calendar.YEAR) != yearNumber) {
// Cases happen where things like 121366 (should be invalid) get parsed, yielding 2022-01-01.
// Throw exception or what-not here.
}
}
catch (Exception e) {
// Date parsing error handling here
}
A sample of VBA code to convert back and forth between JDE Julian Date and Gregorian:
Public Const Epoch = 1900
Public Const JDateMultiplier = 1000
Public Const FirstJan = "01/01/"
Public Function Julian2Date(ByVal vDate As Long) As Date
Dim Year As Long
Dim Days As Long
Dim SeedDate As Date
' Day Number
Days = vDate - (Int(vDate / JDateMultiplier) * JDateMultiplier) - 1
' Calendar Year
Year = ((vDate - Days) / JDateMultiplier) + Epoch
' First Day of Calendar Year
SeedDate = CDate(FirstJan + CStr(Year))
' Add Number of Days to First Day in Calendar Year
Julian2Date = DateAdd("d", Days, SeedDate)
End Function
Public Function Date2Julian(ByVal vDate As Date) As Long
Dim JYear As String
Dim BeginDate As Date
Dim JDays As Long
' Calendar Year
JYear = Format(Year(vDate), "0000")
' First Day of Calendar Year
BeginDate = CDate(FirstJan + JYear)
' Day Number
JDays = DateDiff("d", BeginDate, vDate) + 1
' Add Number of Days to Year Number
Date2Julian = ((CLng(JYear) - Epoch) * JDateMultiplier) + JDays
End Function
I have tried to make it as clear and simple as possible, and to this end I have intentionally left out any error trapping. However, you should be able to add the code to a VBA module and call them directly from your own code.
I also include some useful snippets of T-SQL:
Todays Date as JDE Julian Date:
(datepart(yy,getdate())-1900) * 1000 + datepart(dy, getdate())
Convert JDE Julian Date to Gregorian (DD/MM/YYYY), replace XXXXXX with the column name containing the JDE Julian Date:
convert (varchar, dateadd (day,convert (int, right(XXXXXX,3)) - 1, convert (datetime, ('1/1/' + convert ( varchar, (cast(left(right(XXXXXX+1000000,6),3) as varchar) + 1900))))),103)
If you require a different Gregorian format, replace the 103 value (right at the end) with the applicable value found here: https://msdn.microsoft.com/en-us/library/ms187928.aspx
I have an easy way for C using time now and epoch 1970, 01, 01 midnight if anybody is interested.
But this is for Julian Day Numbers which is not the same as JDE but they are similar in respect to using math to compute days and I'm sure this idea could be adapted for JDE. Sometimes people just confuse the two like I do. Sorry. But still this is an example of using a time reference which should always be done and since most computers use this it would be just as easy for us not to get too bogged down in dates and just use days before or after this epoch.
Since JDE is now owned by Oracle, they also now support Julian_Day. see:
https://docs.oracle.com/javase/8/docs/api/java/time/temporal/JulianFields.html
#include <stdio.h>
#include <time.h>
#define EPOCH (double) 2440587.5 /* Julian Day number for Jan. 01, 1970 midnight */
int main ()
{
double days = time(0)/86400.0;
printf ("%f days since January 1, 1970\n", days);
printf ("%f\n", days + EPOCH);
return 0;
}
Wow, there's a lot of complicated code in some of these answers just to convert to and from JDE julian dates. There are simple ways in Excel and VBA to get there.
FROM JULIAN
Excel (assuming julian date is in A1):
=DATE(1900+LEFT(A1,LEN(A1)-3),1,RIGHT(A1,3))
VBA (from julian date, j, stored as String):
d = DateSerial(1900 + Left$(j, Len(j) - 3), 1, Right$(j, 3))
VBA (from julian date, j, stored as Long):
d = DateSerial(1900 + Left$(j, Len(CStr(j)) - 3), 1, Right$(j, 3))
TO JULIAN
Excel (assuming date is in A1):
=(YEAR(A1)-1900)*1000+A1-DATE(YEAR(A1),1,0)
VBA (to a Long, j):
j = (Year(d) - 1900) * 1000 + DatePart("y", d)