In a data flow from one table to another, I would like to cast a date.
The date leaves the source table as a string in this format: "2009-01-05 00:00:00:000 + 01:00".
I tried to convert this to a date using a tConvertType, but that is not allowed apparently.
My second option is to cast this string to a date using a formula in a tMap component.
At the moment I tried these formulas:
- TalendDate.formatDate("yyyy-MM-dd",row3.rafw_dz_begi);
- TalendDate.formatDate("yyyy-MM-dd HH:mm:ss",row3.rafw_dz_begi);
- return TalendDate.formatDate("yyyy-MM-dd HH:mm:ss",row3.rafw_dz_begi);
None of these worked. When inserting the result into the target-table (MySQL, InnoDB) a receive the error message that the date is not valid. The format of the target field is a MySQL Date field.
How can I cast the date to the desired format?
Talend offers you a nice way of handling date formats.
You can easily change the date format in the Schema editor tab at the tMap window.
It works for both tMap input and output flows.
I've added a picture for a better illustration.
to cast this string to a date using a formula.... error message that
the date is not valid. The format of the target field is a MySQL Date
field.
What I understand from your question is, you want to insert a date into MySQL Date field.
But the method that you are using, returns the 'String' type.
TalendDate.formatDate(String pattern, Date date); //formats a date into Date/Time string
So in that case, if your field is of 'Date' type.
TalendDate.parseDate("yyyy-MM-dd", TalendDate.formatDate("yyyy-MM-dd",row3.rafw_dz_begi));
Related
I have a column which contians date as string but in many formats like - dd/MM/yy, dd/MMM/yyy .. etc etc. And I am using the following code to convert all strings to one specific date format (yyyy-MM-dd) in hive :
select
from_unixtime(unix_timestamp('31/02/2021','dd/MM/yyyy'),'yyyy-MM-dd')
but this gives me 2021-03-03 in HIVE.
Is there any other way to identify such invalid dates and give null.
Assume, you recognized format correctly and it is exactly 'dd/MM/yyyy' and date is invalid one '31/02/2021'.
unix_timestamp function in such case will move date to the next month and there is no way to change it's behavior. But you can check if the date double-converted from original string to timestamp and back to original format is the same. In case it is not the same, then the date is invalid one.
case
-- check double-converted date is the same as original string
when from_unixtime(unix_timestamp(date_col,'dd/MM/yyyy'),'dd/MM/yyyy') = date_col
--convert to yyyy-MM-dd if the date is valid
then from_unixtime(unix_timestamp('31/02/2021','dd/MM/yyyy'),'yyyy-MM-dd')
else null -- null if invalid date
end as date_converted
Can I accept "yyyy-mm-dd" instead of "yyyymmdd" when I put data into a date type column in DataStudio?
From the following Q&A and the official documentation, it looks like "yyyymmdd" only.
Google data studio table shows date column as null
https://support.google.com/datastudio/answer/6401549?hl=en#zippy=%2Cin-this-article
However, since the date type of Snowflake, the source of the linkage, only allows "yyyy-mm-dd", the
I'd like to know how to do this efficiently, because I'm changing the date type to a character type, linking it to DataStudio, and then changing it back to a date type in DataStudio.
https://docs.snowflake.com/en/sql-reference/data-types-datetime.html#date
I would appreciate it if you could tell me.
Not exactly sure how things are mapped from Snowflake to DataStudio, but Snowflake can produce yyyymmdd dates.
SELECT TO_VARCHAR('2021-08-19'::DATE, 'YYYYMMDD');
If DataStudio requires that the data type from the source be a DATE then perhaps you could alter the date_output_format of your connection session or user to be 'YYYYMMDD'.
https://docs.snowflake.com/en/sql-reference/parameters.html#label-date-output-format
Import your data as it is into Data Studio. In the schema screen, ensure your source date field ("yyyy-mm-dd") is formatted as text. Then add a calculated field with the formula:
TODATE(source_date_field,"%Y-%m-%d","%Y%m%d")
TODATE reference
Formate the new calculated field as date.
I am trying to change the format of a timestamp field to a date field in a certain format with AWS Redshift.
I googled a lot and the common "best practice" that I found was to cast the timestamp to a date and then use to_char to bring it into the right format. In the end I want to use the date field in a Qlik Sense dashboard where it is input for a date picker extension that apparently requires the format DD/MM/YYYY to work.
Current date format in the DB:
9/2/2019 6:38:00 AM (which I would describe as M/D/YYYY H:MM:SS ZZ)
Desired output:
DD/MM/YYYY, resulting in the value 02/09/2019
Current status:
to_char(cast(timestamp_field as date), 'DD/MM/YYYY') --> result: 02/09/2019
However, the date picker extension in Qlik Sense still does not work and I guess that's because the output is a string and not a date. Casting the string to a date returns in an error.
cast(to_char(cast(timestamp_field as date), 'DD/MM/YYYY') as date) as date_picker_date
Connector reply error: SQL##f - SqlState: 57014, ErrorCode: 30,
ErrorMsg: [Amazon][Amazon Redshift] (30) Error occurred while trying
to execute a query: [SQLState 57014] ERROR: Error converting text to
date
I am new to Redshift and would have expected to be able to cast to date with a format string as parameter, but apparently that's not a thing. Can someone enlighten me on how to solve this?
You can try with the timestamp at the beginning of the script of the QlikSense.
It will be at the start of the script. in the Main section as shown below:
SET TimestampFormat='M/D/YYYY h:mm:ss[.fff] TT';
here SET is being used to define the time stamp in that application.
You can try changing here to your requirements.
Else 2 options 1 use a stamp which you have already used or create your own extension from the lib
Stamp use can be like this:
Datestamp_Table:
Load *,
Date(Date#(StringDate,'M/D/YY'),'DD/MMM/YY') as Date;
LOAD * INLINE [
StringDate
8/7/97
8/6/97];
Output will be like this:
Stringdate Date
8/7/97 07/Aug/97
8/6/97 06/Aug/97
I have Excel data and trying to insert the data into MongoDB using Talend Big Data for Open Studio. This is my job,
tFileInputExcel --> tMap --> tMongoDBOutput
In excel sheet, i have a date value column in this format 7/13/2017(MM/dd/yyyy) as string type and I am trying to insert this column value as ISO format ISODate("2017-07-13T00:00:00.000Z") in MongoDB.
This is my Job:
tFileInputExcel:
tMap:
tMongoDBOutput:
When execute this job, I'm getting the below error.
Error:
When i change the parse format like this TalendDate.parseDate("MM/dd/yyyy",row1.ClosingDate) , I'm getting SimpleDateFormat error. Simple Date Format Error
How to resolve this issue?
you can do simply if you mongodb column schema is date:
TalendDate.parseDate("MM/dd/yyyy",row3.newColumn)
That will automatically convert the date in the date model that your mongoDB column have.
You can change in your schema in Talend the date Model like "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'".
This is a very common mistake doing in reading data without understanding the underlying data types.
I have blogged about this especially for Talend: https://www.tobiasmaasland.de/2017/07/20/using-date-in-talend-etl-jobs/
But let me explain a bit.
Sometimes Excel tries to convert data in the cell even if one might think the cell type is set to String. Insted, it is set to Date. As such, no conversion is needed and the type needs to be Date in the input component.
If it is a String and an error occurs, the the structure of the String is either not the same everywhere or some cells are empty (null). So you might be lucky with
TalendDate.parseDate("MM/dd/yyyy", (row1.ClosingDate == null), "01/01/1970", row1.ClosingDate)
I just assumed you might want to use a placeholder date insted of having null.
This heavily depends on the actual data type in the cells, if every cell has the same data type and if all the data is formatted correctly.
To sum up one of the facts in my blog post: Don't use String for dates. Use Date for dates in Excel. It makes everything easier.
I would like to separate DateTime to Date using talend.
the DateTime column type is in Timestamp without timezone type and the Date column type is date.
I have entered
TalendDate.getDate("yyyy-MM-dd HH:mm:ss", row1.datetime)
at the middle column for TMap.
I have entered
TalendDate.getDate("dd-MM-yyyy",Var.getTimestamp)
at the output.
However I got this error message:
"Detail Message: The method getDate(String) in the type TalendDate is not applicable for the arguments (String, Date)".
Please help.
The function getDate will accept only one parameter which is the pattern the date need to be displayed and moreover, TalendDate.getDate("dd-MM-yyyy") will return current date in mentioned pattern, which is the result would be
10-03-2017 // Since Today's date is 10th March 2017.
In order to format the date you have, there is another function that talend provides, which is
TalendDate.formatDate("dd-MM-yyyy",row1.datetime)
Say if row1.datetime has value as "2017-02-23 12:12:12", then the result would be
23-02-2017
Hope this would solve your problem.