I'm using hive to create and try to load file content into the table.
There's a column type "Date" and the date format in the file is dd/mm/yy, for example: 01/12/2013
But when I trie to load the data into table from the file, the column values corresponding to the "Date" is always NULL, as if failed to load the date content.
I put the column content into a txt file and upload to the hdfs, so, the column may be:
id, name, birthdate
and corresponding value are:
1, "Joan", 04/05/1989
But the "04/05/1989" seems can't be read into the table, always null.
Please tell me if the format in my txt file is wrong or I need some specific grammar when loading date type data into Hive table.
Thanks!
Date data type format is YYYY-MM-DD. You need to format field accordingly.
More details on
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-date
Related
I am working with tableau which is connected to database. The one column in it is date information which is in sting format has missing values hence it is very difficult to convert it to DATE type. Is there any way I can convert blanks with null or 0s? Following code did not work.
If you go to "Data Source" tab you can set the column type as Date and everything that is not a date should be set to NULL.
For example, if you import this CSV
"C1";"C2";
"31/01/2021";"A";
"24/12/2001";"B";
" ";"C";
"";"D";
"1/1/2001";"E";
And you set Date on column C1 as in the following image.
alla blank values should be set to NULL.
So if I have a source file containing 10 columns and my target contains 11 columns and the extra column is of type date . The source file contains a name like 'cust20201212' now I wish to extract only the date part and add this to my target table column in the date type column. Is it possible to achieve this using talend. I just want to extract the date as 2020-12-12 and 2020-12-01 and store it in the date column of oracle table.
Can we use tregexextract in this scenario?
You need firstly to get your Filename in the flow, or in a variable . Do you have it in a context variable, or does it come from a tFileList ?
If you have a tFileList in your job , you can access it with global Variable
: ((String)globalMap.get("tFileList_1_CURRENT_FILE"))
When you have this filename, you have to parse your filename to extract the data :
TalendDate.parseDate("yyyyMMdd",StringHandling.LEFT(StringHandling.RIGHT(*PLACE_HERE_FILENAME*,12),8))
StringHandling.RIGHT get the last part of your filename (8 chars + extension (4 chars) = 12)
StringHandling.LEFT gets you the first 8 chars of this expression (20201201)
TalendDate.parseDate convert the string representing your date to an actual date.
Then you can pass this new data to your oracle db.
i am working on ETL job in datastage , a simple one Source ---> tRANsformer -----> destination
the source is a csv file , the destination is db2 base , so the prob is that the csv file contains a string timestamp like this
and i need to put it the db2 stage this is my table that i created with a script
this is the transformer config
and this is my prob this error appears
that means this in english
update_or_insert, 3: Unhandled conversion error in the "SEC_DAT_DATE_INSERT" zone from the source
type "timestamp" to the target type "timestamp [microseconds]":
source value = "*****************". The result does not accept a NULL value and there is no
handle_null to specify a default value
I don't know what it means that's the prob if anyone could help that would be nice thanks
First off, verify how Excel has handled the timestamp. Change the display format so that it conforms to ISO 8601, namely YYYY-MM-DD HH:NN:SS format, before you export it to CSV. Check the CSV file using a text editor (Notepad or Wordpad) to confirm that the timestamp format is correct.
Then change the StringToTimestamp() function so that it reflects the new format (or leave out the format entirely if this is your default format.
Note that the Else part of your expression uses a string. Perhaps you need to wrap that in a StringToTimestamp() function.
i will suggest you to check weather you have marked that column as key in the source (can happen by mistake) if so then deselect the key check box and see weather nullable is set to YES for that column in source ,if not then try and run on nullable column with YES selected. hope this helps.
I have two tables with similar columns
Apply append, all details were ok except for the date of the new table.
Old data dates are available, but the new one is missing and specified as "null"
I check their format, both are the same
Anyone once knows what is the issue.
Below screenshot for reference
enter image description here
It's look like new table has different column name for date.
I have a set of pivot tables that use external csv files as their data sources. The csv files originally contained dates in the format dd/mm/yy (e.g. 31/01/13). The pivot tables did not recognise these as dates. I converted the dates in the csv files to dd/mm/yyyy (e.g. 31/01/2013) but these were still not recognised as dates by the pivot tables.
I tried setting up a calculated field =DATEVALUE(date_from_csv) but when used in the pivot table (I'm using the Max option to select the most recent date) I get #VALUE! errors.
I have tried converting the csv file to xlsx and also importing the data into the workbook that contains the pivot table - but I can't change from the external connection to use the internal data. I don't want to rebuild the pivots as there are a lot of variables and formatting that would take ages to redo.
Any ideas??
The problem was caused by the date column being blank for some rows and I found that if I moved a row to the top (after the header line) that had all the fields filled in, then Excel got the formats correct and the pivot tables now work!