Date not getting parsed in beamSql - apache-beam

I have this below error :
SEVERE: 2018-03-17T17:15:42.272Z: java.lang.IllegalArgumentException: Unable to encode element 'BeamRecord [dataValues=[2/12/2017], dataType=BeamRecordSqlType [fieldNames=[c0], fieldTypes=[12]]]' with coder 'org.apache.beam.sdk.coders.BeamRecordCoder#56ce2595'.
I have specified dataValues=[2/12/2017] as TIMESTAMP datatype. link
Any idea whats going wrong here ?

It is likely you're using incorrect type for the TIMESTAMP field. BeamRecord expects a value of type java.util.Date in the TIMESTAMP fields.
If your input is coming from JSON, for example, you might need to parse the timestamp field using java.text.DateFormat, see example here.
If you're specifying the TIMESTAMP in the query, then try specifying the timestamp in this format: TIMESTAMP '1999-02-22 01:01:58'. See examples here.

Related

GET v2/entities filtering by dates

I'm performing an GET /v2/entities query to an exisiting Orion Context Broker and I would like to filter the retrieved entities by TimeInstant attribute, which is in "date" format: 2019-11-14T06:00:23.00Z. I would like to get ONLY those entities which TimeInstant value is > 2019-11-14T06:10:23.00Z. I haven't found examples showing how to deal with attributes different from "int" values.
You have an example for instance in slide 65 of the Orion introductory presentation:
GET /v2/entities?q=TimeInstant>2019-11-14T06:10:23.00Z
Note that in order that query to work, TimeInstant attribute must be of DateTime type (or ISO8601 type, which is an alias). More information on DateTime type in the NGSv2 specification.

Insert yyyyMMdd string into date column using Talend

I have the follow situation:
A PostgreSQL database with a table that contains a date type column called date.
A string from a delimited .txt file outputting: 20170101.
I want to insert the string into the date type column.
So far i have tried the following with mixed results/errors:
row1.YYYYMMDD
Detail Message: Type mismatch: cannot convert from String to Date
Explanation: This one is fairly obvious.
TalendDate.parseDate("yyyyMMdd",row1.YYYYMMDD)
Batch entry 0 INSERT INTO "data" ("location_id","date","avg_winddirection","avg_windspeed","avg_temperature","min_temperature","max_temperature","total_hours_sun","avg_precipitation") VALUES (209,2017-01-01 00:00:00.000000 +01:00:00,207,7.7,NULL,NULL,NULL,NULL,NULL) was aborted. Call getNextException to see the cause.
can see the string parsed into "2017-01-01 00:00:00.000000 +01:00:00".
When I try to execute the query directly i get a "SQL Error: 42601: ERROR: Syntax error at "00" position 194"
Other observations/attempts:
The funny thing is if I use '20170101' as a string in the query it works, see below.
INSERT INTO "data" ("location_id","date","avg_winddirection","avg_windspeed","avg_temperature","min_temperature","max_temperature","total_hours_sun","avg_precipitation") VALUES (209,'20170101',207,7.7,NULL,NULL,NULL,NULL,NULL)
I've also tried to change the schema of the database date column to string. It produces the following:
Batch entry 0 INSERT INTO "data" ("location_id","date","avg_winddirection","avg_windspeed","avg_temperature","min_temperature","max_temperature","total_hours_sun","avg_precipitation") VALUES (209,20170101,207,7.7,NULL,NULL,NULL,NULL,NULL) was aborted. Call getNextException to see the cause.
This query also doesn't work directly because the date isn't between single quotes.
What am i missing or not doing?
(I've started learning to use Talend 2-3 days ago)
EDIT//
Screenshots of my Job and tMap
http://imgur.com/a/kSFd0
EDIT//It doesnt appear to be a date formatting problem but a Talend to PostgreSQL connection problem
EDIT//
FIXED: It was a stupid easy problem/solution ofcourse. THe database name and schema name fields were empty... so it basically didnt know where to connect
You don't have to do anything to insert a string like 20170101 into a date column. PostgreSQL will handle it for you it's just ISO 8601's date format.
CREATE TABLE foo ( x date );
INSERT INTO foo (x) VALUES ( '20170101' );
This is just a talend problem, if anything.
[..] (209,2017-01-01 00:00:00.000000 +01:00:00,207,7.7,NULL,NULL,NULL,NULL,NULL)[..]
If Talend doesn't know by itself that passing timestamp into query requires it to be single quoted, then if possible - you need to do it.
FIXED: It was a stupid easy problem/solution ofcourse. THe database name and schema name fields were empty... so it basically didnt know where to connect thats why i got the BATCH 0 error and when i went deeper while debugging i found it couldnt find the table, stating the relation didnt exist.
Try like this,
The data in input file is: 20170101(in String format)
then set the tMap like,
The output is as follows:

Get Oracle Timestamp value via orm lite in java

I have the oracle database field value "11-JUL-16 02.51.45.000000000 AM" for field date_updated.
When I retrieve records via orm lite query and iterate over the result set .. I get the data in this format "2016-7-11.2.51. 45. 0" where the java pojo object mapping field is of type String.
Aim is to update these timestamps after processing them. I am not able to covert the date to update(parse error) or retrieve the date as is.
Searched allover but couldn't find an answer. I tried changing the pojo field type to Date/Timestamp(sql) but couldn't get it to work. Any Help would really appreciate ..

Grails date conversion from and to Mongo

I am using the Grails MongoDb plugin and trying to get the Date conversion.
In my config I have defined different dateFormats
grails.databinding.dateFormats = [
'yyyy-MM-dd',
'yyyy-MM-dd HH:mm:ss.S',
"yyyy-MM-dd'T'hh:mm:ss'Z'"
]
However when I read a record from mongodb into my Customer domain class, I still get this error: What am I doing wrong?
org.springframework.beans.TypeMismatchException: Failed to convert
property value of type 'java.lang.String' to required type
'java.util.Date' for property 'lastUpdateDate'; nested exception is
org.springframework.core.convert.ConversionFailedException: Failed to
convert from type java.lang.String to type
#org.grails.databinding.BindingFormat java.util.Date for value
'1994-01-29T00:00:00Z'; nested exception is
java.lang.IllegalArgumentException: java.lang.NumberFormatException:
For input string: "1994-01-29T00:00:00Z"
It looks like you have manually inserted the value of lastUpdateDate in your MongoDB database which got stored in String format instead of MongoDB's ISODate() format. Delete or modify that record in database and this will work fine.

BizTalk and DB2 CLOB

Has anyone had experience dealing with DB2 stored procedures which take a CLOB input parameter and calling that stored procedure from BizTalk?
I've tried changing the schema type to string, base64binary, hexbinary, byte, but no matter what I get this error:
Error details: The parameter value for parameter 1 could not be converted to a native data type. Parameter Name: P_EML_BODY, Data Type: Long strings of input text<br> More long strings of input text <br>More long strings of input text, Value : CharForBit
It might be the way the parameters are being created and in what order. Take a look here. Are any of them null or empty?