To transfer data from Ms sql server 2008 to Snowflake I used talend , but every time I get error as
java.io.IOException: net.snowflake.client.loader.Loader$ConnectionError: State: CREATE_TEMP_TABLE, SQL compilation error: error line 1 at position 68
invalid identifier '"columnname"'
at org.talend.components.snowflake.runtime.SnowflakeWriter.close(SnowflakeWriter.java:397)
at org.talend.components.snowflake.runtime.SnowflakeWriter.close(SnowflakeWriter.java:52)
at local_project.load_jobnotes_0_1.Load_Jobnotes.tMSSqlInput_1Process(Load_Jobnotes.java:2684)
at local_project.load_jobnotes_0_1.Load_Jobnotes.runJobInTOS(Load_Jobnotes.java:3435)
at local_project.load_jobnotes_0_1.Load_Jobnotes.main(Load_Jobnotes.java:2978)
Caused by: net.snowflake.client.loader.Loader$ConnectionError: State: CREATE_TEMP_TABLE, SQL compilation error: error line 1 at position 68
invalid identifier '"ID"'
at net.snowflake.client.loader.ProcessQueue.run(ProcessQueue.java:349)
at java.lang.Thread.run(Thread.java:748)
Caused by: net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error: error line 1 at position 68
The column does exist in my Snowflake DB still I get error as column does not exist
On analysing what query Talend executing in snowflake I found that It tries to create a temporary table to store data but in doing so it selects all column from table between “ ” double quotes and hence error comes as invalid identifier '"columnname"'
If I execute the same query manually without double quotes its works fine , can you please let us know what is workaround of this issue
Query executed by talend in snowflake for your reference
CREATE TEMPORARY TABLE "Tablename_20171024_115736_814_1"
AS SELECT "column1","column2","column3"
FROM "database"."schema"."table" WHERE FALSE
The issue is most likely due to a case mismatch between the object names in Snowflake and what is being sent through the connector. On the Snowflake side, all object names are stored as UPPER CASE. Suggest you try passing COLUMN1, COLUMN2, etc and see if that works.
You can also try setting the QUOTED_IDENTIFIERS_IGNORE_CASE to true, it might help.
I found that this issue is due to mixed case database or schema names not properly being applied by Talend. I discover a hack by updating the Snowflake connector role parameter and added something such as this screenshot:
Related
I am working on DB migration from Oracle to Postgres and came across a situation where a legacy Java code is calling a stored procedure from Oracle. It is a simple call without parentheses like this : Mgr.AdmSP.myproc = ad.ctlpk.proc
where ad is schema name, ctlpk is Package name and proc is Stored Procedure in Oracle
we have migrated procedure proc to Postgres but when it comes to calling, Postgres expects () for procedure or function calls even if it does not take any parameters.
So we tried to run it like this : Mgr.AdmSP.myproc = ad_ctlpk.proc(). Here ad_ctlpk is postgres schema name. After executing we are getting error : syntax error at or near "(".
Since this is a database migration only, we are not expected to make any changes to other components except database objects. Is there a way to mimic the Oracle like procedure calling style in Postgres?
Like Postgres have function now() and current_timestamp which perform same functionality but with different syntax. I am using Postgres Ver 11.11.
Adding some details as per comments for more details. In Oracle, it is defined as FUNCTION Proc RETURN INT and as of now we converted it into Postgres function as
CREATE FUNCTION ad_ctlpk.proc() RETURNS INTEGER
I am not sure about the Framework and other java code details since I dont have access to Java code. We have access to executables only, which when executed gives below error
[[AdmThread 1] [WARN:AbstractThreadedPublisherManager$ExceptionHandlingThreadGroup:48] Error in thread AdmThread 1 : Problem in doWork
Caused by: Problem running proc procedures
Caused by: ERROR: syntax error at or near "("
Position: 38, State: 42601, ErrorCode: 0
com.ubs.datait.common.distribution.exception.DistributionRuntimeException: Problem in doWork
at common.distribution.manager.AdmThread.doWork(AdmThread.java:72)
at common.distribution.common.AbstractDistributionThread.run(AbstractDistributionThread.java:30)
Caused by: common.distribution.exception.DistributionRuntimeException: Problem running proc procedures
at common.distribution.manager.AdmThread.runAdminProc(AdmThread.java:100)
at common.distribution.manager.AdmThread.doWork(AdmThread.java:67)
... 1 more
Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at or near "("
Position: 38
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2552)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2284)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:322)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:481)
Update: :
Since Postgres does not support creating/calling functions without braces (), we finally had to ask for changes in java code to add () to the function calls where required.
When I have searched for "DB2 SQL
Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null"
I got to know that we get this error due to
"THE VALUE OF INPUT VARIABLE OR PARAMETER NUMBER position-number IS INVALID OR TOO LARGE FOR THE TARGET COLUMN OR THE TARGET VALUE, position" and number should give you the number of the column that is causing your troubles.
In that case Is it the "driver" column causing trouble for error message
"com.ibm.db2.jcc.am.SqlDataException: DB2 SQL Error: SQLCODE=-302,
SQLSTATE=22001, SQLERRMC=null, DRIVER=3.65.97"
for this case Do I have to add driver or do I have to change any driver? If I have to change driver which driver do I have to change??
I'm using Mulestudio and am trying to insert into the Postgres database some data. I am modifying the log4j.properties file and below is how it looks like:
log4j.rootLogger = DEBUG, postgres
#
log4j.appender.postgres=org.apache.log4j.jdbc.JDBCAppender
log4j.appender.postgres.layout=org.apache.log4j.PatternLayout
log4j.appender.postgres.driver=org.postgresql.Driver
log4j.appender.postgres.URL=jdbc:postgresql://127.0.0.1:5432/testing
log4j.appender.postgres.user=postgres
log4j.appender.postgres.password=pw
log4j.appender.postgres.sql=INSERT INTO LOGS VALUES ('%x', '%d{yyyy-MM-dd}','%C','%p','%m');
The error message that I get is
log4j:ERROR Failed to excute sql
org.postgresql.util.PSQLException: ERROR: syntax error at or near "edu"
'edu' is the first part of my project name (edu-stream-ucdnews). The instance of 'edu' only comes up in the title name and not in my data. I know that the error arises when I have the '%m' when I try to insert the data because when I change it to a hard-coded message like 'Hello', I don't get any error.
How do I solve this issue?
Are you sure your are connected to the database?
Try putting the fields name in your sql and lets check if we get a better error log:
log4j.appender.postgres.sql=INSERT INTO LOGS (field1, field2, ...) VALUES ('%x', '%d{yyyy-MM-dd}','%C','%p','%m');
I am using IBM RAD.
I am executing following query.
Conn.prepareStatement("update UPLOAD set STATUS='Decrypted' WHERE PATH ='"+path+"'");
the datatype of PATH in DB2 is VARCHAR.
i am getting following error
SQLCODE=-401,SQLSTATE=42818,SQLERRMC==,
The error message means that you are comparing different data type.
eg :
'12'=12
I have followed the instructions from here and installed the updated plugin. The error has become:
Query error
Message: net.sf.jasperreports.engine.JRException:
Error executing SQL statement for : null Level: SEVERE Stack Trace:
Error executing SQL statement for : null com.jaspersoft.hadoop.hive.HiveFieldsProvider.getFields(HiveFieldsProvider.java:113)
com.jaspersoft.ireport.hadoop.hive.designer.HiveFieldsProvider.getFields(HiveFieldsProvider.java:32)
com.jaspersoft.ireport.hadoop.hive.connection.HiveConnection.readFields(HiveConnection.java:154)
com.jaspersoft.ireport.designer.wizards.ConnectionSelectionWizardPanel.validate(ConnectionSelectionWizardPanel.java:146)
org.openide.WizardDescriptor$7.run(WizardDescriptor.java:1357)
org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:572)
org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:997)
After downgrading to 4.5.0 the error has become (the connection is verified and I am able to query the table from hive):
Query error
Message: net.sf.jasperreports.engine.JRException: Query returned non-zero code: 10, cause:
FAILED: Error in semantic analysis: Line 1:14 Table not found 'panstats' Level:
SEVERE Stack Trace: Query returned non-zero code: 10, cause:
FAILED: Error in semantic analysis: Line 1:14 Table not found 'panstats'
com.jaspersoft.hadoop.hive.HiveFieldsProvider.getFields(HiveFieldsProvider.java:260)
com.jaspersoft.ireport.hadoop.hive.designer.HiveFieldsProvider.getFields(HiveFieldsProvider.java:32)
com.jaspersoft.ireport.hadoop.hive.connection.HiveConnection.readFields(HiveConnection.java:146)
com.jaspersoft.ireport.designer.wizards.ConnectionSelectionWizardPanel.validate(ConnectionSelectionWizardPanel.java:146)
org.openide.WizardDescriptor$7.run(WizardDescriptor.java:1357)
org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:572)
org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:997)
I am using Hive 0.8.1 on OS X Lion 10.7.4.
Is your query as simple as select * from panstats? I suspect that the query is not the problem, but you'll want to confirm that first.
You could try querying that table from a tool like SQuirreL SQL. If that tool also cannot get the data, then it's probably a Hive issue. If it can... then it's probably an issue with iReport or the Hive plugin.
It sounds like Hive is not configured to share metadata. It uses the annoying default configuration with Derby, so outside connections don't get access to your panstats table. I came across this article about configuring Hive earlier this year. It documents using MySQL instead of derby. If that's indeed the problem, then it's just a Hive configuration issue. Following that article would solve things both for SQuirreL and for iReport.