I am trying to use io.confluent.connect.jdbc.JdbcSourceConnector in bulk mode using the query .
query = select name, cast(ID as NUMBER(20,2)),status from table_name
Is this possible?
If possible am I missing something?
I am getting
exception (org.apache.kafka.connect.runtime.WorkerTask:148)
org.apache.avro.SchemaParseException: Illegal character in: CAST(IDASNUMBER(20,2))
Use an alias for the 2nd column. This alias must be the name of the target avro-field.
Related
We are using liquibase to deploy the data warehouses. We are facing the issue with the default schema name property of liquibase. When we set the value for the same to the schema in which we want the deployment to be done, it gives the following error -
Caused by: liquibase.exception.DatabaseException: ERROR: syntax error
at or near "$" Position: 40 [Failed SQL: (0) SET SEARCH_PATH TO
, $user, public]
The $user, should be in quotes and hence the issue is occurring. Is anyone also facing a similar issue with Redshift while deploying databases using liquibase?
https://docs.aws.amazon.com/redshift/latest/dg/r_search_path.html
Looks like the issue is related to Liquibase version 4.11.0, just use Liquibase v4.10.0
Instead of using --defaultSchemaName parameter, use --liquibase-schema-name parameter to set the schema for liquibase tables.
workaround
execute in database something similar
ALTER USER user_name SET search_path to 'public'
To transfer data from Ms sql server 2008 to Snowflake I used talend , but every time I get error as
java.io.IOException: net.snowflake.client.loader.Loader$ConnectionError: State: CREATE_TEMP_TABLE, SQL compilation error: error line 1 at position 68
invalid identifier '"columnname"'
at org.talend.components.snowflake.runtime.SnowflakeWriter.close(SnowflakeWriter.java:397)
at org.talend.components.snowflake.runtime.SnowflakeWriter.close(SnowflakeWriter.java:52)
at local_project.load_jobnotes_0_1.Load_Jobnotes.tMSSqlInput_1Process(Load_Jobnotes.java:2684)
at local_project.load_jobnotes_0_1.Load_Jobnotes.runJobInTOS(Load_Jobnotes.java:3435)
at local_project.load_jobnotes_0_1.Load_Jobnotes.main(Load_Jobnotes.java:2978)
Caused by: net.snowflake.client.loader.Loader$ConnectionError: State: CREATE_TEMP_TABLE, SQL compilation error: error line 1 at position 68
invalid identifier '"ID"'
at net.snowflake.client.loader.ProcessQueue.run(ProcessQueue.java:349)
at java.lang.Thread.run(Thread.java:748)
Caused by: net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error: error line 1 at position 68
The column does exist in my Snowflake DB still I get error as column does not exist
On analysing what query Talend executing in snowflake I found that It tries to create a temporary table to store data but in doing so it selects all column from table between “ ” double quotes and hence error comes as invalid identifier '"columnname"'
If I execute the same query manually without double quotes its works fine , can you please let us know what is workaround of this issue
Query executed by talend in snowflake for your reference
CREATE TEMPORARY TABLE "Tablename_20171024_115736_814_1"
AS SELECT "column1","column2","column3"
FROM "database"."schema"."table" WHERE FALSE
The issue is most likely due to a case mismatch between the object names in Snowflake and what is being sent through the connector. On the Snowflake side, all object names are stored as UPPER CASE. Suggest you try passing COLUMN1, COLUMN2, etc and see if that works.
You can also try setting the QUOTED_IDENTIFIERS_IGNORE_CASE to true, it might help.
I found that this issue is due to mixed case database or schema names not properly being applied by Talend. I discover a hack by updating the Snowflake connector role parameter and added something such as this screenshot:
I am using flyway for a deployment and the tables built as a result of flyway are all fine.
The issue I have is with the schema_version table. I am unable to query an individual column in the table. I am only able to perform a select *.
The error message I am getting is:
10:35:49 [SELECT - 0 row(s), 0.000 secs] 1) [Error Code: -206, SQL State: 42703] DB2 SQL Error: SQLCODE=-206, SQLSTATE=42703, SQLERRMC=SCRIPT, DRIVER=4.13.127. 2) [Error Code: -727, SQL State: 56098] DB2 SQL Error: SQLCODE=-727, SQLSTATE=56098, SQLERRMC=2;-206;42703;SCRIPT, DRIVER=4.13.127
Try enclosing the column name in double quotes:
select "SCRIPT" from flyway.schema_version
This might help.
Flyway Schema table is designed to be in lower case.
You can change the table name using below config property.
flyway.table=SCHEMA_VERSION
For more details you can check this
https://flywaydb.org/documentation/faq#case-sensitive
Try to make your query like this:
SELECT "version", "installed_on" FROM "schema_version";
I'm using Mulestudio and am trying to insert into the Postgres database some data. I am modifying the log4j.properties file and below is how it looks like:
log4j.rootLogger = DEBUG, postgres
#
log4j.appender.postgres=org.apache.log4j.jdbc.JDBCAppender
log4j.appender.postgres.layout=org.apache.log4j.PatternLayout
log4j.appender.postgres.driver=org.postgresql.Driver
log4j.appender.postgres.URL=jdbc:postgresql://127.0.0.1:5432/testing
log4j.appender.postgres.user=postgres
log4j.appender.postgres.password=pw
log4j.appender.postgres.sql=INSERT INTO LOGS VALUES ('%x', '%d{yyyy-MM-dd}','%C','%p','%m');
The error message that I get is
log4j:ERROR Failed to excute sql
org.postgresql.util.PSQLException: ERROR: syntax error at or near "edu"
'edu' is the first part of my project name (edu-stream-ucdnews). The instance of 'edu' only comes up in the title name and not in my data. I know that the error arises when I have the '%m' when I try to insert the data because when I change it to a hard-coded message like 'Hello', I don't get any error.
How do I solve this issue?
Are you sure your are connected to the database?
Try putting the fields name in your sql and lets check if we get a better error log:
log4j.appender.postgres.sql=INSERT INTO LOGS (field1, field2, ...) VALUES ('%x', '%d{yyyy-MM-dd}','%C','%p','%m');
I am using IBM RAD.
I am executing following query.
Conn.prepareStatement("update UPLOAD set STATUS='Decrypted' WHERE PATH ='"+path+"'");
the datatype of PATH in DB2 is VARCHAR.
i am getting following error
SQLCODE=-401,SQLSTATE=42818,SQLERRMC==,
The error message means that you are comparing different data type.
eg :
'12'=12