JDBC Producer in streamsets that could not write data into MySql - streamsets

I had configured the JDBC connection configuration in the pipeline.
and when the application executes i get the following error on the logs.
"java.sql.SQLSyntaxErrorException: Table 'databaseName.aim_table' doesn't exist"
The databaseName is not what I have set.
I have tried many times. it shows the same message that could not find the table in different database, and the question is all the db occurred in the sdc.log are that I had never configured ,and the correct database is never used ,so I want to know how could it find the wrong db and I had checked before start the pipeline and it shows successful:

Do you have anything set in the Schema Name configuration for JDBC
Producer? This should be blank for MySQL, since you're setting the
database/schema name in the connect URL.
Check that your MySQL driver matches the server. In particular, using
the current version 8.0.x JDBC driver with a 5.x.x server seems to
result in this problem. Download the older 5.1.x driver (currently
5.1.46) and it should work.
refer this

This problem is indeed caused by the wrong version of the driver package. I found the correct driver package and successfully wrote the data to the target table. add aonther point, I have set the SCHEMA NAME to blank and defined the database name in the connect URL for mysql.
My English is not good. Please forgive me.

Related

postgres update query not works from Metabase

I'm using the open source Metabase version, and I was able to connect my PostgreSQL databases correctly.
but when im trying to run a update query from metabase its throws below error.
ERROR: cannot execute UPDATE in a read-only transaction
but when I execute same query from terminal and pgAdmin its worked and updated the rows as well.
so i thought the issue is the connection between metabase and PostgreSQL,
then i tried to change the connection propertes as below way.
I have tried below arguments as well.
default_transaction_read_only=off
readonlymode=ignore
readonlymode=off
but im still geting the same error message.
Metabase isn't geared toward modifying any data in the source databases it knows about.
Seemingly as a way to enforce this usage pattern, Metabase's developers are using the setReadOnly method when creating JDBC connection objects which is likely overriding any driver-level properties you've set. (This happens in the file src/metabase/driver/sql_jdbc/execute.clj in the Metabase codebase.) You could attempt to change this by compiling your own custom version of Metabase, but you might achieve some unexpected results.

You have specified a database that is not empty, please specify an empty database

I'm trying to connect to an RDS cluster in AWS that's an Aurora PostgreSQL database. It's a brand-new database that I created along with the instances that I have Jira deployed to. However, when I try to connect to the instance from the Jira configuration screen I get this error
You have specified a database that is not empty, please specify an empty database.
I haven't touched this database at all, why is it giving me this error? I have one read and one write database in my cluster and the "hostname" is the endpoint for my write database, which is what the docs say. Could this be an issue with the Jira version I'm using?
This is the download link I'm using in my user-data script to install Jira. I'm also using PostgreSQL version 12.11
https://www.atlassian.com/software/jira/downloads/binary/atlassian-servicedesk-4.19.1-x64.bin
I switched to a different PostgreSQL version and now it's working.
PostgreSQL version 12.11 was giving me the error and switching to version 13.7 works as expected.

Is there a way to use Flyway on AS400?

I need to implement migration tool like Flyway in order to use Jenkins to deploy DB changes.
I tried to add jt400.jar file and added configuration as follows:
flyway.url=jdbc:as400://192.168.171.251:446/DBDEV
flyway.driver=com.ibm.as400.access.AS400JDBCDriver
as a driver and it would not connect with this message:
ERROR: No database found to handle jdbc:as400://192.168.171.251:446/DBDEV
I also tried with using IBM DB2 driver and had configuration
flyway.url=jdbc:db2://192.168.171.251:50000/DBDEV
flyway.driver=com.ibm.db2.jcc.DB2Driver
this time I am getting this kind of refusal message
ERROR:
Unable to obtain connection from database (jdbc:db2://192.168.171.251:50000/DBDEV) for user 'DEVUSER':
[jcc][t4][2043][11550][4.26.14] Exception java.net.ConnectException: Error opening socket to server
/192.168.171.251 on port 50,000 with message: Connection refused (Connection refused).
ERRORCODE=-4499, SQLSTATE=08001
With this test migration I am trying to create a simple table by executing this sql
CREATE TABLE PERSON (
ID INT NOT NULL,
NAME VARCHAR(100) NOT NULL
);
Anyone had this situation and solved it?
I believe that at present there is no support for flyway to work with IBM i (as/400) regardless of whether you use jt400.jar or an IBM jdbc driver.
You can either use a different database-schema versioning tool, or find a fork of flyway that supports i-series (or pay someone to create and support such a fork, it is open source...).
It seems that currently (flyway 7.7.2) does not recognize a URL that starts with "jdbc:as400:" as a Db2 URL, so it throws an exception, which is the reason that the jt400.jar style URL is rejected with exception:
"No database found to handle ..."
The github history tells a story (see: https://github.com/flyway/flyway/issues/105).
Looks like the devs did not succeed to get the AS400 support added due to the lack of a suitable available i-series testing/dev environment (and also available to travis ci) . There may have been at least one PR for such support in the past, although it seems to be removed.
If you try to use the IBM db2jcc4.jar driver to connect to i-series (as400) with a url similar to: jdbc:db2://hostname/dbname, and you explicitly use an IBM jre , and have the relevant license file (e.g. db2jcc_license_cisuz.jar on the CLASSPATH), then flyway will connect and then report the exception similar to:
Unsupported Database: AS 7.4
The flyway source code shows that flyway does not recognize this database product-name and version, at current flyway version 7.7.2.
Are you sure DBDEV is the name of your Db2 data base on the IBM i?
Use the Work with RDB Directory Entry (WRKRDBDIRE) from the green screen, and look for the *LOCAL entry.
Or use the Access Client Solutions (ACS) "Schemas" tool to see a list of DB on your system.
The above shows 2 DB's, UT29p63 and Dbtest

Connecting to PostgreSQL Data Source in SQL Server Import Export Tool

I'm trying to setup an easily-replicable (or even manual and I do it once a month or so) process for moving data from a large Azure PostgreSQL database to a more manageable Azure SQL database for end users that are most familiar with SQL Server. I've successfully connected to the PostgreSQL database via PGAdmin, so I know all my connection string info.
I started by installing the latest ODBC driver from here.
I then used a connection string which was given to me from the Azure portal, filled in the proper database name and password, and attempted to use the following drivers:
PostgreSQL ODBC Driver(UNICODE)
PostgreSQL ODBC Driver(ANSI)
I am getting the following error with either of them:
ERROR [IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified
What step am I missing in this process? Or how best can I troubleshoot this?
After more research, I attempted to add the ODBC driver here:
And got the following error (I'm not sure why Tableau is relevant to this?):
Thank you.

After restarting DB2 service, the application server gets ERRORCODE=-4499, SQLSTATE=58009 in database connections

We have an application on IBM WebSphere Application Server 7.x and it connects to a remote database on z/os DB2 10.x. For annual operation, DB2 shut down and restarted. After starting the database, we first get
com.ibm.websphere.ce.cm.StaleConnectionException
and then we get
The database manager is not able to accept new requests, has terminated all requests in progress, or has terminated this particular request due to unexpected error conditions detected at the target system. ERRORCODE=-4499, SQLSTATE=58009
The connection between WebSphere and DB2 tested by 'test Connection' in WAS datasource. Both systems are up and running but there is no correct connection between them! There was no change in DB2, WAS, and JDBC driver.
Update: The JDBC driver version is 4.15.134, connection properties is IBM WebSphere default setting and the connection is direct to DB2. Another problem later showed that while the connection still has the problem, executing the query directly on z/OS's DB2 gets the same the error. The query consist of a select with a join on two different tables, selecting on each table is ok, but the final query does not work and gets ERRORCODE=-4499, SQLSTATE=58009.
Update 2
The detail of environment is: IBM WebSphere Application Server 7.0.0.45, DB2 10.1, Java version 1.6 SR16 and z/OS 1.13.
This specific query gets the error in all environments, on all application server, z/os SPUFI, database viewer, such as DBeaver.
Any help is greatly appreciated.
Finally, we found the solution, ran REORG and RUNSTATS on both tables and on all their partitions, and the error vanished both on the application and SPUFI. I guess something went wrong during restart and tables corrupted. Now everything is ok.
If I got you correctly, you complain on inability of the driver to reestablish the database connections after the DB2 for Z/OS restart.
If yes, then have you tried to set the corresponding connection properties described at the following link?
Configuration of Sysplex workload balancing and automatic client reroute for Java clients