I am using data frame to write into the database using a JDBC connection but neither the data is getting populated nor I am getting any error.
Any direction from where I should start the debugging?
Code line being used:
df.reparition(8).write.mode('append').jdbc(url, table, properties,)
I have tried the python lib psycopg but with this jdbc connection it is not working
Related
I'm using the open source Metabase version, and I was able to connect my PostgreSQL databases correctly.
but when im trying to run a update query from metabase its throws below error.
ERROR: cannot execute UPDATE in a read-only transaction
but when I execute same query from terminal and pgAdmin its worked and updated the rows as well.
so i thought the issue is the connection between metabase and PostgreSQL,
then i tried to change the connection propertes as below way.
I have tried below arguments as well.
default_transaction_read_only=off
readonlymode=ignore
readonlymode=off
but im still geting the same error message.
Metabase isn't geared toward modifying any data in the source databases it knows about.
Seemingly as a way to enforce this usage pattern, Metabase's developers are using the setReadOnly method when creating JDBC connection objects which is likely overriding any driver-level properties you've set. (This happens in the file src/metabase/driver/sql_jdbc/execute.clj in the Metabase codebase.) You could attempt to change this by compiling your own custom version of Metabase, but you might achieve some unexpected results.
I'm trying to setup an easily-replicable (or even manual and I do it once a month or so) process for moving data from a large Azure PostgreSQL database to a more manageable Azure SQL database for end users that are most familiar with SQL Server. I've successfully connected to the PostgreSQL database via PGAdmin, so I know all my connection string info.
I started by installing the latest ODBC driver from here.
I then used a connection string which was given to me from the Azure portal, filled in the proper database name and password, and attempted to use the following drivers:
PostgreSQL ODBC Driver(UNICODE)
PostgreSQL ODBC Driver(ANSI)
I am getting the following error with either of them:
ERROR [IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified
What step am I missing in this process? Or how best can I troubleshoot this?
After more research, I attempted to add the ODBC driver here:
And got the following error (I'm not sure why Tableau is relevant to this?):
Thank you.
I had configured the JDBC connection configuration in the pipeline.
and when the application executes i get the following error on the logs.
"java.sql.SQLSyntaxErrorException: Table 'databaseName.aim_table' doesn't exist"
The databaseName is not what I have set.
I have tried many times. it shows the same message that could not find the table in different database, and the question is all the db occurred in the sdc.log are that I had never configured ,and the correct database is never used ,so I want to know how could it find the wrong db and I had checked before start the pipeline and it shows successful:
Do you have anything set in the Schema Name configuration for JDBC
Producer? This should be blank for MySQL, since you're setting the
database/schema name in the connect URL.
Check that your MySQL driver matches the server. In particular, using
the current version 8.0.x JDBC driver with a 5.x.x server seems to
result in this problem. Download the older 5.1.x driver (currently
5.1.46) and it should work.
refer this
This problem is indeed caused by the wrong version of the driver package. I found the correct driver package and successfully wrote the data to the target table. add aonther point, I have set the SCHEMA NAME to blank and defined the database name in the connect URL for mysql.
My English is not good. Please forgive me.
I'm new to postgres so this problem is probably a relatively easy one for someone else. However, I have spent many frustrating hours trying to figure out the solution. I have an Access Database of metadata that must be kept updated for sending records to other groups. I also have a database using PostgreSQL and PGAdmin which also has these same metadata tables. Currently these tables in the Postgres database get updated manually by exporting the Access tables as excel files, and then importing them into the SQL tables. It's not the most efficient process and could lead to errors in the SQL database if someone forgets to check before running any queries that they are using the most recent data from Access. So I would like to integrate some of the tables from my Access database with my Postgres database.
Originally I tried just installing drivers to export the Access tables directly to Postgres which worked, but not in the way that I wanted since it just brings in a table which I would still need to manually update. From my understanding, I can create a server connection in postgres to access and that would then bring in updated data using a foreign data wrapper.
I tried to use ogr_fdw.
CREATE EXTENSION ogr_fdw;
When I try:
CREATE SERVER metadata
FOREIGN DATA WRAPPER ogr_fdw
OPTIONS (
datasource 'H:\Databases\20170712.accdb',
format 'ODBC' );
I receive: ERROR: unable to connect to data source "H:\Databases\20170712.accdb"
SQL state: HV00D
When I try:
CREATE SERVER metadata
FOREIGN DATA WRAPPER ogr_fdw
OPTIONS (
datasource 'H:\Databases\20170712.accdb',
format 'ACCDB' );
I receive: ERROR: unable to find format "ADDCB"
HINT: See the formats list at http://www.gdal.org/ogr_formats.html.
I also tried MDB and received the same error. However, MDB is the code name given by the website but it says that it needs JDK/JRE to compile and I'm not really sure if that's another type of driver that I would need or what it is.
When I try:
CREATE SERVER metadata
FOREIGN DATA WRAPPER ogr_fdw
OPTIONS (
datasource 'H:\Databases\20170712.mdb',
format 'ODBC' );
I receive: ERROR: unable to connect to data source "H:\Databases\20170712.mdb"
SQL state: HV00D
Hint: Unable to initialize ODBC connection to DSN for DRIVER=Microsoft Access Driver (*.mdb);DBQ=H:\Databases\20170712.mdb,
[Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified
However I thought after looking at the github help page for ogr_fdw didn't need ODBC and special drivers https://github.com/pramsey/pgsql-ogr-fdw/blob/master/FAQ.md.
A lot of this is probably due to my limited knowledge of the terminology when I'm reading through a lot of this stuff. Also my Access database is an .accdb file but since that wasn't working I tried around with mdb and ODBC as the "format" too. If anyone has any suggestions I would greatly appreciate it.
Thanks!
I am trying to setup a database connection (to neo4j) using its JDBC driver. I am using Database Explorer within 2014b.
I get the error message "Unable to find JDBC driver". I have tried adding the driver using javaaddpath ('D:\driver path\filename.jar') and then using any combination of this in the "Driver" field in Connection Parameters. I have also tried "edit classpath.txt" as suggested elsewhere but I can't save it (Access Denied).
So any suggestions?