I'm using tomcat connection pool org.apache.tomcat.jdbc.pool.DataSource. The connections appear in my database pg_stat_activity with empty application_name.
How could I set that application name in my java app, so I know where each connection comes from (as there will be multiple applications accessing the same db)?
You could specify the application name in the connection string.
Documentation here.
Example:
jdbc:postgresql://localhost:5435/DBNAME?ApplicationName=MyApp
Take care: the param names are case sensitive.
Use set command:
set application_name to my_application;
You can add this to the JDBC URL that you use in your connection pool definition:
jdbc:postgresql://localhost/postgres?ApplicationName=my_app
If you want to change it dynamically e.g. to reflect different modules inside your application, the you can use the SET command as shown by klin.
If you want to set programmatically, the PostgreSQL drivers also have a method you can call:
PGPoolingDataSource ds = new PGPoolingDataSource();
ds.setApplicationName(applicationName);
For those using normal connection string and not jdbc:
postgresql://other#localhost/otherdb?application_name=myapp
If you're using Python library psycopg2, here's how you can do
import psycopg2
psycopg2.connect(host=host_ip, database=db_name, user=user, password=db_pwd, application_name="Python Local Test Script")
Related
I need to poll DB2 database table until a record is created and once it is successful,i need to trigger a task in dag.
For this I have to do below:
Create a DB2 connection in airflow
Use SQL sensor to poll db table using the connection created in above step.
But I don't see the connection type option for DB2 (screenshot attached). Do I have to install something? Please help.
PS: I am new to Airflow.
There is no DB2 specific connection.
For general connections you can just use the Generic connection:
For older Airflow versions that doesn't have Generic connection you can use any other connection type (HTTP/MySql for example). It doesn't really matter. Airflow looks for connection by the connection id. The type is almost meaningless in that perspective.
Yes the airflow connection does not appear to be a specific connection for db2, so you have to choose generic in connection Type.
I also recommend to install the security mechanism for connection.
the next package: pip install airflow-provider-db2
I have tried to connect to DB2 successfully on Airflow 2.2.5.
Airflow doesn't has specific provider for DB2. You need to use JDBC to connect.
So you need to install some libraries of Airflow and download DB2 JDBC driver to fulfill this task.
List of all libs and provider need to install below:
Apache-airflow-provider-jdbc/Version:2.1.3
JayDeBeApi/Version:latest
JPype1/Version:latest
You also need to download DB2 JDBC drivers. This driver include some packages and license file like:
db2jcc4.jar
sqlj4.zip
jdbc4_LI_en
Because it will use JDBC to connect,openJDK also need to install on your docker.
Setting environment variables JAVA_HOME, PATH and CLASSPATH are needed.
All of above actions can be done on your docker file.
When you have completed to build your image, You can start your airflow to set connection.
Connection URL syntax: jdbc:db2://Host:Port/Database
You can replace HOST,PORT and Database.
Driver Path: You can input where your class file(db2jcc4.jar) located.
Hope these tips can help you. I am New to Airflow too.
Using ODBC Data Source Administrator (32-bit), I have set up a User Data Source configured with the PostgreSQL ANSI driver. Then, I use that DSN to create a Linked Server in SSMS.
I want to link another server in SSMS, also using a PostgreSQL ANSI driver, but when I go through the steps to Add the second DSN, I am returned to the configuration page for the existing DSN using that driver.
Naturally, each linked server has a unique IP address, so how do I Add a second Data Source instance in ODBC Data Source Administrator, using the same driver as the first instance?
After reading up on Microsoft's Documents pages, the missing piece became clear. I needed to name each instance uniquely. Inside the DSN driver setup/configuration window, I populated the "Data Source" field with a distinct name, even while each instance used the same PostgreSQL ANSI driver.
The following link is where I started. Even though it doesn't explicitly go into detail for this particular use case, it planted the seed of this idea.
https://learn.microsoft.com/en-us/sql/odbc/admin/managing-data-sources?view=sql-server-ver15
I had configured the JDBC connection configuration in the pipeline.
and when the application executes i get the following error on the logs.
"java.sql.SQLSyntaxErrorException: Table 'databaseName.aim_table' doesn't exist"
The databaseName is not what I have set.
I have tried many times. it shows the same message that could not find the table in different database, and the question is all the db occurred in the sdc.log are that I had never configured ,and the correct database is never used ,so I want to know how could it find the wrong db and I had checked before start the pipeline and it shows successful:
Do you have anything set in the Schema Name configuration for JDBC
Producer? This should be blank for MySQL, since you're setting the
database/schema name in the connect URL.
Check that your MySQL driver matches the server. In particular, using
the current version 8.0.x JDBC driver with a 5.x.x server seems to
result in this problem. Download the older 5.1.x driver (currently
5.1.46) and it should work.
refer this
This problem is indeed caused by the wrong version of the driver package. I found the correct driver package and successfully wrote the data to the target table. add aonther point, I have set the SCHEMA NAME to blank and defined the database name in the connect URL for mysql.
My English is not good. Please forgive me.
How can I set the application name when connection to Postgres with the psqLODBC driver?
You get the application name from pg_stat_activity.application_name.
I tried different Parameters in the connection string.
APP=...; application_name=...; APP_NAME=... in all different variations.
Nothings works and pg_stat_activity.application_name always returns an empty string.
Once you have established a connection in your application, you can run a SQL statement to change the application_name dynamically.
set application_name = 'doev';
You run that just like any other statement (that does not return a result) with your programming language. But remember if you have turned off autocommit in your connection, you need to commit that SET statement. If you are running in autocommit mode, that is not necessary.
I want to set the application name of the connections of my application. So when I list the rows in pg_stat_activity I can have a non empty application_name column.
I have setup the following JDBC url for connecting to my Postgresql database:
jdbc:postgresql://localhost:5435/MyDB?application-name=MyApp
I have tried also this url with no more success.
jdbc:postgresql://localhost:5435/MyDB?application_name=MyApp
What is the correct parameter name ?
Here is my JDBC driver version: 9.1-901.jdbc4
Looking at the PostgreSQL JDBC 9.1 documentation, connection parameters, the correct property name in the JDBC url is ApplicationName:
ApplicationName = String
Specifies the name of the application that is using the connection. This allows a database administrator to see what applications are connected to the server and what resources they are using through views like pg_stat_activity
So try:
jdbc:postgresql://localhost:5435/MyDB?ApplicationName=MyApp
Be aware some comments suggest that this is broken in the 9.1 version driver. Given it is a more than 5 year old version, you should upgrade to a newer version anyway. Check https://jdbc.postgresql.org/ for the latest version and use that.