I have a Compose for Postgresql service on IBM Bluemix which isn't allowing me run PostGIS functions on my cloud foundry rails app. I have run "CREATE EXTENSION PostGIS;" and I have also added the adapter to database.yml. Compose for Postgresql says PostGIS comes installed by default
I am using Ruby on Rails with the rgeo gem and the error is
ERR NoMethodError: undefined method `st_point' for #
Can you please let me know if there is anything I need to do to get PostGIS working?
Please raise a support request asking for the postgis plugin to be enabled on your compose instance.
Answered my own question. The problem was with the rgeo gem and the adapter. I needed the postgis:// adapter for working with the gem.
Bluemix does not allow you to change the adapter in their connections. It will always be postgresql. To get around this I set a CUSTOM_DATABASE_URL environment variable with the connection string postgis://<username>:<password>#host:port/<db_name>. Using the cf client this would look like
cf set-env <app-name> CUSTOM_DATABASE_URL postgis://<username>:<password>#host:port/<db_name>
Then in the command for my container in the manifest.yml, I prepended setting DATABASE_URL = CUSTOM_DATABASE_URL, specifically
DATABASE_URL=$CUSTOM_DATABASE_URL &&.....
Its a workaround for now until Bluemix allows us to change the adapter in the connections.
Related
I was using Postgres addon on Heroku and was connecting to the database using DATABASE_URL env var.
I now need to switch to Google Cloud Platform Postgres. I've created an instance and successfully added my local connection to the Authorised networks. Yet it seems that Heroku does not provide a static IP for its apps.
My question is then – is it possible to connect my Heroku app to a Postgres database from the Google Cloud Platform? If yes – what's the best way to do it?
You will want to run the Cloud SQL Proxy alongside your application. This will allow your Heroku App to connect to Cloud SQL without the need to worry about changing and adding IPs to your Authorized Networks.
This thread might be useful for your use-case. Node and Cloud SQL with Heroku
I need to poll DB2 database table until a record is created and once it is successful,i need to trigger a task in dag.
For this I have to do below:
Create a DB2 connection in airflow
Use SQL sensor to poll db table using the connection created in above step.
But I don't see the connection type option for DB2 (screenshot attached). Do I have to install something? Please help.
PS: I am new to Airflow.
There is no DB2 specific connection.
For general connections you can just use the Generic connection:
For older Airflow versions that doesn't have Generic connection you can use any other connection type (HTTP/MySql for example). It doesn't really matter. Airflow looks for connection by the connection id. The type is almost meaningless in that perspective.
Yes the airflow connection does not appear to be a specific connection for db2, so you have to choose generic in connection Type.
I also recommend to install the security mechanism for connection.
the next package: pip install airflow-provider-db2
I have tried to connect to DB2 successfully on Airflow 2.2.5.
Airflow doesn't has specific provider for DB2. You need to use JDBC to connect.
So you need to install some libraries of Airflow and download DB2 JDBC driver to fulfill this task.
List of all libs and provider need to install below:
Apache-airflow-provider-jdbc/Version:2.1.3
JayDeBeApi/Version:latest
JPype1/Version:latest
You also need to download DB2 JDBC drivers. This driver include some packages and license file like:
db2jcc4.jar
sqlj4.zip
jdbc4_LI_en
Because it will use JDBC to connect,openJDK also need to install on your docker.
Setting environment variables JAVA_HOME, PATH and CLASSPATH are needed.
All of above actions can be done on your docker file.
When you have completed to build your image, You can start your airflow to set connection.
Connection URL syntax: jdbc:db2://Host:Port/Database
You can replace HOST,PORT and Database.
Driver Path: You can input where your class file(db2jcc4.jar) located.
Hope these tips can help you. I am New to Airflow too.
Is there an official way of installing an extension on a GCP Postgress Cloud SQL instance via Terraform?
Closest I've found is this unofficial Postgres resource but it's not immediately clear how to link the two. This issue on their tracker sort of shows how, but far from a step by step guide.
if it makes any difference, I'm trying to provision a Postgres Cloud SQL instance with PostGIS.
Thanks.
Terraform is a deployment tool, to create all your infrastructure. To install an extension on Postgres, you need a installation tool, because you need to connect to the database and to run a command.
It's the same thing if you want to create a user in the database and you want to grant some privileges on it.
In summary, you can't achieve that with Terraform. I recommend you to have an installation tool, such as Ansible to perform this action.
An alternative is to create, with Terraform, a micro VM with a startup script that connect the database, run the command and destroy itself at the end.
I installed Orange and I have data in a local PGSQL server.
PGSQL listens on the default port which is 5432.
I have the psycopg2 lib installed, and I also wrote a short python script which pulls some data from the database to check the module is insatlled correctly.
Firewall is down.
Python Env Path is set to use 3.4.4 which is what Orange3 uses.
When I add a sql table widget, I get an error suggesting "please install a backend to use this widgt"
Documentaion in Orange site mentions that all that needs to be done for the DB integration is just installing the python module, but this doesnt work for me.
Help would be appreciated.
Links:
https://docs.orange.biolab.si/3/visual-programming/widgets/data/sqltable.html
I am building a Django site on Google Compute Engine, and I want to install my database in SQL Cloud. It is possible?
What is the most common way to do this? Installing MySQL on virtual machine or use a Cloud SQL instance?
Thank you.
You can use either Google Cloud SQL or manage your own SQL database, depending on your needs.
To use Cloud SQL, you'd want to follow the instructions here: https://developers.google.com/cloud-sql/docs/external
If you want to manage your own SQL database, you can install MySQL or some other database on an instance. Depending on your needs, you can start with a g1-small with a fairly large disk attached and then later use a larger instance type to run your database.
If you're running your own database, you'll need to make sure to take regular backups and copy them off the database machine, to someplace like Google Cloud Storage. If you're using Cloud SQL, you can use the console or the API to schedule database backups.
This answer is following up from "Well, the problem is that to use Cloud SQL, I must connect using JDBC. I'm using Python. How I can do?"
I am not from Python world, but I recently connected my Java app on GCE instance to a Cloud-Sql DB (via cloud-sql-proxy approach, as described here: https://cloud.google.com/sql/docs/compute-engine-access) and didn't see any reason why it shouldn't work for Python too.
Here is what I just tried and easily connected my test Python app to a Cloud-Sql DB, via the cloud-sql-proxy:
Step 1: Download and run the proxy on a local port, like below (this establishes a channel between the local port 3306 and the Cloud-SQL database instance identified by the connection name "PROJ_NAME:TIMEZONE:SQL_NAME"):
sudo wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64
sudo mv cloud_sql_proxy.linux.amd64 cloud_sql_proxy
sudo chmod +x cloud_sql_proxy
sudo ./cloud_sql_proxy -instances=PROJ_NAME:TIMEZONE:SQL_NAME=tcp:3306 &
Step 2: Make sure that python-mysqldb is installed
sudo apt-get install python-mysqldb
Steo 3: Ran the following test program to connect to the Cloud-SQL db, via the local socket 3306, setup by the proxy:
import MySQLdb
conn = MySQLdb.connect(host= "127.0.0.1", port=3306, user="root", passwd="my_root_password", db="my_db")
x = conn.cursor()
try:
x.execute("""INSERT INTO Test(test_id) VALUES ('111')""")
conn.commit()
except:
conn.rollback()
conn.close()
Hope it helps.