Connecting to Cloud SQL MySQL - google-cloud-data-fusion

Question

We would like to test connecting Cloud SQL (mySQL) to BigQuery using Cloud Data Fusion. What is the proper way to connect to CloudSQL as that does not appear to be "build in" at this point in time. What driver is recommended and are there any instructions available?

Answer 1

Here are instructions to use Cloud SQL MySQL in Data Fusion. Note that in the Wrangler section, currently, Cloud SQL instances with Private IP cannot be used. However, they can still be used when running Data Fusion pipelines
Using Cloud SQL (MySQL) in Wrangler (Public IP only)
Obtain the JDBC Driver JAR file by building it using the instructions at https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory
Go to Wrangler
If this is the first time you are configuring CloudSQL for MySQL, click on the Add Connection button from the Wrangler screen and choose Database.
Click “Google Cloud SQL for MySQL.”
Upload the previously built JAR as illustrated, and click the Next button.
Click the Finish button to complete the upload.
Once the driver has been uploaded you will see a green check mark indicating that your driver has been installed.
Click the Google Cloud SQL for MySQL to create a new connection. Once the connection modal opens click on the Advanced link if present.
Enter connection string as
jdbc:mysql://google/<database>?cloudSqlInstance=<instance-name>&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useSSL=false
where represents the database you created in the prerequisites section, and refers to you instance connection name as displayed in the overview tab of of the instance details page, e.g:
Example:
jdbc:mysql://google/mysql?cloudSqlInstance=cloud-data-fusion-demos:us-west1:mysql&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useSSL=false
Enter the username and the password you configured for this CloudSQL instance
Click Test Connection to verify that the connection can successfully be established with the database.
Click Add Connection to complete the task.
Once you’ve completed all the steps you will be able to click on the newly defined database connection and see the list of tables for that database.
Using Cloud SQL (MySQL) in Pipelines (Public and Private IP)
Perform steps 1-6 in the Wrangler section above
Open the pipeline Studio
From the plugin palette on the left, drop the Cloud SQL source plugin to the canvas, and open it by clicking Properties.
Specify the plugin name as cloudsql-mysql (Presumes that you have perform.
Specify the connection string as below:
jdbc:mysql://google/?cloudSqlInstance=&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useSSL=false
where represents the database you created in the prerequisites section, and refers to you instance connection name as displayed in the overview tab of of the instance details page, e.g.:
jdbc:mysql://google/mysql?cloudSqlInstance=cloud-data-fusion-demos:us-west1:mysql&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useSSL=false
Enter the query that you would like to import data from as the Import Query
Enter the username and password to use for the database. You can also use a secure macro for the password.
Click Get Schema to populate the schema of the plugin.
Configure the rest of the pipeline, and deploy.