I would like to build a Google Cloud PostgreSQL database using the instructions here
I was able to successfully create the Postgres databases with appropriate tables and views locally.
What do I need to do in order to get the data on Google Cloud PostgreSQL? My goal is to have remote access to this data.
You have 2 options, The first one is use the Cloud SQL proxy as is described here. As the shared links say, the Cloud SQL Proxy provides secure access to your instances without the need for Authorized networks or for configuring SSL.
On the other hand, the second option is only to configure access to your instance under Authorized networks using or not SSL. The complete steps are listed here
You could connect to Cloud SQL from a local test environment using cloud sql proxy. See quickstart-proxy-test.
The workflow is:
Your Application(Running Locally) => cloud sql proxy (Running locally) => GCP remote Cloud SQL service
Related
This question is about infosec, data privacy, specifically HIPAA compliance on GCP.
Is there any advantages for self managing Postgres server (built on GCP Compute instances using lets say Terraform) my own Vs using the managed offering, i,e. Cloud SQL
Thanks in advance
Google Cloud SQL Postgres is a fully managed option for deploying PostgreSQL to Google Cloud. The fully managed option is convenient, but is mainly suitable for cloud-native applications, or applications rebuilt for the cloud.
It has Built-in encryption for database tables, temporary files, backups, and any data transferred over Google’s internal networksSecure connections via SSL/TLS or the Cloud SQL Proxy.
Update1
As you are referring to HIPAA You can check this guide for HIPAA Compliance on Google Cloud Cloud sql encrypts the data at rest using the 256-bit Advanced Encryption Standard (AES-256), or better, with symmetric keys: that is, the same key is used to encrypt the data when it is stored, and to decrypt it when it is used. You can use your own encryptions as well with CMEK for cloud sql
And also you mentioned Infosec. I have not completely understood the term. I assume that you are referring to securing information from vulnerabilities. You can use Cloud Armor, which is a network security service that provides defenses against DDoS and application attacks like cross-site scripting (XSS) and SQL injection (SQLi).
Self hosted Postgres gives you full control over your PostgreSQL database on GCP, letting you to fine-tune server parameters, modify database configuration, and tune performance, just like in a local deployment.
Update2
As per this thread, it seems like postgresql is not HIPAA compliant.
For Encryption at rest on postgresql use can PostgreSQL TDE and Pgcrypto as discussed in this similar thread
For self hosted postgres You can also use shielded VM using which you can protect enterprise workloads from threats like remote attacks, privilege escalation, and malicious insiders
I am not sure on your application requirement, But based upon my
understanding about both cloud sql and self hosted postgres I
would recommend considering cloud sql as the best option as it is
fully managed by google and also complies with HIPAA and encryption.
For more information about pros and cons of Google Cloud SQL Postgres and Self hosted Postgres, Check this document
I have been trying to connect the my Data Fusion instance with my Cloud SQL PostgreSQL server. I have ensured that both of them are in the same VPC and I am using the Socket-factory library 1.2. What should I do? I am new in Data Fusion.
Edit:
I forgot to mention the error. I am trying to connect through wrangler using the Database option and using the connection string:
jdbc:postgresql://google/DATABASE_NAME?cloudSqlInstance=INSTANCE_CONNECTION_NAME&socketFactory=com.google.cloud.sql.postgres.SocketFactory&useSSL=false
The error I receive is 403 Forbidden. I have given Cloud SQL client permissions to both the service account and Dataproc account. I am using the latest Data Fusion version 6.6.0.
I'm making a Flutter app with a login page, and I've been trying to query data from a Google Cloud MySQL server and serve it to my app. I was able to replicate the result I want in python using bigquery but I need something like that but in a dart/flutter version(any other ways work too as long as the result is the same)
Any help would be really appreciated!
It is not recommended to connect to the Cloud SQL databases directly over the internet from the client application. So one should avoid connecting to the Cloud SQL database directly from the Flutter Application. In this case, it would be a good idea to create an API endpoint to do the operations in Cloud SQL and hit that endpoint from the Flutter Application.
To achieve the above, we can follow these steps:
Deploy a Http Cloud Functions which connects to the Cloud SQL
database
In this step we have to deploy a Http Cloud Functions and from the Cloud Functions we can connect to the Cloud SQL database. Inside the Cloud Functions we can do the operations on the Cloud SQL database. This document explains how to connect to Cloud SQL from Cloud Functions.
Call the Http Cloud Functions URL from the Flutter Application
After deploying the Cloud Functions, we will get a unique URL for it, which looks like this https://GCP_REGION-PROJECT_ID.cloudfunctions.net/FUNCTION_NAME. We have to call this URL from the Flutter Application. This blog explains how to call the Cloud Functions URL from the Flutter Application.
I'm following the instructions on how to connect from GKE to Cloud SQL: https://cloud.google.com/sql/docs/postgres/connect-kubernetes-engine
It talks about YOUR-GSA-NAME. Google cloud creates "Compute Engine default service account" by default. Should I pick this one or create another service account for GKE only? What is the recommended way?
The Compute Engine default service account won't be able to connect to Cloud SQL out of the box, you'll have to add permissions to it (Cloud SQL Client role) for it to be able to connect.
I would create a new one however, as you likely don't want all GCE instances to be able to connect to Cloud SQL, and for permissions, best practice is to limit access. So just create a new SA (service account) with the Cloud SQL Client role (and any other permissions you might need GKE to access) and use that one.
This is all found in IAM -> Service Accounts in the console.
I'm trying to use Cloud SQL from my VM instance.
When creating the VM Instance I activated Cloud SQL Option for it.
The Cloud SQL instance authorizes my Compute Engine Project to access it.
At first I was expecting to have some tools like google_sql.sh installed on my VM since I had activated Cloud SQL on it but no :-/
In Cloud SQL docs it says that I should copy my local access token to my VM Instance.
My local machine is Mac OSX so the tokens are stored in :
~user/Library/Preferences/com.google.cloud.plist
but on my Linux VM it's stored in:
~user/.java/.userPrefs/com/google/cloud/sqlservice/oauth2/prefs.xml.
Do I have to create a prefs.xml and copy it on my VM? (but I guess the XML schema is not the same between com.google.cloud.plist and prefs.xml?)
Does someone have perfs.xml example I could use as a template (unless schema is exactly the same as com.google.cloud.plist which I doubt)?
Thanks all for your help.
The simplest thing is actually to include service account scopes when you create your instance. This page in the compute engine docs describes how to do it. This maintains an access token in the compute engine instance's metadata server which the Cloud SQL tools can then access when they need to authenticate. A similar technique works for cloud storage and other products.