I am trying to connect my thingworx to azure postgres database. I have two azure accounts. When I am creating my postgresql database in one of the accounts and connecting then it is working fine.
But when I am trying to connect to the azure postgres database present in the second account , the connection is failing and I am getting the error as follows:
Unable to Invoke Service GetStudentData on Database_Functions : FATAL: Client from Azure Virtual Networks is not allowed to access the server. Please make sure your Virtual Network is correctly configured.
Apparently, the postgresql is part of a Virtual Network service endpoint and a service endpoint tag was enabled.
To solve the problem disable the service endpoint and add the public IP to the Connection Security section.
It seems a firewall rule problem, not at all related to TW itself.
Related
I have cloud postgres setup on GCP with public IP enabled. Previously, I have been able to connect via google sql proxy but now I am getting following error:
errors parsing config:
googleapi: Error 503: Policy checks are unavailable., backendError
If I try to list sql instances using gcloud sql instances list that also throws same error. As per https://status.cloud.google.com/ there is no disruption in any services.
Postgres server is running with no issues and other tools accessing the db directly are also running, the issue with accessing the db locally using google cloud proxy.
I have a Google Cloud SQL PostgreSQL database in which I can connect by using SSL and by entering my IP address in allowed connection settings. However, I do not want to list all the IP addresses that is going to connect to this database (because I do not know all the IP addresses). I have around 15 people which I want them to login to my database using QGIS and they should be able to change the data as this is a research. Security is not a big issue as this database will be online for a very short period of time. What connection method can you suggest? The users are not very proficient so I need to setup everything.
I hope you're doing fine.
I would like to suggest to set the connections with the Cloud SQL proxy as it will provide the security needed without using ssl or the need of authorize any network. so basically the set up is to:
Enable the API
Install the proxy client on your local machine
Determine how you will authenticate the proxy
If required by your authentication method, create a service account
Also you can find the steps on "Connecting to Cloud SQL from external applications"
Hope this works for you as I have never used it with QGIS but I believe that as you are using a proxy it won't be hard from there to use it with QGIS as if you connected to a local server.
I have 2 GKE cluster both private and public and using cloudproxy as sidecar container for gke app to access cloudsql instance.
public cluster setup for development/testing
Cloud SQL is enabled with both private and public IP.
GKE app is using cloudproxy with default option of ip types (public,private) as below
Cloud SQL doesn't have any authorized network.
In this case, my app is able to connect CloudSQL and works smoothly. As far as I understand, here connection to cloudsql should be happening with private becuase there is no authorised network configured.
private cluster setup for production
Cloud SQL is enabled with both private and public IP.
GKE app is using cloudproxy with default option of ip types (public,private)
cloudsql-proxy setting in deployment file
- name: cloudsql-proxy
image: gcr.io/cloudsql-docker/gce-proxy:1.11
command: ["/cloud_sql_proxy"]
args: ["-instances=$(REAL_DB_HOST)=tcp:$(REAL_DB_PORT)","-credential_file=/secrets/cloudsql/credentials.json"]
case 1
Cloud SQL doesn't have any authorized network.
Result: Application is not able to connect with Cloud SQL
case 2
Cloud SQL have private GKE NAT gateway as authorized network
Result: Application is not able to connect with Cloud SQL
May be removing cloudproxy from application will work (I am yet to test) but it discourages the usage of proxy during dev env as it will need changes in deployment file during production deployment.
I am not able to understand what is causing the connection failure with cloudproxy in gke private cluster. Should we not use cloudproxy in private cluster?
Update
The reason due to which cloud proxy not able to connect cloud sql was disabled Cloud SQL Admin API. I have updated my answer in answer section.
It looks like the question here is "Should we use the Cloud SQL proxy in a private cluster?" and that answer is "it depends". It's not required to connect, but it allows for more security because you can restrict unnecessary access to your Cloud SQL server.
The Cloud SQL proxy doesn't provide connectivity for you application - it only provides authentication. It has to be able to connect via the existing path, but then uses the Service Account's IAM roles to authenticate the connection. This also means that it doesn't have to come from a whitelisted network because it's been authenticated by a different means.
If you want to use the proxy to connect via Private IP (instead of defaulting to public), use the -ip_address_types=PRIVATE - this will tell the proxy to connect with the instance's Private IP instead. (Please note that if the proxy lacks a network path (eg, isn't on the VPC) that proxy will still be unable to connect.)
#kurtisvg has provided an informative answer to it.
However the real issue was SQL Admin API and enabling it fixed the issue. After looking into the logs I found below entry.
Error 403: Access Not Configured. Cloud SQL Admin API has not been used in project XXXXXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/sqladmin.googleapis.com/overview?
The issue for me was enabling Private cluster in GKE cluster :(
Because of private GKE cluster it wasn't having access to external IP addresses and fix was to create a NAT gateway with cloud router as per https://cloud.google.com/nat/docs/gke-example.
Hint if it's the issue is you won't be able to ping to google.com etc from the container after logging into it.
I can't connect from azure resource (aks node) to Azure postgres using pgcli. I also tried directly from node and got the same error message:
FATAL: Client from Azure Virtual Networks is not allowed to access the server. Please make sure your Virtual Network is correctly configured.
Firewall rules in the resource are on:
Allow access to Azure services: ON
Running the same pgcli login command on my computer and on another azure resource seems to work fine.
Adding Firewall rules to all IPs return the same error.
Curl from the problematic server (host:5432) returns a reply, so it's not an outbound issue.
What does the error mean?
A VM where the connection originates from is deployed to a virtual network subnet where Microsoft.Sql service endpoint is turned on. Per documentation:
If Microsoft.Sql is enabled in a subnet, it indicates that you only want to use VNet rules to connect. Non-VNet firewall rules of resources in that subnet will not work.
For connection to succeed there must be a VNet rule added on PostgreSQL side. At the time the question was asked VNet Service Endpoints for Azure Database for PostgreSQL just got to public preview so I assume it might not have been available for the OP.
Solution
As of November 2020, Service Endpoints for Postgres is GA and instead of disabling the service endpoint one can add a missing VNet rule to the PostgreSQL server instance and reference the service endpoint-enabled subnet. It can be done via Portal or Azure CLI
Apparently, the vm is part of a vnet that a service endpoint tag Microsoft.sql was enabled.
I found this answer. To solve the problem I disabled the service endpoint and added the public IP to the Connection Security section.
I encountered the same problem.
All I did was to switch Allow access to Azure services to ON .
I am trying to send email using one of our on-premises servers from one of my web roles hosted on azure. We've got a Windows Azure Connect endpoint installed on this on-premises server which has an SMTP server.
We've configured the web role so that it contains an activation code I acquired using the windows azure portal and the azure subscription we have. The web role has been deployed to azure with this configuration. Looking in the virtual network section of the portal I can see our on-premises server listed as well as the instance of said web role. I Created a group connecting the local endpoint to the web role instance.
The problem I'm having now is figuring out exactly what I have to do in order for the emails I send from the web role to be relayed through the smtp server on the on-premises server.
My first thought was to just specify the local endpoint name as it appears in our azure portal as the host to use when I create my SmtpClient object in code. Of course this didn't work as I received an SmtpException just saying Failure Sending Email.
So my question is once I've set everything up as described above, what do I need to do in ,my web role code and/or configuration in order to use the local endpoint as the smtp host for sending out my emails??
How about open your firewall for the SMTP on both your azure VM and local server.
As I know the azure VM firewall disabled the PING (ICMP) but doesn't know if it blocked all ports except those defined in your CSDEF file.