I have a VM instance (e2-micro) on GCP running with postgres. I added my own external ip address to pg_hba.conf so I can connect to the database on my local machine. Next to that I have a nodeJS application which I want to connect to that database. Locally that works, the application can connect to the database on the VM instance. But when I deploy the app to GCP I get a 500 Server Error when I try to visit the page in the browser.
These are the things I already did/tried:
Created a Firewall rule to allow connections on my own external ip address
Created a VPC connector and added that connector to my app.yaml
Made sure everything is in the same project and region (europe-west1)
If I allow all ip addresses on my VM instance with 0.0.0.0/0 then App Engine can connect, so my guess is that I'm doing something wrong the connector? I use 10.8.0.0/28 as ip range while the internal ip address of the VM instance is 10.132.0.2, is that an issue? I tried an ip range with 10.0.0.0 but that also didn't work.
First check if your app uses a /28 IP address range (see the documentation):
When you create a connector, you also assign it an IP range. Traffic
sent through the connector into your VPC network will originate from
an address in this range. The IP range must be a CIDR /28 range that
is not already reserved in your VPC network.
When you create a VPC connector a proper firewall rulle is also created to allow traffic:
An implicit firewall rule with priority 1000 is created on your VPC
network to allow ingress from the connector's IP range to all
destinations in the network.
As you wrote yourself when you create a rule that allows traffic from any IP it works (your app can connect). So - look for the rule that allows traffic from the IP range that your app is in - if it's not there create it.
Or - you can connect your app to your DB over public IP's - in such case you also have to create a proper rule that will allow the traffic from the app to DB.
Second - check the IP of the DB that app uses.
My guess is that you didn't change the IP of the DB (that app uses) and it tries to connect not via VPC connector but via external IP and that's why it cannot (and works only when you create a firewall rule).
This answer pointed me in the right direction: https://stackoverflow.com/a/64161504/3323605.
I needed to deploy my app with
gcloud beta app deploy
since the VPC connector method was on beta. Also, I tried to connect to the external IP in my app.yaml but that needed to be the internal IP ofcourse.
Related
I'm trying to configure network access of a MongoDB cluster to allow connections from an Azure App Service. I found the outbound IP addresses of my App Service in the Azure portal (see Azure docs). And entered them in the IP access list according to MongoDB Atlas docs. I appended "/32" to the IP addresses to allow only a single host (CIDR notation).
However, when trying to connect on App Service start I get an error indicating to check the IP whitelist of the MongoDB cluster.
This actually seems to be the problem, because adding 0.0.0.0/0 (allow access from anywhere) solves the problem.
What could be the problem here?
I double checked the outbound IP addresses of the Azure App Service and the IP access list from the MongoDB Cluster.
What I did was indeed the answer to another question, so I think I'm missing something...
Actually /32 is not a valid CIDR in Azure. The minimum size of a single VNET is /29.
This will restrict your range to only 3 IPs (not 8 as you would expect), as Azure will reserve the first four IPs and the last one for internal routing.
Please consider also that if you are running the MongoDB cluster inside a private network and it is not exposed externally via a network appliance (such as Application Gateway, Load Balancer, Front Door or Traffic Manager), you will need to enable VNET Integration on Azure Web App side.
If this is your case, navigate through your App in the portal and go into the "Networking" blade.
Here you can add VNET Integration, but you should consider that in this case the minimum size of your subnet can only be /28 (you cannot add a smaller subnet)
I only added the IP addresses listed in the "outbound IP addresses" property of my Azure App Service. After adding the IP addresses listed in the "Additional Outbound IP Addresses" property also the App Service connects to the MongoDB cluster successfully.
This is somewhat surprising to me because the documentation on when outbound IPs change says that the "...set of outbound IP addresses for your app changes when you perform one of the following actions:
Delete an app and recreate it in a different resource group (deployment unit may change).
Delete the last app in a resource group and region combination and recreate it (deployment unit may change).
Scale your app between the lower tiers (Basic, Standard, and Premium), the PremiumV2, and the PremiumV3 tier (IP addresses may be added to or subtracted from the set).
..."
None of the above actions happened. 🙄
I'd like to know in detail how to connect google compute engine virtual machine instance and app engine.
I've set up a virtual machine instance on Google compute engine, and my Postgres server is running there, following this tutorial: https://cloud.google.com/community/tutorials/setting-up-postgres
I've deployed my flask app under the same project on Google Cloud Platform, creating an app engine instance.
I searched on how to connect compute engine and app engine together, and it seems it should be possible through a VPC connector: connect Google App Engine and Google Compute Engine
This is what my VPC connector looks like:
Serverless VPC access
Name Network Region IP address range Min. throughput Max. throughput
connector-name default europe-west2 10.8.0.0/28 200 300
On my compute engine, I have my VM instance like so:
Name Zone Internal IP External IP
some-name europe-west2-c 10.154.0.2 (nic0) 34.89.113.193
On my flask app, I'm trying to connect to my remote DB like so:
db = PostgresqlExtDatabase(
"some-name", # databse name
user="postgres",
password="some-password",
host="10.154.0.2", # remote host internal ip
port=5432,
)
db.connect()
This is my app.yaml for the vpc access part, I've followed this reference: https://cloud.google.com/appengine/docs/standard/python/connecting-vpc#configuring
vpc_access_connector:
name: projects/some-name/locations/europe-west2/connectors/connector-name
If I understood correctly, if the VPC connector is present, I should just be able to connect using the internal IP address of my VM instance(this case, 10.154.0.2)?
The problem is, when the app is deployed for production, It is still complaining that it cannot connect:
2020-09-26 12:54:51 default[20200926t134815] Is the server running on host "10.154.0.2" and accepting
2020-09-26 12:54:51 default[20200926t134815] TCP/IP connections on port 5432?
If it's connected internally I assume I don't have to add that internal IP to firewall rules, although I did try that as well. As for firewall rules, I have allowed my local machine's IP address so I can connect to the remote Postgres server via PgAdmin.
I've actually tried External IP(34.89.113.193) as well although that doesn't make sense to me.
I'm a bit of a noob on networks and backend stuff in general, any help would be much appreciated.
UPDATED 1
This is my firewall rules:
Direction
Ingress, Egress
Action on match
Allow
Source filters
IP ranges
92.40.176.9/32
78.146.103.141/32
10.154.0.2
Protocols and ports
tcp:5432
Image for reference: Screenshot for the list of firewall rules
It turns out the firewall / postgres configurations were all ok, but because this VPC connector method was on beta, I needed to run:
gcloud beta app deploy
instead of the usual
gcloud app deploy.
This command then updated gcloud Beta Commands and prompted me to enable API:
API [appengine.googleapis.com] not enabled on project [742932836941]. Would you like to enable and retry (this will take a few minutes)? (y/N)?
After enabling this everything worked fine.
Per the information provided seems like both VPC firewall rules and the connector are well configured.
However, based on the messages
2020-09-26 12:54:51 default[20200926t134815] Is the server running on host "10.154.0.2" and accepting
2020-09-26 12:54:51 default[20200926t134815] TCP/IP connections on port 5432?
Seems like the VM or server using 10.154.0.2 is not accepting requests on port 5432 or the port has not been opened, you can use this site to do a port scan.
Based on the guide you followed to create PostgreSQL you are using Ubuntu as OS, therefore I suggest you open the port in ubuntu and see if the issue persists.
I´ve deploy my demo app on GAE and works fine with mLab , but when I try to deploy mongodb on GCE (MongoDB (Google Click to Deploy) )the deploy is success but I don´t know how to get te URI to set on my app running on GAE.
I try with internal and external IP but it seems dont work !
Thanks
GAE Standard deployments are sand-boxed. Therefore you can not connect to GCE instances' internal IPs. You can imagine it as two different devices on two different private networks that are not capable to communicate with one another using their internal IPs. However, they can always communicate if one of the devices (GCE instance in this case) has a public IP, and it's private network (firewall) allowed traffic through the port required by the device.
On the other hand, if the GAE deployment is in flex environment, you should be able to connect to the db using the API through internal IPs.
I have tried and succeeded with this flex environment example for both internal and external IP addresses. Like you, I used Cloud Launcher to deploy Mongodb which created GCE instances with public IPs and network tags mongodb and mongodb-db. Then I created a db, username and a password by connecting to the primary db instance through SSH.
To use the internal IP, I just created/modified keys.json file per the example, as follows:
{
"mongoHost": "internal IP address",
"mongoPort": "27017",
"mongoDatabase": "db",
"mongoUser": "username",
"mongoPass": "password"
}
So I didn't have to worry about the URI as the code in server.js took care of it through passing this string:
mongodb://${user}:${pass}#${host}:${port}
But for your demo app, you may have to check the MongoDB official documentation for the standard connection string format URI.
As for using public IPs, I had to create a network firewall rule that allows tcp ingress on port 27017 with target tags identical to the network tags in order to limit access through the port to the MongoDB instances only. Next, I modified the keys.json file as above by replacing the internal IP with the public one.
I have successfully created a scale set using (https://github.com/Azure/azure-quickstart-templates/tree/master/201-vmss-windows-customimage) with one instance from the my custom image:
The custom image has microsoft server data center DS1 V2 and node.js installed in and a copy of my application.
From the new portal I can see the IP of the VM.
I have also successfully started it from the powershell.
However the RDP on that IP with port 3389 or 50000, 50001 are not working.
I tried to add some inboud rules but the situation is same (may inboud rules not correct).
Please can you let me know the steps required after the scale set is created in order to make a RDP ?
You won't be able to connect to the IP of the VM from outside the VNET as each VM has an internal IP address. You need to connect to the public IP of the load balancer (you probably meant that implicitly but being precise just in case).
You will also need to create inbound NAT rules to map a port range on the load balancer public IP address to port 3389 on the backend, and double check that RDP was enabled on the source image before uploading it.
Take a look at this template for an example of inbound NAT rules.. https://github.com/Azure/azure-quickstart-templates/blob/master/201-vmss-windows-nat/azuredeploy.json
I'm still having problems accessing the cloud SQL instance from a GCE container. When I try to open up mysql, I get the following error:
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial
communication packet', system error: 0
The connection works fine from my local machine, though (The instance has a public IP and I have added my office's IP to the 'allowed Networks'). So, the instance is accessible through the internet just fine.
I guess the db's access control is blocking my access from the gce network, but I'm unable to figure out how to configure this.
I added my project to "Authorized App Engine Applications" in the Cloud SQL control panel, but that doesn't seem to help.
EDIT:
If I add "0.0.0.0/0" to Allowed Networks, all works well. This is obviously not what I want, so what do I need to enter instead?
EDIT2: I could also add all public IPs from my kubernetes cluster (obtained through gcloud compute instances list) and add them to the cloud sql access list manually. But, this doesn't seem to be right, does it?
The recommended solution is to use SSL connection with that 0.0.0.0/0 CIDR. This is to limit the connection to the correct key. I also read that they won't promise you a specific IP range so the CIDR /14 might not work some times. I had to do the SSL connection with my Cloud SQL for the same reasons.
You should use the public IP addresses of the GCE instances to correctly allow traffic to your Cloud SQL instance (as you mentioned in EDIT2).
You can find more information in Cloud SQL documentation: https://cloud.google.com/sql/docs/gce-access
If you add the /14 CIDR block for your Container Engine cluster as the source address range does that work?
To find the CIDR block for your cluster, click on the cluster name in the Google Cloud Console and find the row labeled "Container address range".