So I've been looking into simplifying some of our project solutions and by the look of it, google cloud functions has the potential to simplify some of our current structure. The main thing I'm curious about is if GCF is able to connect to internal nodes in a Kubernetes cluster hosted in google cloud?
I'm quite the rookie on this so any input is greatly appreciated.
Google Cloud has a beta (as of this writing) feature called Serverless VPC Access that allows you to connect your serverless features (Cloud Functions, App Engine Standard) to the VPC network where your GKE cluster is. This would allow you to access private IPs of your VPC network from Cloud Functions.
You can read the full setup instructions but the basic steps are:
Create a Serverless VPC Access Connector (under the "VPC Network -> Serverless VPC Access" menu in the console)
Grant the cloud function's service account any permissions it will need. Specifically, it will at least need "Project > Viewer" and "Compute Engine > Compute Network User".
Configure the function to use the connector. (In the console, this is done in the advanced settings's "VPC Connector" field).
Related
I've set up the peering connection between MongoDB Atlas and Googles "default" VPC and the connection is labeled as "active" on both ends.
The ip range of the vpc is whitelisted in MongoDB Atlas.
But my node hosted in google-app-engine still gets timed out when accessing the MongoDB.
I use the connection url of mongodb atlas for peered connection in the form of (notice the "-pri"):
mongodb+srv://<username>:<password>#<my-cluster>-pri.rthhs.mongodb.net/<dbname>?retryWrites=true&w=majority
Which part am i missing to establich the connection? Do i need a google vpc connector?
Thanks for any help!
First of all, make sure you are running M10-Cluster or above!!! VPC-peering is not available for M0/M2/M5...
And YES you do need that connector! All "Serverless"-Services from Gcloud (like GAE in standard environment) need it.
create a connector in the same region as your GAE-App following these instructions. You can find the current region of your GAE-App with gcloud app describe
your app.yaml has to point to that connector like this
app.yaml
runtime: nodejs10
vpc_access_connector:
name: projects/GCLOUD_PROJECT_ID/locations/REGION_WHERE_GAE_RUNS/connectors/NAME_YOU_ENTERED_IN_STEP_1
Go to your Atlas project, navigate to Network Access and whitelist the IP-range you set for the connector in Step 1
You may also need to whitelist the IP-range from Step 1 for the VPC-Network. You can do that in GCP by navigating to VPC-Network -> Firewall
If you have questions about how to setup the VPC-Peering between Atlas and Gcloud try this tutorial. They do it for Kubernetes-Engine (no connector needed). But adding my steps from above will hopefully do the trick.
Try Cannot connect to Mongo Atlas using VPC peering from GCP cluster and MongoDB and Google Cloud Functions VPC Peering?.
First step I suggest identifying whether you have physical connectivity (and so need to fix ip whitelist) or don't have connectivity (and need to fix peering configuration).
I have a VPC called 'subnet-1' where I have 3 VM Instances and 1 SQL Instance (Postgresql to be precise). All of them with no Public IP, just Private IP. They are in a kubernetes cluster.
The requirement I have is to be able to connect to the VPC from my on-premise PC in order to use some SQLClient to connect to the PG and see the data, I have read about CloudVPN but don't know how to configure it to have what I want... does anyone of you ever done some configuration like that?
You can use Cloud VPN but that is a site-to-site VPN that requires supported routers and solid knowledge of routing, networking and VPNs.
Using third-party VPNs with Cloud VPN
For small businesses or home based developers that only need to connect a few users to a VPC, use an open source product like OpenVPN. Google Marketplace offers a deployable configuration for your project with a click configuration. Simple and easy to deploy.
Google Cloud Marketplace OpenVPN
In order to connect to Kubernetes Cluster from your on-premises networks you can use Cloud VPN. The configuration of Cloud VPN with the Cluster and On-Premise network is documented here.
I'm evaluating a move from Google Kubernetes Engine to Google Cloud Run, to improve cost and resource efficiency within our company. I'm also in the process of transitioning our workflows from monolithic PHP and Ruby apps to a more nimble Node.js setup, using MongoDB.
For a small organization like ours, I like the idea of managed services such as Google Cloud Run and MongoDB Atlas, however, I'm concerned about the security. In MongoDB Atlas, it seems the only real security measure is to whitelist IP, which I obviously don't have access to through Google Cloud Run.
I'm definitely not a network expert, so I'm wondering if anyone has any ideas for securely connecting Cloud Run to MongoDB Atlas, while still maintaining scalability. If I have to remain on GKE, so be it, I just want to know all of my options before I move forward.
IP whitelist - by its very nature, Google Cloud Run would seem to be anti-static-IP, so this seems to be a non-starter.
I evaluated items such as Cloud NAT and Cloud VPC Peering, but from what I can tell Cloud Run does not have access to the VPC, so it seems like this wouldn't help either.
Cloud Run and Cloud Function have the same underlying infrastructure. Cloud Function have the capability to be connected to a VPC. Thereby, Cloud Run will support a day this capability, I hope by the end of 2019.
If you can, I just recommend you to wait!
Update (October 2020): Cloud Run has now launched VPC egress feature that lets you configure a static IP for outbound requests through Cloud NAT. You can follow this step by step guide in the documentation to configure a static IP to connect to MongoDB Atlas.
I removed a bunch of IAM policies and think this is preventing me from creating k8s clusters in Google Cloud (through the UI).
Every time I click Create cluster, it processes for a bit, before hanging up and throwing the following error:
Create Kubernetes Engine cluster "standard-cluster-1"
Just now
MyProject
Google Compute Engine: Required 'compute.zones.get' permission for 'projects/<MY_PROJECT_ID>/zones/us-central1-a'.
I'm mainly doing this through my host shell (iTerm) and NOT through the interactive shell found on cloud.google.com.
Here's the IAM policy for a user (I use my google email address under the Member column):
Really hoping to get unblocked so I can start creating clusters in my shell again and not have to use the interactive shell on the Google Cloud website.
You are missing ServiceAgent roles. But only service accounts can be granted those roles.
1) First, copy you project number
2) create following members for the Service Agents replacing 77597574896 with your project number and set appropriate roles:
service-77597574896#container-engine-robot.iam.gserviceaccount.com - Kubernetes Engine Service Agent
service-77597574896#compute-system.iam.gserviceaccount.com - Kubernetes Engine Service Agent
77597574896#cloudservices.gserviceaccount.com - Editor
This should work now, because I've tested it with my cluster
In order to create new cluster container - please just add new role in yours IAM settings:
- Kubernetes Engine Admin,
Please share with the results.
I have a question around K8S Service Type "LoadBalancer".
I am working on developing a new "Kubernetes As a Service" Platform (like GKE etc.) for multi cloud.
Question is: K8S Service Type "LoadBalancer" works with Cloud Load Balancers (which are external to Kubernetes). GKE & other cloud based solution provides direct integration with them, so If I create a GKE Cluster & implement a Service Type "LoadBalancer", it will transparently create a new GCP Load Balancer & show Load Balancer IP in Kubernetes (as External IP). Same applies to other Cloud Providers also.
I want to allow a similar feature on my new "Kubernetes As a Service" platform, where users can choose a cloud provider, create a Kubernetes Cluster & then apply a K8S Service Type "LoadBalancer" & this will result creating a Load Balancer on the (user selected) cloud platform.
I am able to automate the flow till Kubernetes Cluster Creation, but clueless when it comes to "K8S Service & External Load Balancer" Integration.
Can anyone please help me how can I approach integrating K8S Service Type "LoadBalancer" with Specific Cloud Load Balancers? Do I need to write a new CRD or is there any similar code available in Git (in case anyone know any link for reference) ?
You have to understand how kubernetes is interacting with cloud provider. Like for example previously I deployed the Kubernetes on AWS with kops. I see that kubernetes uses aws access key & access secret to interact with aws. If I remember correctly, I saw some CLI options in kube-proxy or kubelet to support AWS. (I have searched man pages for all kubernetes binaries for aws options, but I couldn't find any to provide to you).
For example look at the kubelet man page, they provided an option called --google-json-key to authenticate GCP. You will get some idea if you deploy kubernetes on AWS with kops or kube-aws and dig through the setup and its configuration/options etc.(Same applies to other cloud providers)