Is it possible to obtain a static IP address for a Google Cloud Storage bucket for use with DNS? I wish to host it at mydomain.com, and because I also have e-mail at mydomain.com, I cannot use a DNS CNAME record -- I need to use an IP address and an DNS A record.
You can, but doing so requires using Google Cloud Load Balancer: https://cloud.google.com/compute/docs/load-balancing/http/adding-a-backend-bucket-to-content-based-load-balancing. The upside of this approach is that it comes with a number of useful features, like mapping a collection of buckets and compute resources to a single domain, as well as the static IP address you want. The downside is that there's additional cost and complexity.
I recommend just using a subdomain and a CNAME record, if you don't need any of the other features.
Check it with google documentation.
You can manage it on instance page in networking section.
Related
My organisation intends to provide data to 3rd parties by putting that data into files in a GCS bucket and granting the 3rd party's GCP service account access to the data. We want to lock down that access as much as possible, one thing we'd especially like to do is limit the IP addresses that are allowed to issue requests to get the data.
I have been pouring over IAM conditions documentation:
Overview of IAM Conditions
Attribute reference for IAM Conditions
however I'm not able to understand the docs in sufficient detail to know if what I want to do is possible.
I read this which sounded promising:
The access levels attribute is derived from attributes of the request, such as the origin IP address, device attributes, the time of day, and more.
https://cloud.google.com/iam/docs/conditions-attribute-reference#access-levels
but it seems as though that only applies when IAP is being used:
The access levels attribute is available only when you use Identity-Aware Proxy
Is there a way to restrict which IP addresses can be used to access data in a GCS bucket?
I think I’ve just found something that will work - VPC service perimeters. I’ve tried it and it does seem just what I need.
This blog post covers it very very well https://medium.com/google-cloud/limit-bucket-access-in-google-cloud-storage-by-ip-address-d59029cab9c6
I just tested something that I never tried, and finally it works!
So, to achieve this, you need 3 things
Your bucket secured with IAM
HTTPS Load Balancer with
Your Bucket as backend bucket
A HTTPS frontend
A domain name, with your DNS A record configured on the Load Balancer IP
Cloud Armor policy
Create a edge policy
Default rule: deny all
Additional rule: all IP ranges (use /32 range for a specific IP)
Add the policy on your backend bucket
Let that cooking (about 10 minutes to let the load balancer dispatching its configuration, the certificate generation if you use a managed certificates,...)
Then, the requester can perform GET request to the bucket object, with the access token in the authorization header, something like that
curl -H "Authorization: Bearer <ACCESS TOKEN>" https://<DomainName>/path/to/object.file
I have two google cloud instances running on different regions (West and East). Each instances have their own database. I am using Google Load balancer to route the traffic based upon the client's IP address (This is what Google load balancer is doing internally on Network load balancing).
For example, Bob is requesting from the east region and GLB will route the request to east region node only. Similarly, Dave is requesting from the west region and GLB will route the request to west region node.
Scenarios:
1. Bob just sign up and a new entry added to the East region database.
2. Bob tries to fetch his profile but somehow request went to the West (Bob now using a VPN) region and there is no information available.
Is there a way that I can customize GLB? If yes, then I can solve this problem by applying consistency hashing on the load balancer (using userId as a hash function) which will make sure that the request coming from Bob will always go to East region.
You can use the HTTP Load Balancer options: Session Affinity and Client IP Affinity.
There are subtle issues with any method, read the documentation carefully. The biggest issue is for clients behind a NAT (airport, hotel, Starbucks, etc.). Their public IP address is the same for all clients behind the NAT, therefore all traffic will go to the same backend for Client IP based affinity. I recommend using cookies.
Session affinity uses Client IP or Generated Cookie to make traffic decisions. This can keep traffic routed to the same backend.
Session Affinity
Client IP affinity directs requests from the same client IP address to the same backend instance based on a hash of the client's IP address.
Using Client IP Affinity
If you look at the documentation for GitHub pages, it has a pretty strong reccommendation to use a www domain for your custom domain site hosted on GitHub Pages.
From here: https://help.github.com/en/articles/about-supported-custom-domains#www-subdomains
A www subdomain is the most commonly used type of subdomain, in which
the www stands for World Wide Web. For example, www.example.com is a
www subdomain because it contains the subdomain part www.
We strongly recommend that you use a www subdomain for these reasons:
It gives your GitHub Pages site the benefit of our Content Delivery Network.
It is more stable because it is not affected by changes to the IP addresses of GitHub's servers.
Pages will load significantly faster because Denial of Service attack protection can be implemented more efficiently.
Does this mean if I do not use a www domain I will not get the benefits of a CDN or DDOS Attack Protection?
What is the technical reason why there is a difference between a www and non-www domain here?
The difference lies in how you point the site to GitHub's servers in DNS.
The simplest use of DNS is to point a domain name, at whatever level, at an IP address, using an A record. The same address will be used by all users, and can be changed only by the owner of the "zone" where the A record was added - in this case you, configuring the zone of your custom domain.
A smarter way is to alias a particular domain name to a different zone - in this case one managed by GitHub - using a CNAME record. The owners of that zone can then change the IP address as needed, and can even give different answers to different users based on their location (which is where the CDN reference comes from).
The key restriction however is that you can't use a CNANE as the root of a zone. See this Server Fault question for the technical details.
If you own "example.com", you can point an A record for the root of that domain at one GitHub IP address (or a few, selected essentially at random by visitors), but will give GitHub more freedom to route the traffic if you point a CNAME for a sub-domain, like "www.example.com".
Some DNS providers offer tools to work around this limitation by adding a fake record for the root of the domain that looks like a CNAME, but may be labelled differently (e.g. ANAME, ALIAS). When asked for the root A record, the DNS provider looks up the actual A records for that domain and returns those. This is useful for records which change over time, but because the address is being looked up by your DNS provider not the actual visitor, it may still prevent GitHub serving the best address for that visitor.
DNS does not provide a reliable mechanism for forwarding on apex/root records (e.g. example.com) but does for subdomains (CNAME). In practice this means that while you can point an A record to a single IP address corresponding to a node on Github's infrastructure, they aren't able to route DNS lookups for your apex record to other IP addresses that are closer to the request (CDN) or use DNS to mitigate the effects of a (D)DoS attack.
Some DNS providers do offer synthetic records (ALIAS, ANAME) that mimic the behavior of a CNAME record with apex domains (e.g. dnsimple), but they're not widely available, introduce additional complexity and latency, and don't provide the same level of geographic routing opportunities to Github et al.
I am looking for a way to set a custom ACL policy on one of my Cloud Object Storage (S3) buckets but all the examples I see at https://ibm-public-cos.github.io/crs-docs/crs-api-reference only show how to restrict by username. Essentially I would like to make my bucket private only unless the request is coming from a specific IP address.
Unfortunately, access control is pretty coarse at the moment and is only capable of granting and restricting access to other object storage instances. IP whitelisting is a priority for us and is the roadmap but is not currently supported. Granular access control via policies will be available later this year.
A month or so ago I put up a static website using google cloud storage. Before I could create a public bucket, I was asked to verify that I actually owned the domain after which I was naming the bucket. I had to upload a file from google to the existing host in order for google to verify domain ownership.
I do understand the need to do this. However, if I had just bought a domain and had no other host, I don't see how I would have been able to prove that I owned the domain.
Did I miss a way around this limitation? Is there another, more user friendly way of creating public sites on google cloud storage?
There are three ways to verify domain ownership:
Adding a special Meta tag to a site's homepage.
Uploading a special HTML file to a site.
Adding a DNS TXT record to a domain's DNS configuration.
The first two require the domain to be hosted somewhere but the third method is purely DNS configuration, so it can be accomplished without hosting the domain. You can read more details about these methods here.
Add CName with the information that the google gives you. That would solve your problem of verifying your domain ownership.