Disable googlebot fetching www - robots.txt

I have www redirect in .htaccess.
So www.example.com gets 301 redirect to example.com
But google still tries to fetch www.example.com also.
Can i disable googlebot fetcing www.example.com?
Eg from webaster tools or robots.txt?

In Google Webmaster Tools you can set your preferred hostname – with or without www, but this comes with no guarantee from Google. As you have 301s from www to non-www, Googlebot will probably respect your wish.

Related

301 Redirect with GatsbyJS on Heroku

I just finished working on an app using GatsbyJS. I'm hosting my web application on heroku. I have a custom domain reversed through heroku with SSL support. Now I'm able to visit the following domains:
http://myapp.com
https://myapp.com
http://www.myapp.com
https://www.myapp.com
Normally you want to have 301 redirects from www to non-www and from http:// to https://. Since the heroku router does not have the ability to force SSL, they want you to handle the redirects on the application level.
I did not found any solutions for SEO optimized 301 redirects within GatsbyJS.
Any help appreciated.
Not familiar with Heroku but if 301 redirect is not possible just ensure your pages have <link rel="canonical" href="" /> tags. That is a way to hint the search engines the URL you want to be indexed. Additionally you may redirect to https:// on the client using the onClientEntry API.
I just figured out how to get it to run on heroku. Since I did not find any solutions from gatsby itself, I adjusted my deployment to match the gatsby docs. I now use heroku buildpacks to build my project. While doing so, you can define a file called static.json.
This file has options like
"https_only": true,
"canonical_host": "yoursite.com"
With these options, heroku prepares the app to have a redirect from www to non-www and http to https. More information here: https://www.gatsbyjs.org/docs/deploying-to-heroku/

Redirect from https www.domain.com to https www.domain.co.uk

I would like to ask a question. I had a domain like
(https)www.domain.com
and i transfered the whole domain to
(https)www.domain.co.uk
but i have some seo issues. I want the
(https)www.domain.com/every_url
to redirect to
(https)www.domain.co.uk
I did it for domain.com/every_url to redirect to
(https)domain.co.uk
but if I hit an article from google that has already the https its shows a previous server error. For hosting I use plesk with dedicated server and i used 301 permanent redirect. Does anyone know how to fix this?
Thanks

301 redirect for Github Pages and CloudFlare SSL

I am using Github Pages as my hosting site for my domain. The pages are hosted at username.github.io. As per github pages documentation I have put the CNAME file in the root directory pointing to example.com
And in my godaddy DNS manager I have added CNAME www to username.github.io
Later I switched to CloudFlare to use the Universal Free SSL for my Github Custom Domain page.
Currently the CloudFlare DNS Manager includes these two items:
A exmaple.com 192.30.252.153
CNAME www username.github.io
Since I have enabled SSL in cloudfare and redirect http (naked or otherwise) addresses to https, I have put a Page Rule as http://*example.com/* with Always use https turned on.
Now all types of addresses are getting redirected to https://example.com (this is my end requirement)
However the 301 redirection from http://www.example.com to https://example.com is happening this way:
http://www.example.com to
https://www.example.com/ to
http://example.com/ to
https://example.com/
This multiple redirection will affect the site loading speed if a user types the address as www.example.com. And (possibly?) these multiple redirections will affect page ranking in search sites.
Hence is it not better to put direct 301 instead of multiple? Or using multiple redirection is what normally all web-masters do in a situation like this?
If no, then someone please guide me to enable the 301 redirection from http://www.example.com directly to https://example.com/ without any multiple redirections.
You can set Page Rules in CloudFlare and change the order to your intended effect.
If this is still problematic you can also enable HSTS which will require the browser to access the HTTPS version after the first time you visit the site. This also makes the site more secure by not allowing anyone to man-in-the-middle your secure connections.

Google Webmaster Tools and http vs https

I notice Google recently changed Webmaster Tools to treat http and https as completely different sites - same as www and non-www version of the URL. So I now need to configure and review four separate sites? What a PITA!
They said the do it because we asked for it. I can see how maybe 0.01% of the Internet may use the protocol to separate content on the same URL. But what a pain for the rest of us!
Is there a way to get it to merge all the data as one?
It's recommended to use either www or non-www, but not both. Same with http and https, pick one, use that, and redirect the other. You can find many answers about how to do the redirect. Here are some examples for Apache: redirect non-www to www, redirect www to non-www, redirect http to https and redirect https to http.

Is it okay to send a 301 redirect from https (if not in use)?

I have my website hosted on the HTTP protocol, but my hosting service provides a default page when accessed through HTTPS. Since I'm not using SSL, from an SEO perspective, which one is more correct?
a) toggle https for my website and do a 301 redirect to http://?
b) just leave it like that (default https page from the web hosting co)
A would be the cleanest solution but it shouldn't really matter. Who, besides yourself, will be trying the https page?
I've seen problems with https vs http for websites with little domain authority, or for websites which link to a checkout with https and then link back out to https instead of http.
We recommend clients to 301 to the http version just in case this causes issues.
Try this Mod Rewrite code:
RewriteCond %{HTTPS} on
RewriteRule (.*) http://%{HTTP_HOST}%{REQUEST_URI}