I'm in this situation:
Domains:
www.example.com
www.example.it
that point at the same content in different languages. E.g.:
www.example.com/audi-car.html
www.example.it/audi-auto.html
and I have only one robots.txt in the root domains.
My question is: How can set my robots.txt to disallow crawling of www.example.it to all bots coming from www.example.com and reverse?
Write different robots.txt for each domain and use .htaccess to redirect robots.txt request based on host from where the request came:
RewriteCond %{HTTP_HOST} ^(.*)\.com$ [NC]
RewriteCond %{HTTPS}s ^on(s)|
RewriteRule ^robots\.txt$ /robots-com.txt [L]
RewriteCond %{HTTP_HOST} ^(.*)\.it$ [NC]
RewriteCond %{HTTPS}s ^on(s)|
RewriteRule ^robots\.txt$ /robots-it.txt [L]
Make sure that RewriteEngine On is placed on top and mod_rewrite enabled.
Related
My self-coded portfolio website is giving me problems when used as an Instagram profile link (let's say www.myurl.com): the Instagram browser displays a 404 error when clicking the profile link. Clicking "Back to Index" on the 404 page shows the website as it should. After checking the URL, I see that Instagram automatically adds a random string to the domain (like www.myurl.com/?fbclid=PAAaa-Vm9q.../) and I assume my website doesn't know how to redirect this. The issue does not occur on a desktop computer, only mobile.
I'm guessing this random string is an SEO tool, which I woefully do not know how to use.
But I need the website link on IG to go through to my homepage without throwing the 404 error, at a minimum. Is there some kind of code / html document / web hosting / dns setting I need to manipulate to get this to work?
I have a feeling this is an issue caused by DNS settings on Cloudflare since I recently set up a proxy with them to use their global SSL certificate.
Some notes about my website in case they are helpful:
hosting on GoDaddy, recently proxied through CloudFlare for SSL certificate
I have the main domain (www.myurl.com) automatically redirect to a subdomain (sub.myurl.com). This subdomain is actually stored in a folder like www.myurl.com/sub/ .
Website is a CMS built on PHP 5
I tried URL shortener services as a stopgap measure. The only one that solved the problem was shorturl.at, but tiny.url and bit.ly still had the 404 issue. I have no idea why.
I looked at the htaccess document in the subdomain root folder, the contents are below. I think the issue may be that the "base" needs to be "rewritten" to the subfolder (www.myurl.com/sub/) but I'm not sure how to correctly change this code:
RewriteEngine on
# Some hosts require a rewritebase rule, if so, uncomment the RewriteBase line below. If you are running from a subdirectory, your rewritebase should match the name of the path to where stacey is stored.
# ie. if in a folder named 'stacey', RewriteBase /stacey
#RewriteBase /
ErrorDocument 404 /404.html
# Rewrite any calls to *.html, *.json, *.xml, *.atom, *.rss, *.rdf or *.txt if a folder matching * exists
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_URI} !public/
RewriteCond %{DOCUMENT_ROOT}/public/$1.$2 !-f
RewriteRule (.+)\.(html|json|xml|atom|rss|rdf|txt)$ $1/ [L]
# Add a trailing slash to directories
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(\.)
RewriteCond %{REQUEST_URI} !(.*)/$
RewriteRule ([^/]+)$ $1/ [L]
# Rewrite any calls to /* or /app to the index.php file
RewriteCond %{REQUEST_URI} /app/$
RewriteRule ^app/ index.php [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)/$ index.php?$1 [L]
# Rewrite any file calls to the public directory
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !public/
RewriteRule ^(.+)$ public/$1 [L]
I'm very much a beginner with webhosting / coding / etc so will appreciate an idiot-friendly explanation!! Thanks in advance.
I need help with a redirect rule.
I created a subdomain sub.abc.com and set it's public_html to that of abc.com.
This means public_html is now accessible either by abc.com or by sub.abc.com .
I want to limit this so that sub.abc.com only provides access to images in wp-content/uploads any other requests to sub.abc.com should be redirected to abc.com/url/url .
Here is what i have tried creating
RewriteCond %{HTTP_HOST} ^cdn.example.com
RewriteCond %{REQUEST_URI} !^/wp-content/uploads/$1
RewriteRule ^(.*)$ http://example.com/$1 [L,R=301]
so if the user is accessing cdn.example.com/wp-content/uploads/1/1/file.jpg then there is no redirect, but if they are trying to access cdn.example.com/post/post or cdn.example.com/anything-else then it will redirect to example.com/post/post or example.com/anything-else .
The following rule should work for you. Remember to put it at the of your htaccess or server.config file before others rules otherwise these rules may override it.
RewriteEngine on
#serve only images on abc.example.com
#redirect to main domain if request is not for images
######################
#if the host is "sub.example.com"
RewriteCond %{HTTP_HOST} ^sub\.example\.com$ [NC]
#and the request is not for images and /wp-content/uploads
RewriteCond %{REQUEST_URI} !/wp-content/uploads/?$ [NC]
RewriteCond %{REQUEST_URI} !\.(jpg|png|gif|jpeg)$ [NC]
#redirect the request to main domain
RewriteRule ^ https://example.com%{REQUEST_URI} [L,R=302]
This will redirect your sub.example.com with request string to example.com if the subdomain is used for requests other then images .
Replace sub.example.com and example.com with your real domain name in the RewriteCond and RewriteRule above.
If this works and you are happy with the redirect then change R=302 to R=301 to make the redirection permanent.
We've been issued a new certificate for our website, but unfortunately without an alternative name for the www subdomain. Visiting the TLD without www works just fine.
Redirecting them via .htaccess doesn't work, probably because the initial connection isn't even made.
I've tried using this:
RewriteCond %{HTTPS} =on
RewriteCond %{HTTP_HOST} ^www.mysite$
RewriteRule ^(.*)$ https://mysite/$1 [R]
Is there any way to "force" users away from www or do we need to be issued a new (fixed) certificate for our domain?
Check this redirect (change example.com to your domain):
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.example\.com [NC]
RewriteRule (.*) https://example.com/$1 [R=301,L]
I'm not very advanced at programming and I need to make a 301 redirection from one specific subdomain page to another. The tricky part, at least for me, is that the pages are on two different subdomains. So far I've only managed to 301 ALL pages to the target or crashing the whole site.
I've used many, but are currently at:
RewriteCond %{HTTP_HOST} !^spiele.deutschedownloads.de/?DriverScanner/Diverse//download/4770$ [NC]
RewriteRule ^(.*)$ http://programme.deutschedownloads.de/?DriverScanner/Drivers/Werkzeuge-Diverse/download/4413 [L,R=301]
What I want to do is simply to 301 redirect
spiele.deutschedownloads.de/?DriverScanner/Diverse//download/4770
To
http://programme.deutschedownloads.de/?DriverScanner/Drivers/Werkzeuge-Diverse/download/4413
Any help is most appreciated!
This should work:
RewriteCond %{HTTP_HOST} ^spiele\.deutschedownloads\.de$ [NC]
RewriteCond %{QUERY_STRING} ^DriverScanner/Diverse//?download/4770/?$ [NC]
RewriteRule ^ http://programme.deutschedownloads.de/?DriverScanner/Drivers/Werkzeuge-Diverse/download/4413 [R=301,L]
First condition matches the domain in case they are both on the same root folder.
Second condition matches the query string.
If both conditions are OK it redirects.
Good morning, i have a problem and i don't find a solution.
I need to point multiple domain to same folder but my hosting when i create a subdomain point to a folder with the same name of the subdomain.
The result that i want to have is:
demo.domainname.tld/index.php will point to www.domainname.tld/shop/index.php?username=demo
demo.domainname.tld/product.php?id=100 will point to www.domainname.tld/shop/product.php?id=100&username=demo
test.domainname.tld/index.php will point to www.domainname.tld/shop/index.php?username=test
test.domainname.tld/product.php?id=100 will point to www.domainname.tld/shop/product.php?id=100&username=test
keeping in the url the subdomain address.
I am creating a ecommerce platform and i want to use an unique ecommerce and load different content by keeping the username from subdomain.
it is possible? how?
A long as all the *.domainname.tld DNS entries point to the same IP as www.domainname.tld and they all share the same document root, you can add these rules to the htaccess in the document root:
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteCond %{HTTP_HOST} ^([^.]+)\.domainname\.tld$ [NC]
RewriteRule ^/?index\.php$ /shop/index.php?username=%1 [L,QSA]
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteCond %{HTTP_HOST} ^([^.]+)\.domainname\.tld$ [NC]
RewriteRule ^/?product.php$ /shop/product.php?username=%1 [L,QSA]
If you want to actually do a redirect, thus changing the URL in the browser's address bar, include a R flag in the square brackets so they look like: [L,QSA,R] or [L,QSA,R=301] for a permanent redirect.