I've recently enabled hotlink protection on MaxCDN, using what is known as Referer Access Control whitelisting.
I've enabled my own domains and my CDN domain, and it's working very nicely, however, when I try to share an image on social media, the og:image is not being picked up correctly.
Using the Facebook Debugger, I can see that an error is being thrown on the og:image
"Provided og:image URL, https://cdn.collectiveray.com/images/webdesign/web_design_blogs.jpg could not be processed as an image because it has an invalid content type."
I believe the problem is that the Facebook crawler is not in the whitelist. I've allow facebook.com, *.facebook.com, fbcdn.com, *.fbcdn.com, fbcdn.net, *.fbcdn.net, yet I am still unable to resolve the above error.
Would anybody know which are the exact domains to whitelist to allow social networks, both Facebook and others, to be able to access the images directly via their CDN URL?
TIA
David
I don’t think the Facebook scraper sends any referrer.
But you can identify it based on the User-Agent it sends, see https://developers.facebook.com/docs/sharing/webmasters/crawler
Details on how to set this up for MaxCDN here, https://www.maxcdn.com/one/tutorial/blank-referers-social-networks/
Related
I have a Facebook page of my blog but whenever I try to schedule post my blog article links on the Facebook page The Facebook automatically delete my schedule post and even some time they do not publish it can it be the reason that my domain name has been declared SPAM by the Facebook how to check it.
Use the Facebook sharing debugger.
If you input a URL that redirects to the blocked URL, it will initially say "this webpage contains a blocked URL. If you input the blocked URL directly, it will say "We can't review this website because the content doesn't meet our Community Standards. If you think this is a mistake, please let us know." You also get this message if you put in a URL that redirects to the blocked URL a second time.
https://developers.facebook.com/tools/debug/sharing/
:It is often hard to tell the reason for this but, you can definitely get out of this by following the proper instructions.
First check if your domain is prohibited or just penalized by the search engine. Or, maybe it is neither prohibited nor punishable by the domain name. It could be, you just managed to screw up your site.
Simply check this via online tool -isitban.com by entering your domain or website url.
Once found banned then, check your website content & remove content which is violating any Community Standards of facebook.
Once you are done with content optimisation then send your Facebook website again for review
Just got this message from the FB debugging tool : is not allowed for the specified application id. I'm trying to post a story using Open Graph on Facebook. Right now I'm using the Graph API Explorer, but will eventually use the Facebook SDK.
I AM using a different domain from the domain I'm using to host the meta tag URL. My understanding was that Open Graph needed a publicly accessible page from which to read meta tags. I was under the impression that this page DID NOT have to live at the same domain as my app. Is that wrong? So, does this mean that the page with all the meta tags has to live on the same domain as my web site?
To be clear: the FB app is configured to use domain X, but I'm hosting the OG page on domain Y. Is that a problem?
Update:
Okay, I'm still getting the error this AM. It seems like I'm getting contradictory info. Given that this is Facebook support, is there some way I can escalate?
I still don't know what the issue is, but I was able to get this working by changing my local dev URL to an address that shared the same domain as the URL that was hosting the OG object I needed.
So, locally I've set my environment up to be: mylocal.sample.com, and the OG is hosted at public.sample.com.
We're serving a Facebook feed in our site, which includes images such as profile pictures that are served off Facebook's cdn. Our site is served over HTTPS, and we generally use protocol-relative urls when linking to outside images. However, the facebook images were not showing up. After digging, i noticed that https links pointing to facebook cdn images throw an SSL error because the Facebook CDN url doesn't match the Akamai URL associated with their certificates.
Compare:
http://profile.ak.fbcdn.net/hprofile-ak-prn1/50164_1132339922_583434524_n.jpg
https://profile.ak.fbcdn.net/hprofile-ak-prn1/50164_1132339922_583434524_n.jpg
This makes using HTTPS or protocol-relative urls for facebook images impossible to access as image sources, and so makes embedding Facebook images into HTTPS sites impossible without triggering mixed content warnings.
Has anyone run into this?
Figured this out:
On your graph api calls, use return_ssl_resources=1, and it will give you the proper image https urls
I'm developing an url shortener and the current specification requires to avoid Facebook from scraping those links and instead an integrated scraper will do that and hand over the og meta tags to Facebook.
Is there anything in Facebook terms that doesn't allow me to reply to its scraper with different content (e.g. an empty page with opengraph meta tags only) from what a regular user would see?
Thanks in advance.
I just did this on my site.
I am redirecing non-mobile devices to another page but want the facebook scraper to get the content a mobile device would get.
This is the User Agent I get:
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
so matching against a regular pattern like this should detect the facebook scraper.
/^(facebookexternalhit.*)/
As mentioned above, this user agent can be spoofed easily which, in my case, doesn't matter.
Only problem could occur if facebook changes the user agent of their crawling script.
I am building a site, in which I denied hotlinking of images. But after I added the facebooks "like" link to my pages, I realized that I want to allow hotlinking for facebook. So, if a user likes a page on my site, facebook should be able to show a related thumbnail of the page in the profile of the user. So, I added an exclusion rule in IIS like
if {HTTP_REFERER} matches pattern ^(https?://)?(\w+.)facebook.(com|net)(/.)*$ , allow.
Alas, it didn't work for me.
After that I googled for an answer. A forum post suggested to use "tfbnw" instead of facebook, so I added that exclusion, too:
^(https?://)?(\w+.)*tfbnw.(com|net)(/.*)*$
But as you might expect, still no chance.
So, I don't know which URL facebook uses to request images when a user uses the like button. I would appreciate any help to uncover this mystery, so that I can allow that URL on my site.
Note: If I disable hotlinking protection, everything works fine. So we know that my problem is just the hotlinking protection.
Can you try whitelisting with IP address? All of FB's crawlers should come from one of the IP addresses returned by
whois -h whois.radb.net '!gAS32934'
Try allowing the domain fbcdn.net:
^(https?://)?(\w+.)fbcdn.(com|net)(/.)*$
This is facebook's content delivery network.