I have checked similar questions asked, but none seem to match the circumstances of this one.
This page is returning a 404 error in Facebook's Object Debugger tool. Other pages on the site work okay, so it shouldn't be any missing meta tags.
Now some of the page content is hidden, but only some, the majority of the page content is available, so surely this shouldn't be causing the issue. If it does then that would have to be regarded as a bug, no?
Anyone have any idea what the issue might be and/or how to fix?
The error message is accurate - your URL is returning a 404 when the Facebook crawler attempts to get the metadata
You'll need to check your server settings or the code which renders that URL to see why it's doing so, here's the output when i made the same request Facebook makes from my own laptop:
$ curl -A "facebookexternalhit/1.1" -i 'http://austparents.edu.au/webinars/parent-webinar-on-the-australian-curriculum-with-rob-randall-ceo-acara/'
HTTP/1.1 403 Forbidden
Date: Tue, 23 Sep 2014 00:03:36 GMT
Server: Apache/2.2.14 (Ubuntu)
Vary: Accept-Encoding
Content-Length: 366
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /webinars/parent-webinar-on-the-australian-curriculum-with-rob-randall-ceo-acara/
on this server.</p>
<hr>
<address>Apache/2.2.14 (Ubuntu) Server at austparents.edu.au Port 80</address>
</body></html>
Related
I don't know if this is the right place to ask this question but I'm hoping to find directions here.
I have a smart tv and I like to watch tv from my country with the SSIPTV app. I found an android app that streams local channels, so I checked the requests with android studio to find the streaming links. Some of them are free, but some others are served through cloudfront. The problem is that I can't add a header needed for cloudfront to authorize the request.
For example: when I try to make a request without the "User-Agent" header, the response is this:
Status Code: 403 Forbidden
Connection: keep-alive
Content-Length: 560
Content-Type: text/html
Date: Tue, 01 Jan 2019 20:57:50 GMT
Server: CloudFront
Via: 1.1 f7e7b00c5c66a4e43041ba24c378d07a.cloudfront.net (CloudFront)
X-Amz-Cf-Id: uZQAVTrQzHsQe2vGyHxY1OYfjHCL-Nz7gCTG-koHcgr1A5HG7fGGOg==
X-Cache: Error from cloudfront
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD>
<BODY>
<H1>403 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Request blocked.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: TZztsUjltHpEhx54wplzupvLmZwjCRPtAvTcbdJ8DL16b1k9-_XwZw==
</PRE>
<ADDRESS></ADDRESS>
</BODY>
</HTML>
But if I set the "User-Agent" header with the value "iPhone" this is the response:enter code here
Status Code: 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type, User-Agent, If-Modified-Since, Cache-Control, Range
Access-Control-Allow-Methods: OPTIONS, GET, POST, HEAD
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Date, Server, Content-Type, Content-Length
Cache-Control: max-age=1
Connection: keep-alive
Content-Length: 366
Content-Type: application/vnd.apple.mpegurl
Date: Tue, 01 Jan 2019 20:51:32 GMT
Server: WowzaStreamingEngine/4.7.6
Via: 1.1 880eb84cefca849ee159a7c4d89c31ea.cloudfront.net (CloudFront)
X-Amz-Cf-Id: pogc8_OBsN2-QeGj_1q8K_vyxrQH-G8a2JmWqSkVt9x57NlbKfDSdQ==
X-Cache: Hit from cloudfront
So, is there a way I could set up a proxy to add the request and then get the content served in my tv app?
If you can configure your TV app to use an HTTP proxy, then this is straightforward, eg in squid this is documented here: http://www.squid-cache.org/Doc/config/request_header_add/
request_header_add User-Agent "iPhone"
My URL is http://citynomads.com which redirects to http://www.citnomads.com and the browser follows the redirect perfectly well, but Facebooks linter is telling me it can't follow the link. I can't see why it's having an issue with this:
$ curl -i citynomads.com
HTTP/1.1 302 Moved Temporarily
Server: nginx/1.0.8
Date: Sat, 28 Apr 2012 16:12:38 GMT
Content-Type: text/html
Content-Length: 160
Connection: keep-alive
Location: http://www.citynomads.com/
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx/1.0.8</center>
</body>
</html>
Linter: https://developers.facebook.com/tools/debug/og/object?q=http%3A%2F%2Fcitynomads.com
How can I rectify that?
I think this is a FB linter bug. Your given linter link now works (has some other bizarre warnings, but not a failed redirect...)
This happened to me too, but posting the same url to facebook does generate a proper preview, so i'm guessing here...
Wouldn't be their first bug, nor the last...
HTH
Ok, so i am developing a new site, and it is very dependent of facebook.
I have looked everywhere and done everything i should but i keep getting this message in my FB comment area: Warning: http://www.videozoo.dk/?videos=klo-aben is unreachable
My header looks as it should like this:
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:fb="http://www.facebook.com/2008/fbml" xmlns:og="http://ogp.me/ns#">
<head profile="http://gmpg.org/xfn/11">
<meta property="fb:admins" content="my fb id nr"/>
<meta property="fb:app_id" content="195385377211689">
<meta property="og:title" content="Videozoo.dk - Dyre video" />
<meta property="og:type" content="Video" />
<meta property="og:url" content="www.videozoo.dk/?videos=klo-aben" />
<meta property="og:site_name" content="Dyre videoer for alle!"/>
<meta property="og:description"
content="Endnu en dyre video på VideoZoo.dk"/>
My fb comment code looks like this:
<div class="fb-comments" data-href="www.videozoo.dk/?videos=klo-aben" data-num-posts="10" data-width="640" data-colorscheme="dark"></div>
My app id for this comment box was made 4 hours ago so it should be updated on the servers by now...
The information in the app matches what is stated above:
Application ID/API Key:
195385377211689
Site-URL:
http://www.videozoo.dk/
Domain:
videozoo.dk
BUT - It is still not working! - is it because my site is new or mabye because the app is not updated on the servers yet??
Please have a closer look and maybe test at this link: www.videozoo.dk/?videos=klo-aben
All ideas are welcome!!
You left the protocol out of your value for og:url, include it there and it may solve the issue, the scraper follows HTTP or og:url redirects, and that URL is likely detected as invalid
{edit} I figured this out, and it's a bit strange, but... {/edit}
When I manually scrape that page it seems to work fine, but when I run it through the URL Debugger it fails due to a HTTP 403 response from your side
I've seen this before with other servers which can't handle some part of Facebook's request - in this case it seems to be because your server is rejecting the request if a HTTP 'Range' header is sent.
Facebook's crawler only requests the first 40KB of the document when scraping, as the meta tags should be in the <head></head> section
My test was:
$ curl -I -H 'Range: bytes=0-40960' 'http://www.videozoo.dk/?videos=klo-aben'
HTTP/1.1 403 Forbidden
Date: Wed, 30 Nov 2011 14:17:54 GMT
Server: Apache/2.2.6 mod_auth_kerb/5.3 PHP/5.2.17 mod_fcgid/2.3.5
Accept-Ranges: bytes
Connection: close
Content-Type: text/html
$ curl -I 'http://www.videozoo.dk/?videos=klo-aben'
HTTP/1.1 200 OK
Date: Wed, 30 Nov 2011 14:18:02 GMT
Server: Apache/2.2.6 mod_auth_kerb/5.3 PHP/5.2.17 mod_fcgid/2.3.5
X-Powered-By: PHP/5.2.17
Connection: close
Content-Type: text/html
I'm not sure if this is something in your code, server config, an intermediate proxy, etc, but it's very likely the cause of your problem
When we visit fm.im, we are redirected to http://friendfeed.com.
Here are some other examples:
ff.im/abc
ff.im/efg
How is FriendFeed able to do this?
.im is the Isle of Man top-level domain (ccTLD). The registry normally requires names to be at least three characters long, unless you pay considerably more.
Two-character domains look cool but aren't particularly useful since IE rejects their cookies (old article, but still mostly true for newer IE versions).
When your browser requests ff.im:
GET / HTTP/1.1
host: ff.im
their webserver responds with a redirect, either to the main FriendFeed site:
HTTP/1.1 302 Found
Date: Sat, 09 Apr 2011 12:29:38 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Content-Length: 0
Location: http://friendfeed.com/
Server: FriendFeedServer/0.1
or to some other place (when using their URL-shortener).
Hi I'm experiencing a super weird problem.
Whenever I post links to my website on Facebook, they come up as Forbidden.
The site itself works great and I have no seen this when linking on other sites.
Could this be a server misconfiguration? Any thoughts on where to look?
here's some Info:
I have a dedicated server running WHM 11.25.0
i have 2 sites hosted here using cPanel 11.25.0
the error msg:
Forbidden You don't have
permission to access
/blog/deepwater-horizon-11/ on this
server. Additionally, a 404
Not Found error was encountered while
trying to use an ErrorDocument to
handle the request.
Apache/2.2.14 (Unix)
mod_ssl/2.2.14 OpenSSL/0.9.8i DAV/2
mod_auth_passthrough/2.1
mod_bwlimited/1.4 FrontPage/5.0.2.2635
Server at www.offshoreinjuries.com
Port 80
UPDATE:
Here is a sample link if it helps. (notice going the linked page directly works fine)
http://www.facebook.com/l.php?u=http%3A%2F%2Fwww.offshoreinjuries.com%2Fblog%2Fdeepwater-horizon-11%2F&h=834ea
UPDATE and ANSWER:
Found the issue and added a complete answer below.
You must have a rule somewhere that reads the HTTP_REFERER and rejects incoming links from Facebook. Seriously. This is what happens between the lines:
No referrer
telnet www.offshoreinjuries.com 80
HEAD /blog/deepwater-horizon-11/ HTTP/1.1
Host: www.offshoreinjuries.com
HTTP/1.1 200 OK
Date: Fri, 28 May 2010 09:19:45 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8i DAV/2 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
X-Powered-By: PHP/5.2.12
X-Pingback: http://www.offshoreinjuries.com/blog/xmlrpc.php
Content-Type: text/html; charset=UTF-8
200 OK, good.
Facebook referrer
telnet www.offshoreinjuries.com 80
HEAD /blog/deepwater-horizon-11/ HTTP/1.1
Host: www.offshoreinjuries.com
Referer: http://www.facebook.com/l.php?u=http%3A%2F%2Fwww.offshoreinjuries.com%2Fblog%2Fdeepwater-horizon-11%2F&h=834ea
HTTP/1.1 403 Forbidden
Date: Fri, 28 May 2010 09:21:04 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8i DAV/2 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
Content-Type: text/html; charset=iso-8859-1
403 Forbidden, bad.
Any other referrer
telnet www.offshoreinjuries.com 80
HEAD /blog/deepwater-horizon-11/ HTTP/1.1
Host: www.offshoreinjuries.com
Referer: http://alvaro.es/
HTTP/1.1 200 OK
Date: Fri, 28 May 2010 09:20:36 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8i DAV/2 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
X-Powered-By: PHP/5.2.12
X-Pingback: http://www.offshoreinjuries.com/blog/xmlrpc.php
Content-Type: text/html; charset=UTF-8
200 OK again.
Your server is actively rejecting visitors from Facebook.
I was finally able to get to the bottom of this behavior.
The default mod_security settings of my host, HostGator include a set of whitelists and blacklists. Upon inspecting these I found .facebook.com/l.php blacklisted.
l.php is a wrapper page that provides a warning that you are leaving facebook. As I understand it since this can be easily exploited, HostGator chose to essentially blacklist all outbound facebook links.
I fixed my problem by removing .facebook.com/l.php from the mod_security blacklist, however I could have also just reset my mod_security settings to Default (vs the HostGator config) via a single click in WHM.