Facebook debugger scrapes default Apache page instead mine - facebook

I made a site: http://pravo-trans.eu/
There is all needed og meta tags. But when I want to share link on any social networks nothings happens. I thought it might be cach. But when I used facebook debugger, it said:
The 'og:type' property is required, but not present.
And it's not true because I wrote in <head> this:
<meta property="og:title" content="Проект правовой помощи людям" />
<meta property="og:type" content="website" />
<meta property="og:image" content="/transgender-project.jpg" />
<meta property="og:description" content="Бесплатные юридические консультации и представительство по делам о смене документов (внесение изменений в записи о рождении, паспорта, трудовые книжки, документы об образовании и другие документы)" />
<meta property="og:url" content="http://pravo-trans.eu/" />
<meta property="og:locale" content="ru_RU" />
<link rel="canonical" href="http://pravo-trans.eu/" />
And most strange thing for me happen when I click on "See exactly what our scraper sees for your URL". There I saw that debugger parsed Apache default page instead mine! https://developers.facebook.com/tools/debug/og/echo?q=http%3A%2F%2Fpravo-trans.eu%2F
How it can be and how I can fix it?

After several hours of trying to debug this issue and playing with DNS settings/servers, I have a solution that works for me.
I noticed that requests from Facebook were coming from an IPv6 server, but my Apache VirtualHost declarations did not include the IPv6 address. To debug, I changed the following line in my Apache .conf file:
<VirtualHost IPv4:80>
to:
<VirtualHost IPv4:80 [IPv6]:80>
...and immediately upon restarting Apache, Facebook was able to successfully scrape my site. (Replace IPv4/IPv6 above with your actual addresses of course.)
If by chance you are using Parallels Plesk, as am I, then this is not a permanent solution because Plesk will rewrite the configuration files, so you have to go into the Plesk panel and make sure that your server's IPv6 address is assigned to the Subscription that owns the domain in question. In my case, only the IPv4 was assigned to the subscription.
The setting can be found under "Change Hosting Settings" for each particular Subscription.

Related

Facebook OpenGraph Issue

I have done extensive searches on the Internet for a solution to this issue, but all that I can find is always related to making timeout adjustments on a Linux machine running Apache. I am running IIS version 10 on Windows 2019 Server. When Facebook changed it's website approximately 30-days ago, the Open Graph image sharing protocol stopped working properly. An attempt to use the Facebook Developer scraper, I get the following timeout error.
Curl Timeout
The request to scrape the URL timed out.
Curl Error
Curl error: 28 (OPERATION_TIMEOUTED)
I also filed a bug report with Facebook, but they simply closed the report, stating that the problem is with my server or network connection. I opened and inspected the server's error logs, and found no issues. I then setup and inspected the IIS logs, and found that Facebook indeed hit the server properly and fetched an image and reported it back. But the timeout error still occurs and the image is not shared upon an attempt to share it. Here are the records from the IIS logs that seem to indicate that Facebook is indeed contacting my server correctly, except for the fact that they are using "http" rather than "https." This has been reported to Facebook.
2020-12-24 18:31:51 W3SVC3 EDENUSA-FS11 10.1.252.250 GET /images/qr_code/edenusa_qr_code.png - 443 - 69.171.249.113 facebookexternalhit/1.1+(+http://www.facebook.com/externalhit_uatext.php) - www.edenusa.com 200 0 0 70
2020-12-24 18:32:02 W3SVC3 EDENUSA-FS11 10.1.252.250 GET /rent-lighting/lighting/rent_lighting.asp - 443 - 69.171.249.111 facebookexternalhit/1.1+(+http://www.facebook.com/externalhit_uatext.php) - www.edenusa.com 200 0 0 21410
And following is a snippet of the required meta code in our header area, from the home page:
<!DOCTYPE html>
<head>
<title>Rent a Stage | Rent a Sound System | Rent Lighting System | Rent Up Lighting</title>
<meta prefix="fb: https://ogp.me/ns/fb#" property="fb:app_id" content="1376081292633720">
<meta property="og:url" content="https://www.edenusa.com/index.asp" />
<meta property="og:image:type" content="image/jpeg" />
<meta property="og:title" content="Rent a Stage | Rent a Sound System | Rent Lighting System | Rent Up Lighting" />
<meta property="og:image" content="https://www.edenusa.com/images/homepage/compressed/indian_temple_in_chino_hills.jpg" />
<meta property="fb:app_id" content="1376081292633720" />
I've worked on this for over a week now, without resolution. Anybody else having this issue, or know of a way to resolve the timeout issue?
This issue was resolved as follows:
We had to remove REST code in the GLOBAL.ASA that goes out and fetches geographic info (City and State only) based upon the client's IP address. The service endpoint is a bit slow, and required that a longer timeout than might be considered "normal" to be used. So when this code branch was commented out, the Facebook CURL timeout error no longer occurred. We are looking at another IP geographic info service that is faster.
After completing step 1, we found that on the home page ONLY, we had to leave the INDEX.ASP portion of the URL in place. We had code the stripped the "index.asp" off the canonical URL. For unknown reason, Facebook looks at the HTTP header, sees that the original URL has the "index.asp" included, and then compares that to the URL specified in the "og:url" meta tag.
In conclusion, the most recent rollout of Facebook includes new code that configured a shorter timeout value for CURL. This caused websites out on the web with a somewhat shorter startup time, to experience this issue. So for now, the only fix is to monitor a site's startup time, and shorten it down enough for the Facebook debugger/scraper to function as it did before the most recent changes.

facebook share Image not showing

I have a sharepoint webpage that has a facebook share button.
however after I shared the webpage, the image is not showing in facebook.
Using the facebook tool 'Open Graph Object Debugger', I got the following error
Errors That Must Be Fixed
Missing Required Property The 'og:type' property is required, but not present.
When Viewing the source of the webpage, I can clearly see that the required facebook meta data is there
<meta property="og:url" content="https%3a%2f%2fxxx.xx.xx/xx" />
<meta property="og:type" content="website" />
<meta property="og:title" content="xxx" />
<meta property="og:description" content="" />
<meta property="og:image" content="https://xxx.xx.xx/xxx.jpg" />
Anyone has any idea?
Check if Facebook Crawler gets your page at all. There is a link at the bottom that allows seeing what Facebook scraper gets from your site. If it's empty, most probably your web server responds differently or doesn't respond at all to Facebook. I have a similar issue. In my case scraper gets the page over HTTP and gets nothing over HTTPS (SSL) despite trusted certificate.
If facebook gets nothing from you server reasons are:
a bad configuration of a web server
firewall blocks facebook scraper's IPs
redirect issue
Also, check status code facebook provides in Debugger if it's not 200 or 206, the problem is definitely not in an Open Graph itself.
In case it's SSL issue and you're using Letsencrypt this SSL params made my site visible to Facebook again:
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
# Disable preloading HSTS for now. You can use the commented out header line that includes
# the "preload" directive if you understand the implications.
#add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
More details might be found here: https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
I had this problem. The solution for me was to call my hosting provider (Godaddy in my case) and tell them to re-start PHP processes on my site.
The issue is related to caching. Your server may be serving old content in an attempt to save time.
I then visited https://developers.facebook.com/tools/debug/ and clicked the "scrape again" button.

Why is gcloud failing to access all Google APIs and printing out HTML inside a Compute Engine VM?

When I SSH into a server, running gcloud compute instances list should list the instances but instead I see the following in the terminal:
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 403 (Forbidden)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{
margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images
/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-de
coration:none}a img{border:0}#media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;pad
ding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-r
epeat;margin-left:-5px}#media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/br
anding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/
images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}#media only screen and (-webkit-min-device-pixel-rati
o:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-we
bkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>403.</b> <ins>That’s an error.</ins>
<p>Your client does not have permission to get URL <code>/0.1/meta-data/service-accounts/default/acquire?scope=http
s%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform%20htt
ps%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute</code> fro
m this server. <ins>That’s all we know.</ins>
I have the correct scopes added to the machine so this should work. It's also peculiar that a command line application tool is printing out HTML (´・ω・`)
Trying to access any other Google API from the VM also fails.
It seems that gcloud is trying to authenticate, but failing to do so, because the service account on the vm is damaged in some way. The error should probably be cleverer in such a circumstance. I will relay it to the teams responsible for the tool.

Redmine rest api returns html document

I try connect pycharm/small python projects to my redmine server, but rest api return this:
Error fetching issues for: /projects//issues.xml?key=(my secret key)&fields%5B%5D=status_id&operators%5Bstatus_id%5D=o&values%5Bstatus_id%5D%5B%5D=1&per_page=10, HTTP status code: 200
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd">
<html>
<head>
<meta http-equiv='content-type' content='text/html; charset=UTF-8'>
<meta name="VERSION" content="">
<meta name="DATA" content="fwdnode7-bl.web-hosting.com (my redmine server)">
<link href=" " rel="shortcut icon" type="image/x-icon">
<title></title>
</head>
<frameset rows='100%, *' frameborder=no framespacing=0 border=0>
<frame src="http://(my redmine server)" name=mainwindow frameborder=no framespacing=0 marginheight=0 marginwidth=0></frame>
</frameset>
<noframes><h2>Your browser does not support frames. We recommend upgrading your browser.</h2><br><br>
<center>Click <a href="http://(my redmine server)" >here</a> to enter the site.</center>
</noframes>
</html>
Information about environment:
Redmine version 2.5.1.stable.13174
Ruby version 2.0.0-p481 (2014-05-08) [x86_64-linux]
Rails version 3.2.18
Environment production
Database adapter Mysql2
REST API and JSON are enabled in redmine's setting
How can I resolve this issue?

Warning: "my page" is unreachable

Ok, so i am developing a new site, and it is very dependent of facebook.
I have looked everywhere and done everything i should but i keep getting this message in my FB comment area: Warning: http://www.videozoo.dk/?videos=klo-aben is unreachable
My header looks as it should like this:
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:fb="http://www.facebook.com/2008/fbml" xmlns:og="http://ogp.me/ns#">
<head profile="http://gmpg.org/xfn/11">
<meta property="fb:admins" content="my fb id nr"/>
<meta property="fb:app_id" content="195385377211689">
<meta property="og:title" content="Videozoo.dk - Dyre video" />
<meta property="og:type" content="Video" />
<meta property="og:url" content="www.videozoo.dk/?videos=klo-aben" />
<meta property="og:site_name" content="Dyre videoer for alle!"/>
<meta property="og:description"
content="Endnu en dyre video på VideoZoo.dk"/>
My fb comment code looks like this:
<div class="fb-comments" data-href="www.videozoo.dk/?videos=klo-aben" data-num-posts="10" data-width="640" data-colorscheme="dark"></div>
My app id for this comment box was made 4 hours ago so it should be updated on the servers by now...
The information in the app matches what is stated above:
Application ID/API Key:
195385377211689
Site-URL:
http://www.videozoo.dk/
Domain:
videozoo.dk
BUT - It is still not working! - is it because my site is new or mabye because the app is not updated on the servers yet??
Please have a closer look and maybe test at this link: www.videozoo.dk/?videos=klo-aben
All ideas are welcome!!
You left the protocol out of your value for og:url, include it there and it may solve the issue, the scraper follows HTTP or og:url redirects, and that URL is likely detected as invalid
{edit} I figured this out, and it's a bit strange, but... {/edit}
When I manually scrape that page it seems to work fine, but when I run it through the URL Debugger it fails due to a HTTP 403 response from your side
I've seen this before with other servers which can't handle some part of Facebook's request - in this case it seems to be because your server is rejecting the request if a HTTP 'Range' header is sent.
Facebook's crawler only requests the first 40KB of the document when scraping, as the meta tags should be in the <head></head> section
My test was:
$ curl -I -H 'Range: bytes=0-40960' 'http://www.videozoo.dk/?videos=klo-aben'
HTTP/1.1 403 Forbidden
Date: Wed, 30 Nov 2011 14:17:54 GMT
Server: Apache/2.2.6 mod_auth_kerb/5.3 PHP/5.2.17 mod_fcgid/2.3.5
Accept-Ranges: bytes
Connection: close
Content-Type: text/html
$ curl -I 'http://www.videozoo.dk/?videos=klo-aben'
HTTP/1.1 200 OK
Date: Wed, 30 Nov 2011 14:18:02 GMT
Server: Apache/2.2.6 mod_auth_kerb/5.3 PHP/5.2.17 mod_fcgid/2.3.5
X-Powered-By: PHP/5.2.17
Connection: close
Content-Type: text/html
I'm not sure if this is something in your code, server config, an intermediate proxy, etc, but it's very likely the cause of your problem