I created a site, but it went throught a increase in structure, and a new folder level was added in url. The site sends a 301 redirect to point users to new url.
The problem is: The facebook button with the Share count is zero. It was counting 408.
I know I can point the counter to the URL to old page, but How to do it correctly using the 301 redirect? I don't want the old url anymore, just to update the count to the new url.
old url:
http://biblia.apelosurgentes.com.br/biblia/
( http 301 )
new url:
http://biblia.apelosurgentes.com.br/pt-br/biblia/
I already Fetched new scrape information on the old url using the facebook debug tool https://developers.facebook.com/tools/debug/
But the count stills not getting transferred to new URL.
I see this as a facebook bug, because it is not following the 301 redirect rule. https://developers.facebook.com/bugs/589414401234160/
Related
On April 12th, Facebook upgraded the FB App we use to share new articles to our page to API version 2.6. Since then, stories get posted, but the image is usually not added to the story.
When I check with the opengraph debugger, I see strange errors like:
Circular Redirect
We could not resolve the canonical URL because the redirect path contained a cycle.
With redirect paths of:
Redirect Path
Input URL arrow-right https://www.bleepingcomputer.com/news/government/34-tech-firms-sign-accord-not-to-assist-government-hacking-operations/
301 HTTP Redirect arrow-right https://www.bleepingcomputer.com/news/government/34-tech-firms-sign-accord-not-to-assist-government-hacking-operations/
Even stranger, sometimes the redirect paths show urls like below, which are not from my site:
Input URL arrow-right https://www.bleepingcomputer.com/news/security/crooks-hijack-router-dns-settings-to-redirect-users-to-android-malware/
301 HTTP Redirect arrow-right https://www.bleepingcomputer.com/news/security/crooks-hijack-router-dns-settings-to-redirect-users-to-android-malware/?utm_content=70198165&utm_medium=social&utm_source=facebook
og:url Meta Tag arrow-right https://www.bleepingcomputer.com/news/security/crooks-hijack-router-dns-settings-to-redirect-users-to-android-malware/
Once I click on the scrape button again, it gets rid of the error, but the images still do not show in the post on my FB page.
It is almost as if the scraper is targeting URLs that are not mine, even though I am sending them correctly and I have the same urls in my og:url and canonical tags.
Any ideas?
We often change URL of articles on the site, and then the old URL returns 301 redirect to the new URL.
During a limited time we can still retrieve share count for the old URL (using FQL) and it continues to grow (I think because of re-shares). But after the page is re-scraped Facebook "realises" that both URLs point to the same page and the old share count is lost, both URLs return the same count:
http://graph.facebook.com/fql?q=SELECT+url%2C+total_count+FROM+link_stat+WHERE+normalized_url+in+%28%27http%3A%2F%2Fwww.dailymail.co.uk%2Fnews%2Farticle-3657160%2FBritain-brink-EU-referendum-result-knife-edge-Leave-scores-HUGE-victory-Sunderland-outperforms-predictions-Swansea-Remain-pins-hopes-London.html%27%2C%27http%3A%2F%2Fwww.dailymail.co.uk%2Fnews%2Farticle-3657160%2FDavid-Cameron-QUITS-Prime-Minister-voters-Brexit-EU-referendum.html%27%29
There are several questions:
Is there any way to "assign" share count of the old URL to the new URL?
Is there any way to retrieve old share count?
Maybe, there is a way to instruct Facebook to never re-scrape a particular URL so it can always have old share count available?
Would re-shares of old URL be added to share count of new URL (after the page is re-scraped and facebook links two objects)?
Scenario of the problem:
We enforced HTTPS on a website. Any URL with HTTP now redirects (301 permanent redirect) to an appropriate HTTPS URL.
To avoid Facebook like/share buttons (that are placed on many pages of the website) loosing previous numbers of likes/shares, we made the buttons to "link" to the old HTTP URLs via the "data-href" property.
Additionally we placed the "og:url" meta tag on some pages, pointing to the old HTTP URLs.
I then scraped that pages at the Facebook debugger tool https://developers.facebook.com/tools/debug to make sure the Facebook gets the fresh data. According to the scraped data, canonical URLs were indeed pointing to the old HTTP URLs just as it should be according to our actions listed above. This was also reflected in the like/share buttons on our pages keeping the old numbers.
A few days later I discovered that some pages lose the old numbers of likes/shares. Checking the pages in the Facebook debugger shows that Facebook now takes HTTPS URLs as canonical. We did not make any changes on our pages, and the "og:url" tag is still pointing to the HTTP URLs. But the Facebook wrongly takes HTTPS URLs as canonical URLs. Now if I scrape the information again in the debugger, it agains becomes normal, showing HTTP as canonical and restoring the old number of likes/shares. But obviously it's not a solution to the problem, because we cannot constantly monitor all our pages and scrape them again and again.
Any ideas of what may causing the problem?
Facebook follows HTTP redirects as well. You need to make your old HTTP URLs available to the scraper, without redirecting it to the HTTPS version. (The scraper can be recognized by its User-Agent, see social plugins FAQ.)
The old HTTP URLs need to be available to the scraper, and not redirected to HTTPS, as the FAQ also mentions:
“This also requires that the old URL still renders a document with Open Graph tags and returns a HTTP 200 response, at least when loaded by Facebook's crawler. If you want other clients to redirect when they visit the URL, you must send your 301 HTTP response to all non-Facebook crawler clients. The old URL should contain its own og:url tag that points to itself.”
I made an error when bulk 301 redirecting from an old domain to new domain with same url structure.
Google bot followed the 301 redirect for each page on the old site to my new site, which gave a 404 error. I tested it browser in worked for a user but somehow did not work for the google bot and I detected it too late.
I fixed the error now and the pages could be (hopefully) accessed by google bot at their new urls.
Question: will google recrawl 301 redirects which led to 404?
They'll crawl it for a little while but eventually the 404 status will tell them the page is gone and they will stop crawling it and remove it from their index. If you fixed the error before they stopped crawling the original URL then they will follow the redirect and associate the new URL with the old URL.
The Facebook OpenGraph debug tool is scraping the wrong page.
If I give it a full URL (pointing to an individual page on my site) that I want it to scrape, instead of scraping that page and finding its meta tags, it scrapes my site's main page and returns those meta tags (which are obviously wrong in this context).
The weird thing is, it will even find and scrape my site's main page even if it's not located at the root of my domain. For example:
I want it to scrape http://mydomain.com/myhomepage/specific_page.html
Instead, it scrapes http://mydomain.com/myhomepage/
This implies to me that the error must be a setting someplace, either on my site or on my Facebook App settings. Would the App settings do that? Redirect to whatever URL is set if a requested URL is a descendent of it?
The URL I'm requesting is not doing a 302 or anything - I can click the link from the FB debug tool even and it will take me to the appropriate page.
A few notes:
specific_page.html is not an actual file, it is routed through index.php using mod_rewrite in Apache's htaccess. I tried being specific with http://mydomain.com/myhomepage/index.php/specific_page.html and it did not work then either.
Another SO question led me to believe that the user-agent might be getting redirected if it doesn't allow cookies (as the Facebook web crawler does not) so I opened a fresh browser, disabled cookies, tried again, and I still reached the appropriate page.
As mentioned in the comments above, in your case this was due to an og:url meta tag, redirecting Facebook's crawler to that URL
In general, cases like this are usually the og:url tag, a HTTP redirect, or a canonical meta tag pointing at the 'other' / 'wrong' URL - Facebook's crawler follows those redirects looking for the final URL