Scrape new page using opengraph and curl - facebook

I am developing a website in which i have created a blog, on that blog people can comment via their facebook. Now i noticed that whenever i create a new blog, the comments plugin shows a warning "url is unreachable".
I already figured out that the way to get rid of this warning is to scrape the new blog.
if i use following query on the commandline:
curl -F "id=http://www.maartenvangenechten.be/blog/post/13/" -F "scrape=true" -k https://graph.facebook.com
the warning disappears, but on the long run this isn't the best way. Also all the data i putted in metatags are outputted, telling me that the page is succesfully scraped
so i tried using php/libcurl for this:
$params = array(
"id"=>$url,
"scrape"=>"true");
$ch = curl_init("https://graph.facebook.com");
curl_setopt_array($ch, array(
CURLOPT_RETURNTRANSFER=>true,
CURLOPT_SSL_VERIFYHOST=>false,
CURLOPT_SSL_VERIFYPEER=>false,
CURLOPT_POST=>true,
CURLOPT_POSTFIELDS=>$params
));
$result = curl_exec($ch);
curl_close($ch);
echo $result;
now this only outputs:
{"id":"214022612077699","url":"http:\/\/www.maartenvangenechten.be\/blog\/post\/13\/"}
and not
{"url":"http:\/\/www.maartenvangenechten.be\/","type":"website","title":"Maartens Homepage","image":[{"url":"http:\/\/www.maartenvangenechten.be\/images\/general\/logo_enlighten.gif"}],"description":"Hier kan je alles vinden over mijn huidige projecten. Bekijk ook zeker de blog, waar ik de verschillende uitdagingen die ik tegenkom zal toelichten","site_name":"VangenechtenDESIGNs","admins":[{"id":"591822147","name":"Maarten Van Genechten","url":"http:\/\/www.facebook.com\/exquisitje"}],"updated_time":"2013-02-22T02:27:18+0000","id":"492686967461912","application":{"id":"482576148470885","name":"MVGPortfolio","url":"http:\/\/www.facebook.com\/apps\/application.php?id=482576148470885"}}
as i would expect
Can't seem to find why

Ok, solved it, after searching the web for a couple of hours...
I tried the function on different browsers, and Opera, Firefox, and even IE returned the expected result, only Chrome gave the problem...
Cleared the cache, history, and about everything else stored via Chrome, and the problem was gone.

Related

Facebook profil picture simple api

I start to code in PHP and I am facing a problem, I used classic method to get User Profil Picture :
https://graph.facebook.com/mark/picture
But as you know, Facebook with little regard for small coders like me. They have cleared this easy method. Now I have to connect on their API to get a facebook profile picture.
I have used fil_get_content on the Mobile Facebook site to retrieve the profile image, but it asks for a captcha.
I am FORCED to go through their API, also I created a APP_ID and APP_SECRET but I do not know how. Despite all the tutorials I've tried these last two hours.
If someone would have a solution, it would be nice.
Thank you, good evening.
Use following Code to get your profile picture
$request = new FacebookRequest(
$session,
'GET',
'/me',
array(
'fields' => 'picture'
)
);
$response = $request->execute();
$graphObject = $response->getGraphObject();
/* handle the result */
Now print your response using $response variable.
In a response you got a url.

Facebook wall post error: OAuthException :: (#1500) The url you supplied is invalid

I have a web-based news app that runs on Heroku. When users post a comment to a news story on my app, my app forwards the comment to the user's facebook wall using fb_graph. Everything worked perfectly until a couple of weeks ago. For no reason that I can explain I am now seeing some baffling behavior.
Now, when a user submits a comment to a story the FB API responds with, OAuthException :: (#1500) The url you supplied is invalid. If, the same user then submits additional comments to the same story those comments are posted to the user's FB feed just fine.
I have used the FB Graph API explorer to confirm that I have valid access tokens, and that my app does accept posts to the token-owner's FB feed.
To make things even more baffling, when running my web app in development on localhost all of the posts go through just fine to my development FB app.
def post_to_facebook(story, post)
auth = Authentication.find_by_provider_and_user_id("facebook", current_user.id)
if auth
me = FbGraph::User.me(auth.token)
if me.permissions.include?(:publish_stream)
begin
me.feed!(
:message => "#{best_name(current_user)} made the following post to NewsWick: #{post.contents}",
:name => story.title,
:link => "https://www.newswick.com/stories/"+story.id.to_s,
:description => "Story posted to the NewsWick world-wide news service" ,
:picture => best_photo(story)[:photo_url]
)
rescue => e
#msg = "Facebook posting error: "+ e.to_s
puts "Facebook feed posting error: #{e.message}"
end
else
#msg = "No longer authorized to post to Facebook."
end
end
return #msg
end
One last thing to note, the only thing that I have changed w/r/t how my app interacts with FB in the last two weeks was that i accepted FB's July Breaking Changes.
Anyone have any clues. This is driving me bonkers!!!
I'm having the same issue only difference is I'm using the javascript api.
Seems like it's a facebook bug, which is already reported here: https://developers.facebook.com/bugs/136768399829531
Yes this is a known bug and Facebook developers are looking into it, well so they claim,however something interesting I found out is:
I post to my Facebook using 2 methods using RestFB API , first, for messages with URLs e.g www.something.com and those without URLs, I realized last night that all posts without URL worked and the ones with URL doesn't.
So I changed all my implementation to send messages to Facebook without using with link parameters for all posts, with or without links.
With link Parameter - throws error #1500
FacebookType publishMessageResponse = resftFBclient.publish(FACEBOOK_PAGE_ID
+"/feed", FacebookType.class, Parameter.with("message", "Hello StackOverFlow!"),
Parameter.with("link", "message with a link , www.me.com"));
With no link parameter - this works even if message contained URL/link
FacebookType publishMessageResponse = resftFBclient.publish(FACEBOOK_PAGE_ID. +
"/feed",FacebookType.class,Parameter.with("message", "My message"));
This works even if message contained URL/link and it creates an clickable link on FB. Could it be that FB is trying to drop the link implementation and letting us figure it out that the former works just as the link implementation? What's the difference anyways?
That's brutal!
Cheers
Babajide
I was trying to solve this problem this problem that seems to be occurring to almost everyone.
I am using the PHP SDK.
What I noticed is that it always returned this error for the first time I tried to post the link. On a second try, it was posted with success.
Really hackishly I then checked for an error and retried to post to the wall.
$errorCount = 0;
function postPicture($phrase)
{
try
{
$image = $_SESSION['photoLink'];
$facebook->setFileUploadSupport(true);
$response = $facebook->api(
'/me/feed',
'post',
array(
'message' => $phrase,
'picture' => 'http://mylink/pictures/facebook.png',
'link' => $image,
'caption' => 'My caption',
'description' => 'My description',
'type' => 'photo',
'name' => 'My name'
)
);
echo 'Success';
}
}
catch (FacebookApiException $e)
{
// You really should check if this $error is #1500 before doing that. I didn't :)
if($errorCount < 2)
{
postPicture($phrase);
$errorCount++;
}
else
{
$e = str_replace('"', "", $e);
$e = str_replace("'", "", $e);
echo 'Error ' . $e;
}
}
}
To solve these problems just make sure you add these og metadata tags in the head section of the page represented by the url you want to share:
<meta property="og:type" content="article" /> //or any other type like blog, website etc....
<meta property="og:url" content="your article url here" />
<meta property="og:title" content="your article title here" />
Good luck!

How do I force the Facebook Like Button to re-scrape my page?

I created a like button on a page (http://www.usna.edu/BillAF/game.php) that sends the URL/data for it's parent page. Unfortunately, when I first put it in, an older version of the page was at the location of the parent page that did not have the open graph meta-tags on it.
So when someone "likes" the page, it sends the wrong title and a "garbage" image of half of the letter "e" on a white and gray square.
I checked: http://developers.facebook.com/tools/debug and it has the updated info. I also tried changing the associated AP ID to a new one, and neither seems to have refreshed what it sends when I "like" it. I also tried adding index.php to the end of the URL it is supposed to be posting (http://www.usna.edu/BillAF) but to no avail.
Is there anything I can do to force it to re-scrape the page? If not, does anyone know how long it will take for facebook to re-scrape it on their own?
A good way to clear the facebook cached image and text is by running the tool Debugger:
http://developers.facebook.com/tools/debug
By putting the url that you are sharing there, it will refresh facebook cache from your website.
Facebook has an API for re-scraping page data, I use the following function:
function purgeFacebookCache($url, $debug=false) {
// Set the variables for Facebook
$fields = array(
'id' => urlencode($url),
'scrape' => true
);
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');
// Connect
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL,'https://graph.facebook.com');
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
$result = curl_exec($ch);
// Debug
if ($debug) {
var_dump($result);
}
}

How to refresh Facebook Like Image (clear cache) [duplicate]

We used the link:
http://www.facebook.com/sharer.php?u=[shared URL]
...to share a particular page. However, Facebook Sharer uses the cached version of the images and the title. Is there a way to quickly clear the Facebook cache or how long do we have to wait until the data gets updated?
I placed <link rel='image_src' href='[preview image]' /> in between the tags.
I found a solution to my problem. You could go to this site:
https://developers.facebook.com/tools/debug
...then put in the URL of the page you want to share, and click "debug". It will automatically extract all the info on your meta tags and also clear the cache.
Use api Is there an API to force Facebook to scrape a page again?
$furl = 'https://graph.facebook.com';
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $furl );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_POST, true );
$params = array(
'id' => '<update_url>',
'scrape' => true );
$data = http_build_query( $params );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $data );
curl_exec( $ch );
$httpCode = curl_getinfo( $ch, CURLINFO_HTTP_CODE );
Facebook treats each url as unique and caches the page based on that url, so if you want to share the latest url the simplest solution is to add a query string with the url being shared. In simple words just add ?v=1 at the end of the url. Any number can be used in place of 1.
Hat tip: Umair Jabbar
I thing these two links have a wide discussion on your problem related stuff. fb:ref clear cashes by calling
fbml.refreshRefUrl
like this
<tt>fbml.refreshRefUrl("http://www.mysite.com/someurl.php")
You can study the related stuff from here fb:ref. I hope it will work for you
The page to do this is at https://developers.facebook.com/tools/debug/ and has changed slightly since some of the other answers.
Paste your URL in there and hit "Debug". Then hit the "Fetch new scrape information" button under the URL text field and you should be all set. It'll pull the fresh meta tags from your page, but they'll still cache so keep in mind you'll need to do this whenever you change them. This is really critical if you are playing with the meta tags to get FB Shared URLs to format the way you want them to inside of facebook.
This answer is intended for developers.
Clearing the cache means that new shares of this webpage will show the new content which is provided in the OG tags. But only if the URL that you are working on has less than 50 interactions (likes + shares). It will also not affect old links to this webpage which have already been posted on Facebook. Only when sharing the URL on Facebook again will the way that Facebook shows the link be updated.
catandmouse's answer is correct but you can also make Facebook clear the OG (OpenGraph) cache by sending a post request to graph.facebook.com (works for both http and https as of the writing of this answer). You do not need an access token.
A post request to graph.facebook.com may look as follows:
POST / HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: graph.facebook.com
Content-Length: 63
Accept-Encoding: gzip
User-Agent: Mojolicious (Perl)
id=<url_encoded_url>&scrape=true
In Perl, you can use the following code where the library Mojo::UserAgent is used to send and receive HTTP requests:
sub _clear_og_cache_on_facebook {
my $fburl = "http://graph.facebook.com";
my $ua = Mojo::UserAgent->new;
my $clearurl = <the url you want Facebook to forget>;
my $post_body = {id => $clearurl, scrape => 'true'};
my $res = $ua->post($fburl => form => $post_body)->res;
my $code = $res->code;
unless ($code eq '200') {
Log->warn("Clearing cached OG data for $clearurl failed with code $code.");
}
}
}
Sending this post request through the terminal can be done with the following command:
curl -F id="<URL>" -F scrape=true graph.facebook.com
Append a ?v=random_string to the url.
If you are using this idea with Facebook share, make sure that the og:url param in the response
matches the url you are sharing. This will work with google plus too.
For Facebook, you can also force recrawl by making a post request to https://graph.facebook.com
{id: url,
scrape: true}
I just posted a simple solution that takes 5 seconds here on a related post here - Facebook debugger: Clear whole site cache
short answer... change your permalinks on a worpdress site in the permalinks settings to a custom one. I just added an underscore.
/_%postname%/
then facebook scrapes them all as new urls, new posts.
if you are using wordpress with a cache plugin, make sure you clear all your caches. Also make sure the image you are using has the recommended facebook size: 1200(w) x 630(h) or 600 x 315.
If you used managed wordpress or caching plugins, you have to CLEAR YOUR CACHE before the facebook debugger tool can fetch new info!
I've been pulling my hair out for weeks figuring out why changes I made wouldn't show up in facebook debugger for 24 hours!!!! The fix is I have to go into my wordpress dashboard, click the godaddy icon on the top, and click "flush cache." I think many managed wordpress hosters have a cache to figure out how to clear it and you'll be golden.

How to clear Facebook Sharer cache?

We used the link:
http://www.facebook.com/sharer.php?u=[shared URL]
...to share a particular page. However, Facebook Sharer uses the cached version of the images and the title. Is there a way to quickly clear the Facebook cache or how long do we have to wait until the data gets updated?
I placed <link rel='image_src' href='[preview image]' /> in between the tags.
I found a solution to my problem. You could go to this site:
https://developers.facebook.com/tools/debug
...then put in the URL of the page you want to share, and click "debug". It will automatically extract all the info on your meta tags and also clear the cache.
Use api Is there an API to force Facebook to scrape a page again?
$furl = 'https://graph.facebook.com';
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $furl );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_POST, true );
$params = array(
'id' => '<update_url>',
'scrape' => true );
$data = http_build_query( $params );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $data );
curl_exec( $ch );
$httpCode = curl_getinfo( $ch, CURLINFO_HTTP_CODE );
Facebook treats each url as unique and caches the page based on that url, so if you want to share the latest url the simplest solution is to add a query string with the url being shared. In simple words just add ?v=1 at the end of the url. Any number can be used in place of 1.
Hat tip: Umair Jabbar
I thing these two links have a wide discussion on your problem related stuff. fb:ref clear cashes by calling
fbml.refreshRefUrl
like this
<tt>fbml.refreshRefUrl("http://www.mysite.com/someurl.php")
You can study the related stuff from here fb:ref. I hope it will work for you
The page to do this is at https://developers.facebook.com/tools/debug/ and has changed slightly since some of the other answers.
Paste your URL in there and hit "Debug". Then hit the "Fetch new scrape information" button under the URL text field and you should be all set. It'll pull the fresh meta tags from your page, but they'll still cache so keep in mind you'll need to do this whenever you change them. This is really critical if you are playing with the meta tags to get FB Shared URLs to format the way you want them to inside of facebook.
This answer is intended for developers.
Clearing the cache means that new shares of this webpage will show the new content which is provided in the OG tags. But only if the URL that you are working on has less than 50 interactions (likes + shares). It will also not affect old links to this webpage which have already been posted on Facebook. Only when sharing the URL on Facebook again will the way that Facebook shows the link be updated.
catandmouse's answer is correct but you can also make Facebook clear the OG (OpenGraph) cache by sending a post request to graph.facebook.com (works for both http and https as of the writing of this answer). You do not need an access token.
A post request to graph.facebook.com may look as follows:
POST / HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: graph.facebook.com
Content-Length: 63
Accept-Encoding: gzip
User-Agent: Mojolicious (Perl)
id=<url_encoded_url>&scrape=true
In Perl, you can use the following code where the library Mojo::UserAgent is used to send and receive HTTP requests:
sub _clear_og_cache_on_facebook {
my $fburl = "http://graph.facebook.com";
my $ua = Mojo::UserAgent->new;
my $clearurl = <the url you want Facebook to forget>;
my $post_body = {id => $clearurl, scrape => 'true'};
my $res = $ua->post($fburl => form => $post_body)->res;
my $code = $res->code;
unless ($code eq '200') {
Log->warn("Clearing cached OG data for $clearurl failed with code $code.");
}
}
}
Sending this post request through the terminal can be done with the following command:
curl -F id="<URL>" -F scrape=true graph.facebook.com
Append a ?v=random_string to the url.
If you are using this idea with Facebook share, make sure that the og:url param in the response
matches the url you are sharing. This will work with google plus too.
For Facebook, you can also force recrawl by making a post request to https://graph.facebook.com
{id: url,
scrape: true}
I just posted a simple solution that takes 5 seconds here on a related post here - Facebook debugger: Clear whole site cache
short answer... change your permalinks on a worpdress site in the permalinks settings to a custom one. I just added an underscore.
/_%postname%/
then facebook scrapes them all as new urls, new posts.
if you are using wordpress with a cache plugin, make sure you clear all your caches. Also make sure the image you are using has the recommended facebook size: 1200(w) x 630(h) or 600 x 315.
If you used managed wordpress or caching plugins, you have to CLEAR YOUR CACHE before the facebook debugger tool can fetch new info!
I've been pulling my hair out for weeks figuring out why changes I made wouldn't show up in facebook debugger for 24 hours!!!! The fix is I have to go into my wordpress dashboard, click the godaddy icon on the top, and click "flush cache." I think many managed wordpress hosters have a cache to figure out how to clear it and you'll be golden.