Facebook Scrapes Beso.com so hard as to heavily impact site performance; how do we get FB to throttle its crawl? - facebook

Our site, Beso.com, which has 3MM UV a month, has started being hit by Facebook's crawler/scraper multiple times a day, to the point where it is causing a severe performance degradation on our site. We have recently implemented the Open Graph, and are encouraging a fair amount of sharing and liking on our site, along with Facebook Connect.
I understand that FB needs access to our site to scrape info, but we desperately need them to throttle down the rate of crawling, or we will be forced to block it entirely. This isn't a solution we wish to go with.
Would there be anyone from FB on this site who could connect us with an appropriate team member, or anyone from an established site who has battled this issue before?
Thanks.

Facebook scrapes open graph objects once per day. I can think of two possible reasons a site would be over-scraped...
1) You have an app that implements open graph actions. The programmer who created that app accidentally set the scrape optional parameter permanentaly to true. See here:
https://developers.facebook.com/docs/opengraph/actions/
2) A single page--what should be one open graph object--has multiple URL's such that Facebook is re-scraping the same material over and over.

Related

Facebook app id for website social plugins

I have like/share buttons on my website and am not currently using an appid. Finding information on whether this is a necessity for the above configuration is difficult to confirm. When going through the Facebook app "create" forms, I'm really confused by a lot of the requirements as it seems to be more focused around actual apps rather than websites, which login via fb etc.
In the past I have created an app for one of my other websites and I don't believe there was any verification/submission process at this point (about 2-3 years ago). Reviewing my old app configuration now I can see that it is not "live" - does anyone know whether Facebook implemented this submission process in the last couple of years, and then de-activated any live apps that were created prior to this?
I now want to setup a new app for my new website and I'm unsure what the different 'action types' mean and what I would need to simply use like/share buttons on the website. Could someone give me a quick run down of what I'd need and what they are?
Any other info/tips people can provide would be greatly appreciated. I'm finding that the like/share DO still work without the app, but sometimes the share button doesn't work properly (I have a feeling this is something to do with the app).
Thanks
If you are just using the Social Plugins, you don't need to have a Facebook App ID.
If you want more integration, it will be required. Creating an App ID is like registering your App, wether an iOS or Android App or just a website, so that you can make Graph API calls for it.
The easiest way to register your website is to go the quickstart: https://developers.facebook.com/quickstarts/?platform=web (or https://developers.facebook.com/apps). There you can type a name and follow the steps.
You can then follow the "Sharing best practices" (https://developers.facebook.com/docs/sharing/best-practices#tags) on how to use the App ID. For example, by including an fb:app_id meta tag on your pages.
There is a review process, since last April, but that is only needed if you use permissions for your App. If you just use the plugins, that is not needed.
Can you explain more what is exactly not working?
Update
The time you create a page the Facebook crawler does not know yet what the Opengraph data for that one is. This will happen after the first share, but if you want to ensure it is correct from the first share on, you can force a re-scrape.
This section explains how that works:
https://developers.facebook.com/docs/sharing/opengraph/using-objects#update
When an app triggers a scrape using an API endpoint This Graph API
endpoint is simply a call to:
POST /?id={object-instance-id or object-url}&scrape=true

How to get number of facebook likes from an individual app?

I've done some searching and can't seem to find an answer to this. I'd like to find out how many Facebook likes came from a specific app and not the overall fan page itself.
Set up:
I have a "fan page" for a company that I manage. This company runs promotions from time to time and usually do so through a FB app that is attached to the fan page. These apps vary in type, but usually are just a page tab app hosted on heroku with a like gate.
Obviously, a like on the like gate results in a like for the fan page. Is there any way to find out how many likes came from that specific like gate / page tab app? FB Insights shows "where likes came from" but nothing I've found lists individual apps.
Thanks in advance.
Even though from page_fans_by_like_source_unique you can get a lot of metrics, there isn't currently one to differentiate between tabs.
The approach I'm currently using, is tracking on my app side, saving the information into a database, because both like and unlike events refresh the app iframe, with a new signed_request. The logic is the one described by #dleavitt here
Sources: http://developers.facebook.com/docs/reference/api/insights/#like_sources

Why are my links to my site disappearing from feeds after 2-3 hours?

I have a web site that gets quite a bit of traffic from Facebook. I've never had any problems sharing links to it on my Facebook page, but starting this afternoon, whenever I post a link to my web site from my facebook page, the post disappears from all users' feeds after 2-3 hours. This has happened 3 times now and I can't understand why. At first I thought it was my page itself, but it seems to be my domain because when I share from a different page (the same domain), it also disappears.
It worked fine this morning but not now. What is going on? Is this a glitch or is it some kind of system FB has in place now? This is really hurting traffic and clicks out from FB to my site.
Thank you.
Edit: looking back, it seems this may have started happening around Tuesday. Is there some kind of system in place now to limit the amount of clicks out you can get from a post? I can't see any other reason it's happening...
There are several possible causes:
Links that you use is a dynamic link which at times can be changed, use the static link. Example your link like this: http://www.domain.com/page.php?var=xxxxxxxx, where the parameter xxxxxxxx values ​​on the links can change at any time may be due to activity updates. This will cause facebook can not find a link that has been logged.
Bandwidth & server connection you less stable.
If I may suggest, avoid links / URLs dynamically & use the SEO Friendly link/URL like this page URL http://stackoverflow.com/questions/8529022/why-are-my-links-to-my-site-disappearing-from-feeds-after-2-3-hours

Build a facebook app or web app?

I want to develop an online application and I am considering EITHER building a website with community features built in or building ONLY a facebook app. I was wondering if other people have had to make the same decision and what things I will need to consider.
The website I want to build will be an educational portal where people can make and take tests online
I disagree with some of the other answers here. There is a huge difference between a) trying to advertise a new place on the web and b) trying to advertise a new functionality of an existing place. Even if this new website would offer a very tight integration with Facebook and some other social platforms. Keep in mind: facebook users really don't like to leave facebook, no matter what the reason would be. That's why the click-through rate for the advertisements is so embarrassingly poor: http://en.wikipedia.org/wiki/Facebook#Company
That said you can of course always do both: build a website and offer the same functionality through a facebook app. However my opinion here is that if you're application only offers its users a single functionality, you'll be better off just doing the latter.
What exactly is the advantage you expect by creating a Facebook-ONLY-App for that?
If it's only about taking tests you can still build a "normal" portal and include some of the Facebook-functionality through the JavaScript-SDK, like posting to the wall, Single-Sign-On, find your friends and so on. This way the user still has the choice if he wants to connect with Facebook or not. This way you also don't minimize your userbase to Facebook-users
(yeah I know, "everyone" has Facebook these days... ;) Still not everyone wants it to be connected to every single site he's using through Facebook)
Considering this comment:
Well I guess its easier for people to recommend my app if it is a facebook app, is the main reason I want to know if facebook is a good option – Zubair Mar 3 at 14:51
Build a website and then add the Facebook 'like' button. See: http://developers.facebook.com/docs/guides/web/#plugins
You should put a Twitter button as well:
http://twitter.com/about/resources/tweetbutton
In my opinion you have to develop both. First develop you website. Although facebook is having millions of users But in my opinion you cannot implement every thing as freely as you can in your web application than facebook application. You should have a website and a page on facebook. You can integrate other facebook social plugins on your website to interact with facebook.
From monitization point of it is easy for new users on website than application on facebook. Other reason website especially related to educational purposes have a huge click through rate which you cannot find on facebook application advertiser.
What is the goal of the website?
If its to make money dont do a facebook app, you have far more control of your site by designing it entirely yourself.
If you want social networking features there are plenty of APIs you can tie in to which will provide you with functionality and allow you to link into facebook / twitter etc.
A website would look more professional, it would allow you to gather statistics on unique hits, revisits etc, having your own database of users means you can gather information and market your site more specifically ( which users took which tests).
A website also allows you to monetize it by adding advertisement if that is your goal, and you can gain search engine rankings.
If you want to get publicity for your website you can use facebook by creating a group / page for the site and promoting it that way.
Also your own website wont leave you vulnerable to changes in Facebook, what if you put in all this work and in a year the terms change and a portion of your app is now in violation of the terms. What if you want to add X feature and facebook wont allow it?
Basically your site = 100% in your controll, thats a big advantage to you. With facebook you loose that advantage but maybe gain a little in being able to use more of their features. Personally id always go for my own site.
You should go for the website first, then add the social elements in the website.
Like you can enable users to login using there facebook credentials. Like/share Button.
And later on, you can also go for the facebook app, when you want to shoot for much much more traffic. Therefore, whenever you think that you have figured out what exactly you want out of your application then only go for it, otherwise try your options with website. Because once your facebook app is up, you will get hell lot of traffic.
Let me know if you need help in creating facebook application or social elements enabled website. I have built an Facebook Easy API on top of all facebook features, which will enable you to easily access anything on facebook and meanwhile reducing your work effort.
You first build it like web app and use Graph API and FBConnect to use Facebook functionalities. Then you need to create a facebook app version also because getting facebook traffic is also required. People from facebook most like come to facebook app then to another web.
You will not need to convert it to facebook app, it will be just less in width and it would be a facebook iframe app. as I some where read that facebook is depreciating fbml and iframe app is recommended.
So now you can make both things, as I think , test app can have flexible layout so that you don't need to change width for facebook iframe. So you can both things by doing one.
thanks
i'm pretty sure many people will not agre with me, but IMHO you should focus on build a good Web-App that work well also on Mobile-Phones. keep it simple, intuitive, responsive, lightweight, cross-browser and straight to the point.
if your only concern is about "recommend your app to other people" make it SEO and Multi-Language too. google will do the rest.
then if you want make your app bold, slowly and planty of useless stuffs start to add all the facebook widget you want.
PS: i'm also on facebook, twitter,
flickr, google etc etc, i'm also
sharing photos, links and usefull
stuffs, my google rss reader is full
of links with tons of nice things, well i
have never had a minute to look at it, when i need something i just start searching google
I agree with most of the answers here—a native website is the way to go. Personally, I don't trust/like FB apps. Dunno what they do, and given the number of scammers out there and FB's lack of responsibility (IMO), I rarely if ever use an FB app.
Creating the website gives users choice about whether they want to share results/integrate with their FB wall/profile. Users don't like to be forced into something.
And in the spirit of adventure that is typical of SO, it's always more fun to build your own website than to build a template-based (sort of), boring and nearly irrelevant (drowning in a sea of other poorly made apps) FB app. But that's just my 2¢
In your case, I would do a hybrid. First, build your website, but integrate it with Facebook via connect. This way you can concentrate on building your value added services and let Facebook worry about the community.
I would also not ignore the Facebook app. Now, with iframes being fully supported on Facebook, you can adapt your existing site to work within Facebook with minimal effort, as long as you keep this requirement in mind when building your original application.

Facebook connect/oath for a newbie

I am currently starting to research a project that wants to use Facebook instead of a log in system.
the brief needs to be able to allow users to have their own customisation settings in a user panel, and also be able to connect with facebook friends within the context of the site.
Am I barking up the wrong tree?
Also, how much of the elements we pull from facebook can be styles our way. My designers is starting concepts before I get to that point of development and I don't want to waste their time designing layouts that can not be achieved.
Facebook allows you to pull a wealth of information. The user has to OK the permission for you app to access his/her Facebook profile, but you can get friends lists and their thumbnail pictures too!
Take a look here for a start:
http://developers.facebook.com/docs/authentication/
Its a bit of work, but worth it. You can pull straight data and store it on your server. According to Facebook, you can only keep the data for 24 hours or so (If I remember correctly) so you will need to "refresh" the data every day or so.