We have spent several days looking into FB Graph API and third party tools for scraping FB data but cant figure out if it is even possible to scrape what we are looking for and if it falls into FB policies (really not looking forward to start a lawsuit with FB).
We need to obtain statistic of how often is specific question (read - problem that we will try to solve) posted on Facebook. We need to get all FB posts filtered by three criterium:
Location - country or city of user that posted the post
Time - Some reasonable period of time, for example a full month, week or day
Keyword - keyword that can be associated with questions that we are looking for
We would then takes this data set and manually go over it in order to distinguish whats relevant to us and what is not. Maybe use some language processing engine like wit.ai or api.ai to use data set to teach app to regonize which posts are relevant and which not. But thats on us, later.
So the question: Is it possible (technically and also from FB policies point of view) and what would be the steps to get FB posts filtered by three criterium stated above?
I want to embed a stream of facebook posts onto a web page that is comprised of posts all around facebook containing a specific hash tag, but I'm not quite sure how to go about it. I had to do the same with twitter and instagram, but those were all fairly easy to accomplish. I'm just kind of looking for the best option right now, not so much on specifics. I've seen a couple of times the graph api for facebook as an option, but every time I see those they seem to be from a year ago, so not sure if it's out of date or if there is a better option out by now. Any recommendations on ways to go about it would be greatly appreciated.
See my answer here on how you can use the Search API for hashtags:
Need help on employing Graph Search parameters for hashtag query on facebook
Basically, you can call
https://graph.facebook.com/search?q=%23selfie&type=post&access_token={user_access_token}
Be aware that you have to use v1.0 of the Graph API, because in v2.0 searching for public posts will no longer be possible (https://developers.facebook.com/docs/apps/upgrading#upgrading_v2_0_graph_api):
Public post search is no longer available.
(/search?type=post&q=foobar)
Graph API v1.0 will only be available until 30th of April, 2015.
I am very new to Facebook Ads API so some help or advices about best practices I can use in my case would be appreciated a lot.
The situation:
We have multiple clients that are running advertising campaigns on social sites and Search Engines like Facebook, LinkedId, Google Adwords, Bing and etc. They want to have information from all their campaigns in one database and use this database to run various reports in order to manage their campaigns better (move money from one campaign to another and have better results). So basically we need to perform daily (or should I say nightly) sync. Also we need to mark adds in Facebook that are part of campaigns that are running via our tool (also not a problem to do once logged in). This is done automatically.
That is not a problem in many platforms - we use Google and Microsoft API to connect to the platform, read required data on daily basis or do some changes to URLs automatically.
The question
I was a bit surprised to find out that in Facebook Ads API the only way to authenticate is via tokens that expire quite fast (long term tokens last for 60 days). I am wondering if it is possible to authenticate via API with user name and password or obtain the ticket that is not expiring. The reason - most of our processes will run on nightly basis so any interaction with user to get fresh ticket is not possible.
Asking for permission every 60 days also seems like not an option and offline_access privileges that would suit us perfectly are no longer supported as I understood correctly from the documentation.
So what are authentication options? Is the only possibility to ask user to re-confirm our access rights every 60 days? Is it possible to get access permission once and live with it until user revokes it, or this is not possible?
Thanks in advance for your answers and suggestions!
Facebook did an exception for ads api access tokens. It is still possible to get neverending access token. Reed more here https://developers.facebook.com/roadmap/offline-access-removal. Also, you can check access token type, expiration date and other params here https://developers.facebook.com/tools/debug. Our access tokens shown as Expires: Never.
I want to find the list of all facebook users who attended a particular college using fql.
Is that possible?. If yes then, how do I write that query?.
It appears that this is not possible, looking both at FQL and the Graph API documentation. I don't even believe there's an easy way to get a list of fans of the page. The most obvious reason for them preventing this is to prevent marketers from getting lists full of personal information to use for sending promotions or offers (spam).
I'm a graduate student whose research is complex network. I am working on a project that involves analyzing connections between Facebook users. Is it possible to write a crawler for Facebook based on friendship information?
I looked around but couldn't find any things useful so far. It seems Facebook isn't fond of such activity. Can I rely on the Facebook API?
Update (Jan-08-2010): Thank you very much for the responses. I guess I probably need to contact Facebook directly then. Cheers
Update (Feb-16-2011): A new book, "Mining the social web", just came out. In it, there is a chapter devoted entirely for mining Facebook using Python. Cheers.
You can't rely on the Facebook API unfortunately. To get friend information, you need to use something like friends.get(). However, any Facebook API method that returns user information like this requires that you have an active session key from that user, and generally the way you get an active session key is to have the user come to your Facebook application or page.
In summary, the information you are talking about is essentially private. You can't pick a person from Facebook, get their friends, and get those friend's friends, and so on. To me this is a good thing for privacy, but of course it prevents arbitrary analysis.
I'd throw out the idea of writing a quick and dirty application with some user appeal that you could use for research. If a group like S**t My Dad Says (funny, not really safe for work) can get 120,000 users in a couple of months, you could probably plead your case with a small research application and get a reasonable amount of users.
The problem is that facebook friendship information is typically private and only accessible to friends. It should be a lot easier to build this network on Twitter, if this is an option for you.
As others have stated, this is typically private information. If, however, Facebook per se isn't a requirement, you could use Google's Social API. A snippet from the Google Social Graph API page: "With the Social Graph API, developers can now utilize public connections their users have already created in other web services. It makes information about public connections between people easily available and useful."
Here's an article on using it in Ruby:
http://articles.sitepoint.com/article/google-social-graph-api-ruby-rails#
This lifecode post provide a basic python script to scrape your facebook friends contact info.
The output of this script, is the profile ID, profile pame, profile URL, e-mail address and mobile/phone number (if provided by friend).
WARNING: This is against Facebook TOS. Use at your own risk.
Info provided for educational and research purposes
http://ruel.me/blog/2010/11/26/scrape-your-facebook-friends-contact-info-with-python/
You can use http://www.facebook.com/directory/ to get the public listed people.