I know pretty well that this topic has been discussed very intensively (I read a lot all over the day).
Still, how probable is it that FB might allow me to create a frontend crawler for a non-commercial, non-public research university project?
My crawler should repeatedly lookup a very limited number of specific public fan pages and collect anonymized data like number of fans, status updates and their number of likes and number of comments each.
What I would like to show is what kind of topics in media pages are "liked" and discussed most and how that develops over time. I know about FB's restricted TOS. Thanks for your opinion on that.
The second question concerns technological approach / authorization: Reading a fan page's number of fans, status updates and their number of likes each - could I even use the API/OpenGraph for such a crawler? I think for reading page walls, you need an access token at any cost, and realizing an automatic "crawler" via an application therefore is not possible I guess (as apps only react to users' actions and cannot act like cron jobs for example)?
As you see, I am pretty new to FB development and logic. Thanks so much for your expertise.
If you mainly target public pages then you should be ok.
You need to have a facebook app and then you can authenticate as the app from your program.
You will get an app token with which you should be able to use in order to crawl public pages data.
If you check the documentation for the Page object you'll see in the tables (fields and connections) that most of what's in the Permissions column is either "No access token or user access_token" or "any valid access_token or user access_token", if you have the app token you're good.
Also, and I think this is something you'll be interested in, the Page object has the "talking_about_count" field.
So, yes you can do it, at least most of it.
As for the TOS, since all of this is perfectly ok and straight from their official documentation, there's no problem.
Related
I need to quickly get the names of about 1000 users that I now only have the facebook id and access tokens of. I'm not comfortable with the FB api yet so I was considering just writing a scraper to retrive the name from the FB page of the user (since I have the id of the users).
Is this allowed? I assume it's not "best practice" but how severe is it? Will it get me banned for instance? The data will only be used to complete our user database so no advertisement
Alternativly: Can anyone point me in to a good (and up to date) guide of how to get user info using the FB api (keep in mind that I have the ID and the access.tokens of all my users).
No, scraping is not allowed and you MUST use the Graph API: https://www.facebook.com/apps/site_scraping_tos_terms.php
/me?fields=name&access_token=[user-access-token] returns the name of a User - You may run into API limits though, but if it´s a one time thing it should not really matter. If you run into limits, just wait a bit and get the next batch.
I'm planning on launching a website and an accompanying facebook page. Although each of these 'sites' will have their own goal, I'd like to have a 2-way sync set-up for part of the user generated content on my site, such that:
Content published on my site (by editors, or users) that satisfies some criteria are pushed to the wall of the accompanying facebook page
The other way around: content published on the facebook page wall by users, is pushed to my website. (I want to be in complete control of how to format, display, annotate that content, so I'm not looking for some ready-made widgets)
For 2. I'm pretty sure I can poll for a newsfeed or channel of the specific facebook page at regular intervals, but I'm wondering if a tighter integration is possible, i.e: pushing the changed content(delta's, river whatever you want to call it) to a dedicated rest-endpoint from which my website can consume the contents or something.
For 1. I need an automated way to post to the wall of the facebook page I own. Preferrably with the user-credentials (if okayed by the user) if that user was logged-in on my site through Facebook, or otherwise with some fallback credential, e.g: my own facebook credentials.
Just looking for some pointers where to start looking in the vast Developer Facebook resources, or, better yet, some libraries (prefferably Node.js, but others are okay to look at architecture) that take care of some of the plumbing
1) You can post on page with different ways
as a user : You need user access token and he should be a fan of the page or If your fan page allows wall posts from any user in its settings.
as a page : You need page access token
You can get page access token and extend it for long time and store it in DB. You can use it 60 days.
FYI:
Extended access token
Tutorial
2) Using graph API you can pull at regular intervals. There is a real time notification system also but it very limited for pages (its not useful for your case).
FYI: Real Time Updates
After some advice.
I have a client wishing to have an app which lets them download a voucher to redeem in store but only after they post details of this offer onto 5 friends' walls.
My understanding of current Facebook policy suggests that:
1) Promotions can't require users to take any action on Facebook other than to like the page (likegate);
2) Messages posted to multiple friends at once should be unique and personally initiated - i.e. an app shouldn't bulk post to multiple users. Instead users should share things on their own wall....
Am I correct in this understanding?
If so, can anyone point me at the relevant facebook policies - can't seem to find them on the site.
Yes, those are exactly what I read on the policies page. However to be absolutely sure, you should have your corporate attorney read them and consult with you.
See
http://developers.facebook.com/policy/
and
http://www.facebook.com/promotions_guidelines.php
I want to create an iPhone app that displays (among other things) a specific Facebook wall. For a good user experience I didn't want an app that required the user to have a Facebook account and I didn't want to force the user to have to log in to Facebook to see the latest "news" in the app. I started out by getting the wall RSS feed and tried parsing it ... I can "see" all the data I need ... but that is getting complicated quickly and has too many variables that are making the final results less than stellar. I have read through the Facebook iOS programming tutorials and it seems to me like the SDK forces the user log in, which I don't like.
My question ... Is there a way to use the Facebook SDK with hard coded profile credentials to access a specific wall without forcing the user to login? If possible, is that a recommended approach? Any other ways to skin this cat?
I have read through the Facebook tutorial and searched through many postings on this site but haven't found an answer to this ... sorry if this a newbie question and has already been answered.
Item I.2. of the Facebook API policy list says
You must not include functionality that proxies, requests or collects
Facebook usernames or passwords.
It sounds to me like that's what you're proposing to do; i.e., the user will be able to see a certain wall, but using hard coded credentials (not their own). In other words, your credentials are proxying for the user.
I do not know if it is technically possible to do this (I imagine it is) but I don't think it's a good idea, and I do think it's a violation of the Facebook API terms of service.
First you need to get the a access_token by parsing your app id and secret.
https://graph.facebook.com/oauth/access_token?client_id=YOUR_APP_ID&client_secret=YOUR_APP_SECRET
Then send following request to get the data you want. Note that only public data will be accessible.
https://graph.facebook.com/FACEBOOK_USER_ID/?access_token=ACCESS_TOKEN
I'm a graduate student whose research is complex network. I am working on a project that involves analyzing connections between Facebook users. Is it possible to write a crawler for Facebook based on friendship information?
I looked around but couldn't find any things useful so far. It seems Facebook isn't fond of such activity. Can I rely on the Facebook API?
Update (Jan-08-2010): Thank you very much for the responses. I guess I probably need to contact Facebook directly then. Cheers
Update (Feb-16-2011): A new book, "Mining the social web", just came out. In it, there is a chapter devoted entirely for mining Facebook using Python. Cheers.
You can't rely on the Facebook API unfortunately. To get friend information, you need to use something like friends.get(). However, any Facebook API method that returns user information like this requires that you have an active session key from that user, and generally the way you get an active session key is to have the user come to your Facebook application or page.
In summary, the information you are talking about is essentially private. You can't pick a person from Facebook, get their friends, and get those friend's friends, and so on. To me this is a good thing for privacy, but of course it prevents arbitrary analysis.
I'd throw out the idea of writing a quick and dirty application with some user appeal that you could use for research. If a group like S**t My Dad Says (funny, not really safe for work) can get 120,000 users in a couple of months, you could probably plead your case with a small research application and get a reasonable amount of users.
The problem is that facebook friendship information is typically private and only accessible to friends. It should be a lot easier to build this network on Twitter, if this is an option for you.
As others have stated, this is typically private information. If, however, Facebook per se isn't a requirement, you could use Google's Social API. A snippet from the Google Social Graph API page: "With the Social Graph API, developers can now utilize public connections their users have already created in other web services. It makes information about public connections between people easily available and useful."
Here's an article on using it in Ruby:
http://articles.sitepoint.com/article/google-social-graph-api-ruby-rails#
This lifecode post provide a basic python script to scrape your facebook friends contact info.
The output of this script, is the profile ID, profile pame, profile URL, e-mail address and mobile/phone number (if provided by friend).
WARNING: This is against Facebook TOS. Use at your own risk.
Info provided for educational and research purposes
http://ruel.me/blog/2010/11/26/scrape-your-facebook-friends-contact-info-with-python/
You can use http://www.facebook.com/directory/ to get the public listed people.