I am developing a social media monitoring application. Currently, we are entering Facebook page ids into the application to collect data from possible customers' Facebook walls (so we have a realistic sample for the customer for direct promotion).
These page ids are used to collect wall postings and comments and to compute statistics (e.g. to show most used words), and are presented to the user in a special view. Requirements are to collect all postings and comments without exception in near-live time. We currently have about 130 page ids in the system, with more to come.
Right now, I am using the Graph API for this, with several disadvantages:
FB API access is restricted to 600 request/10 minutes. To get a near-live view, I need to access the API at least each two hours. As we are using API requests in other parts of the program, too, it is obvious that the limit is hit sooner or later (actually, this already happens)
The responses are mostly redundant: to receive current comments, I have to request the wall postings (comments are enclosed in postings) with the URL http://graph.facebook.com/NAME/feed...
The probability for hitting the limits is dependent on the number of postings on the several walls
I cannot get all comments with this method (e.g. comments on postings some time ago)
I am currently trying out how to switch to (or to complement Graph API usage) using FQL by querying the stream and the comment tables but this also has limitations:
I cannot restrict my query to a specific timespan, leading to redundancy again
The max number of posts I am getting for each one of my 130 page ids is 61 - (why 61?)
I need an unpredictable number of additional requests because I need to get special objects like videos and links in separate requests.
My question now is - if anyone is doing similar things: How did you solve these problems? How do you get a pseudo-live-stream of a larger number (up to, say 1,000) of walls?
Letting the customer grant extra permissions to us is currently not an option.
you will probably have to meet with FaceBook and work out a contractual deal for greater access to their data. I would bet that the answer will be no, no and no, seeing as it appears you are trying to monetize their data, and furthermore, do so without the explicit permission of the users, but hey give it a shot.
I have a similar task - By default FB return only last ~50 posts or all in last 30 days(whichever is smaller) in FQL you should use created_time filter to receive more results. my current problem is that via FQL I receive no more than ~500 posts from any FB page wall even when LIMIT increased:
'select post_id from stream where source_id = 40796308305 and created_time <'.time().' LIMIT 1000000 ;'
this FQL request to CocaCola FB Page returns now ~300 posts only (less than 2 day posts).
If you find a better solution pls advise :)
Related
While figuring out how to use the Instagram Graph API for Hashtag searches I found out that the rate limit kind of makes it impossible for my use case to search hashtags. Maybe I just don't know the best practices for the Hashtag search so here is my use case:
A client wants an application to manage coupon/discounts initiatives for his clients. Each of those clients have a store and want to offer discounts for everyone posting on instagram with a store-specific hashtag. My client wants to create/update/delete stores in an application and define the hashtag for a store. Thats all done. Now I need to trigger hashtag searches for each store. And heres the problem:
There are 70 stores (at first, may be more in the future). Each store has a unique hashtag and wants to know who posts something with their hashtag. When someone posts with a hashtag the application should know after max. 20 seconds. This means triggering a hashtag search 70 times (because there are as many hashtags as there are stores) every 20 seconds (I know once I get the hashtag ID I can save it and don't need to search for the acutal term anymore, but I still need the newest media for the hashtag ID).
Thats one thing. The other thing is I can only query 30 different unique hashtags in a week (facebook api limitation). But I will need to query at least 70 different hashtags. And the stores don't want to create a facebook account and instagram business page, so they can oauth into my application just for that.
So at the moment the way of doing it is kind of create 70 fake-pseudo-accounts (one for each store) and use those acocunts to prevent hitting the rate limit. But I don't think this is the way facebook wants me to use the graph api.
In this post Facebook API rate limit increase an answer from 2017 shows that there is a possibility to request a rate limit increase and in the linked picture is even the sentence "This is a page management app with few users but many calls". But more recent comments show that this is not possible anymore and I couldn't find any rate limit increase option in the developer portal.
https://stackoverflow.com/a/39408611/6315447 This answer even says there is no possibility anymore and If you hit the rate limit you're doing something you are not supposed to. But how should I use the API then?
Am I missing something? What is the best practice for such a use case?
Thanks in advance.
We have spent several days looking into FB Graph API and third party tools for scraping FB data but cant figure out if it is even possible to scrape what we are looking for and if it falls into FB policies (really not looking forward to start a lawsuit with FB).
We need to obtain statistic of how often is specific question (read - problem that we will try to solve) posted on Facebook. We need to get all FB posts filtered by three criterium:
Location - country or city of user that posted the post
Time - Some reasonable period of time, for example a full month, week or day
Keyword - keyword that can be associated with questions that we are looking for
We would then takes this data set and manually go over it in order to distinguish whats relevant to us and what is not. Maybe use some language processing engine like wit.ai or api.ai to use data set to teach app to regonize which posts are relevant and which not. But thats on us, later.
So the question: Is it possible (technically and also from FB policies point of view) and what would be the steps to get FB posts filtered by three criterium stated above?
Let's say a user selects 50 facebook ids that are his friends.
I take each facebook id and post a message on each person's wall. (Making 50 publish_stream api calls)
What is the limit for Facebook? How do I check how many I can post on behalf of the user?
There is no longer a limit on doing this. But in my company we stopped practicing publish_stream at all after Facebook banned 3 applications using this and we had to make long conversations with them. It's written nowhere on what basis but there is definetly an algorithm catching applications that publish a lot.
When we did that I found many comments on different limits. It seems that it all depends on the application history. This means, if your applicatin is new and generates many streams in the first day it will be caught as spamy. If the app has long history and large user base you can afford that.
Using Facebook's Graph API, I am currently looping through all friends to retrieve their videos, and then present a list of these videos to the user. It obviusly takes a while to do this depending on number of friends,
Is there any way to just say to FQL - give me all videos that I have permsission to view?
Facebook doesn't want applications to be able to "mine data". They don't want applications to be able to collect that much data on anything they want. In most cases an ID must be specified.
User ID
Page ID
Group ID
Event ID
etc...
Even in such a case there is also limitations and multiple calls will have to be made. The limitations themselves are not only limited to accessing data with FQL or the Graph API.
There are also limitations on your access to the API. This is called application throttling and it can be enforced on your application if they find you doing an abnormally large amount of calls to the API. There are also other limitations such as making multiple sequential posts or even duplicate posts to your users.
I'm afraid that the answer is no.
Just playing around with the graph API explorer:
http://developers.facebook.com/tools/explorer?method=GET&path=fql%3Fq%3DSELECT%20vid%2C%20owner%2C%20title%2C%20src%20FROM%20video%20WHERE%20owner%3Dme%28%29
I know that to get a list of my friends I can do:
SELECT uid2 FROM friend WHERE uid1=me()
I then updated the video query to:
http://developers.facebook.com/tools/explorer?method=GET&path=fql%3Fq%3DSELECT%20vid%2C%20owner%2C%20title%2C%20src%20FROM%20video%20WHERE%20owner%20IN%20%28SELECT%20uid2%20FROM%20friend%20WHERE%20uid1%3Dme%28%29%29
And now I get a paginated list of all videos belonging to my friends. This one query is certainly better than pounding the API to death with multiple calls. :)
I'm trying to get statistics for likes on my domain. I would like to get all likes (if possible with user ids) for all pages on my domain (which has tens of thousands of pages)
What does domain_like_adds actually return?
SELECT metric, value FROM insights
WHERE object_id=[domain-id] AND
metric='domain_like_adds' AND
end_time=end_time_date('2011-01-03')
AND period=period('month')
Returns blank, does anyone know what data domain_like_adds returns?
Regards,
Niklas
I don't think there's any way you're going to get user IDs as that is a major privacy invasion, but I believe domain_like_adds indicates how many NEW likes your domain got in the given time period, as opposed to the cumulative likes your domain has earned until that point. It doesn't appear there's a viable way to determine the # of likes of all objects in your domain for all time without tracking it from the beginning and/or going back and summing up historical data.
You can make a sitemap.xml of your site and crawl the urls against the Facebook Graph API. I actually made a Ruby script to do this: http://bobbelderbos.com/2012/01/ruby-script-facebook-like-stats-blog/. I don't think you can get the users that 'liked' your pages, but this script might be useful to find out what URLs are most popular.