Efficient Facebook Page data processing - facebook

There are multiple similar questions, but I could not find any helpful answer.
I have created Facebook application and going to fetch Page/Post data from Facebook on behalf of that application.
- I have special read_insights permissions on Page.
- I created access token with that permission for my Facebook App. (which will be automatically extended).
Today I faced to problem with getting transient errors:
Error, Code: 4, Message: Application request limit reached
Error, Code: 17, Message: User request limit reached
There are answers related these problems (Facebook api: (#4) Application request limit reached , Facebook OAuth Error: Application request limit reached).
According to the Facebook Platform Policy you should contact them,
If you exceed, or plan to exceed, any of the following thresholds please contact us as you may be subject to additional terms: (>5M MAU) or (>100M API calls per day) or (>50M impressions per day).
My App performed 300K calls during week , so it shouldn't exceed these limitations. I have contacted to Facebook but they haven't reply yet.
I would like to know what are the best practices for fetching data from Facebook. I need to fetch
1. /page_id?fields...
2. /page_id/posts?fields...
3. /post_id/likes?fields...
4. /post_id/comments?fields...
5. /page_id/insights/
6. /post_id/insights/
For 2-4 I can't use batch requests because of paginated results, I am taking with limit=100, for 5-6 I have created batch requests with specific insight URLs that I need, and already can't decrease number of calls anymore.
When I faced to
Error, Code: 4, Message: Application request limit reached
I created new App and for fetching Insights I granted read_insights permission by same Facebook user.
Then I faced to
Error, Code: 17, Message: User request limit reached
So I assume that if token would be generated by other user who has read_insights permission to new application it could work.
Can someone suggest what will be best approach to solve my problem? Should I configure multiple Page access tokens for my App (generated by different users) and switch between them when for one of tokens Error #4 or Error #17 is occurred?

Have you tried the following?
Using the realtime updates API to be notified about new posts and new comments on posts, rather than polling
(looks like you are doing this) Ensuring you're using the page access token for a page to fetch data for that page, and not the token of a user or another page
Caching data that's unlikely to change to avoid re-fetching
Requesting multiple posts in a single call rather than one post per call (using the ?ids=x,y,z syntax documented here: https://developers.facebook.com/docs/graph-api/making-multiple-requests )
Using those methods you may be able to avoid hitting this entirely

Related

How to Avoid Facebook Graph API Limit with million of users

I have a WordPress webpage with posts retrieving from a public Facebook page. The FB page is not mine. However the problem is that I have millions of visitors on my Web page and every time a user visits the web page it make an API call to FB Page. Since facebook allows only a limited number of API calls in a time frame, My limit reaches instantly with such a huge number of visitors. Is there any solution to this problem. an idea in my mind is:
1. retrieve posts from Facebook and store them locally and display them every time a user visits. Is it possible? If Yes where to start ?
Or can we get more API calls by paying facebook or things like that. I am ready to pay as far as my API calls can be made sufficient for my needs.
I am open to any solution and would be very thankful for any help to resolve the problem.
There are several possible solutions to this problem
Storing responses in database
You can add a middlepoint to your requests to Facebook API using your application. This would mean that you would have a database table which stores Facebook-related information, possibly along with a lifecycle time, like:
facebook_data(user_id, lifecycle_time, ...)
Whenever you would theoretically need to send a request to Facebook API, you can check the database table to see whether the user already has a record in that table and whether it is still valid. If so, give this data to the user. If not, send an API request to Facebook and store the response in this table.
Storing responses in localStorage/memory/file
You can also store Facebook-related data in the localStorage of the web browser of the memory of an app or a file, or even a local database specific for each user. This would prevent a lot of communication and server load your app is issuing on your server(s).
Queueing user requests to be sent
If the Facebook-related data is not very urgent to your users, you can queue requests to be sent and send a single request instead of a request for each user's each visit. You can do this via a cron job.
Periodically sending requests to Facebook
You can group your users into batches and periodically update their values via Facebook and storing in your database.
Combination
Naturally, you can combine the approaches, for instance, you can store in local memory, file, or localStorage values and in the database in the same time, so first locally stored information is searched for, not needing even a request if it exists and is still valid. If not, then checking the database record and using that if it exists and is still valid. And if the data is not found in local resources, nor your database, then you can send an API request.

Facebook GraphAPI Reduce amount of data with limit

So I'm struggling to find where this is documented (if at all), but I'm getting the following error message when requesting data from the FB GraphAPI.
"Please reduce the amount of data you're asking for, then retry your request"
The call I'm making is:
/v2.3/user1/posts?fields=object_id&limit=100
If I change it to:
/v2.3/user2/posts?fields=object_id&limit=100
It returns 100 items.
Why would it work for one user, and not the other?
Both requests are authenticated via an access token (not belonging to either user) and I get the same error whether running it from my code, or the Facebook Graph API console of developers.facebook.com
The response from CBroe is correct. Facebook returns this error if it finds that too many internal resources are needed to respond to your request.
Therefore you have to do what the response says: limit it.
This can be done in (afaik) 2 ways:
Use the limit parameter and reduce the amount of responses you expect from the API
Provide a timeframe (using since and / or until) to fetch only data (posts / videos) for a specific timeframe.
We had the same issue as you, but with retrieving videos from a page. Unfortunately using the limit parameter did not work, even when I set it to limit=1. But by using the since / until parameters we finally got results.
Therefore I suggest to implement a timeframe in order to reduce the amount of data, or alternatively, split the amount of requests you make. e.g. if you want all posts from the past 3 months and run into the mentioned error: split your requests in half using since and until. If that still does not work: keep splitting...
=> Divide and conquer ;)
Hope it helps,
KR, ebbmo
Recent bug filed on FB talks about the same error. They seem to accept that this could be a bug, but not much other information forthcoming.
https://developers.facebook.com/bugs/1904674066421189/
There are both app-level and user-level rate limits that are enforced on Graph API calls. In your case, it could be that you've made a large number of calls in a short time with user1.
You can check out this page for more about Facebook's rate limits: https://developers.facebook.com/docs/marketing-api/api-rate-limiting (even though the URL refers to the Marketing API, the information also applies to the Graph API.)

Facebook api, feed vs post and the weird results (and maybe some other feedback)

after studying pretty long on the FB api i finally understand the context these 'apps' work in and get access tokens etc. etc.
There is however, something that's bothering me. I'm pretty sure someone already asked, but i can't find the right search words, and im sorry in advance!
Ok here goes:
I'm building a webpage onto which i would like to show my last 10 facebook posts. The authentication flow i'm using is:
login to my site (Django app),
use the javascript api to log in to FB and get an access token,
send this access_token to the server via and xhr request,
have the server send a request to FB, changing the access_token into
a long-lived token and storing that token in the database so i can
use that long lived token for later requests.
QUESTION 1: Is this the right way to go?
QUESTION 2: I read and confirmed that my long-lived tokens are
valid for 60 days. The docs say this token is refreshed
automatically, how does this work, do i just have a log in again?
With the token (currently i just use access_token - short one - ) i query '/me/feed' or '/me/posts/' (all i want are status updates, photo's, shared links)
I understand the '/me/feed' > '/me/posts', because essentially, '/me/posts' are a subset of '/me/feed'.
QUESTION 3: Am i correct assuming posts are a subset of feed?
The results i am getting from these queries seem 'a bit off', as the feed edge gives me the 3 latest objects, whereas the posts edge gives me 16 post objects. I am playing with the 'limit' parameter, but that just responds 'erratically'. I found out the default limit is 500.
QUESTION 4: Why do i get more results from the posts edge than from
the feed edge?
QUESTION 5: Why don't i get 500 results as per default?
QUESTION 6: How come using limit=10 does not work properly?
Hopefully someone can point me in the right direction and help me with my questions.
Thanks in advance,
Niels

Application Request Limit issue (Occuring Random with Random Scenarios)

I have tried raising this concern on Facebook/Support/Bugs but they said I should post implementation issues here. I have read it everywhere and it seems to be quiet open issue till now. I am not sure, If this will be solved or not.
So, what we are doing is, we have clients - Android and iOS.
Apps on Android/iOS allows users to login into the app and generate the token on the basis of permissions set we have, and we are passing this token to server for fetching further data as and when required for client. As our userbase is increasing we are getting Application request limit reached quiet often.
We are fetching photos of users and their friends using FQL. So, when parallely fetching photos for around 8-10 different users, we are reaching the Application request limit sometimes, which is quiet random and we are not aware of the actual scenario when it breaks up and how. According to facebook the limit, which is 1M calls per day, but we are hitting around 80K - 1 Lac API calls in a day, but as users are increasing it is stretching a bit further, Less than or equal to 200 approax calls/user. We tried doing batch calls as well and we hit the application request limit as well.
If anyone of you could help us understand the complete concept of API limit and how this can be handled, then we will really appreciate the help. We want to understand how API limit is decided and it's rate is calculated over which interval so that we will be able to configure on our side accordingly.
Earlier in the day, we ran into a unique API call issue. Our server started to break for API calls for user tokens that are with us, we (on our systems, other than server) tried fetching the data for those tokens (Simple calls - /me or /me/home), and it was working alright for us but not for server, then we tried setting up another server and redirected the requests to our new server then this server works well for the same set of users. Not sure, what went wrong in this case and how it breaks up. Please help.
Many Thanks,
Reno Jones
Did you look at the Insights -> Developer section of developer.facebook.com for your app?
This will show you a breakdown per api call, including warnings and ones that are currently being throttled and why.
Also, are you sure you're using User token authorization and not just your App token?
Beyond that, we take the information from Insights to find api calls to cache on our side rather than hitting Facebook every time. You will likely have to do something similar if you're not already. They have limits for calling too often, as well as for requesting too much data. For those, we had to reduce the limits of historical data we requested.

Facebook Application Request limit reached

I am getting an FBerror "This operation can't be completed: Application request limit reached".
Does anybody know why is it so? How to check the limit? How to increase the limit? What depends on the limit allocation?
I recently ran across this issue doing a large number of requests using an application access token (the initial project requirements mandated that the user shouldn't have to authorize the app).
After much frustration, we finally were put in touch with a contact at Facebook who provided the following info in response to my question regarding request limits:
There is a limit, but it's pretty high, it should be difficult to hit unless they're using the same access tokens for all calls and not caching results, etc. It's 600 calls per 600 seconds per access token.
Ultimately we ended up requiring the user to authorize, as Facebook does not seem to distinguish between user access tokens (one token per user) and application access tokens (one token for all users) when calculating its seemingly arbitrary request limits.
If you are running into this error with a user access token, you may need to optimize your API calls (possibly by combining FQL queries or replacing multiple Graph requests with a single FQL query).
try this with your php code:
50 continuous FQL calls. After a pause of 10 seconds (sleep (10)) You repeat.
if($nr%50==0)
{
sleep(10);
echo "\n\n---Bloque #".++$numBloque."---\n\n";
}