Facebook API batch request is very slow - facebook

I'm writing a web app that allows users to search data from their Facebook profile as well as that of their friends (who've also authorized the app). Initially I was looping through the friends and making separate API calls to get the friend data, but that was SUPER slow. So I switched to building an array of API requests and submitting one batch request.
The batch request is supposed to be the solution to my problem, but it's STILL agonizingly slow. My page loads are around 15 seconds and I cannot figure out why. The documentation claims that each request in the batch is processed in parallel, but it sure doesn't seem that way. Is it relevant that each of my batched requests takes an unique access token? The documentation doesn't indicate that this is a problem, but the documentation doesn't say a lot of things...
Here's an example of one of my batch queries:
[
{"method":"GET",
"relative_url":"\/#####\/friends?fields=name,first_name,last_name,id,work,education&access_token=#####"},
{"method":"GET",
"relative_url":"\/#####\/friends?fields=name,first_name,last_name,id,work,education&access_token=#####"},
{"method":"GET",
"relative_url":"\/#####\/friends?fields=name,first_name,last_name,id,work,education&access_token=#####"}
]
Just to give you some context, the app makes one API request to get the user's friends. Then it loops through those results and builds a batch request for each friend (that has authorized the app) and sends a second API batch request using the PHP SDK ($json_batch is a batch request like the one shown above):
$rawdata = $facebook->api('?batch='.$json_batch, 'POST');
(exemplified above). The cumulative results are checked for matches against the user's search query and echoed back to the user. Any ideas why this should take 20 seconds to happen??
UPDATE:
I added some code to track the time at various times during the execution of the script...
The entire class runs in 11-13 seconds. The first FB api call (to the graph) takes 0.6 seconds. The second batch call is 10-11 seconds! But WHY?

I'm not sure why the batch requests take that long, but to answer the access token per request, no it's necessarily to do so, unless you put different tokens, if you are using the same token then you can simply (as in the documentation):
curl \
-F 'access_token=…' \
-F 'batch=[ \
{"method": "GET", "relative_url": "me"}, \
{"method": "GET", "relative_url": "me/friends?limit=50"} \
]'\
https://graph.facebook.com
You can however get the same exact information using a single FQL query:
SELECT uid, username, name, first_name, middle_name, last_name, work, education
FROM user
WHERE uid IN (SELECT uid2 FROM friend WHERE uid1 = me())
This also removes the need to query separably for the friends of the logged in user first, but if you do have a list of friend ids you can put those in the IN clause instead of the inner query.
Maybe this fql solution will return faster.

Related

Get posts from multiple pages

I'm trying to fetch most recent posts of several pages using facebook FQL. Here is my query:
# https://developers.facebook.com/tools/explorer/145634995501895/?fql=SELECT%20message%20FROM%20stream%20WHERE%20source_id%20in%20(755751084454823%2C%20616632928446954)
SELECT message FROM stream WHERE source_id in (755751084454823, 616632928446954)
And it returns 0 results. On the other hand, sending exact same query but for a single page ID returns posts for that given query properly. So, those two queries works great:
# https://developers.facebook.com/tools/explorer/145634995501895/?fql=SELECT%20message%20FROM%20stream%20WHERE%20source_id%20in%20(616632928446954)
SELECT message FROM stream WHERE source_id in (616632928446954)
# https://developers.facebook.com/tools/explorer/145634995501895/?fql=SELECT%20message%20FROM%20stream%20WHERE%20source_id%20in%20(755751084454823)
SELECT message FROM stream WHERE source_id in (755751084454823)
I can't quite understand why it behaves like that. Any help much appreciated.
To get posts from multiple Pages, use separate calls to the feed endpoint with the Graph API. Example code for this can be found in the docs:
https://developers.facebook.com/docs/graph-api/reference/v2.6/page/feed
You can use Batch Request to make those calls faster, it will take as long as the slowest API call in the batch: https://developers.facebook.com/docs/graph-api/making-multiple-requests
FQL is deprecated and will stop working in August 2016. You can´t use it in new Apps anymore.

Limit Number of Posts coming from /feed Facebook Graph API

When I use /{page_id}/feed?access_token=xxxx, this give me all the posts on the page, both by user and page. I want to limit and control the posts. I want to put constraints like:
Timestamp (that is to get posts after a particular timestamp)
Post id (to get post after a particular post)
Since getting all the posts from feed is irrelevant and in-effective. Is there any way to accomplish this ?
You can use
GET /{page_id}/feed?limit={nr_of_posts_to_return}&since={timestamp}
to be able to limit the number of results and specify the starting timestamp. Have a look at the reference here:
https://developers.facebook.com/docs/graph-api/using-graph-api/v2.0#paging
For your second Use Case you'd need to use the Batch API imho, because with a single Graph API request you can't filter on specific Posts. Instead, you need to use the Batch API to split this in two queries as described here:
https://developers.facebook.com/docs/graph-api/making-multiple-requests/#operations
The request would then look like this:
curl \
-F 'access_token={your_access_token}' \
-F 'batch=[{ "method":"GET","name":"get-post","relative_url":"{your_post_id}?fields=created_time"},{"method":"GET","relative_url":"{your_page_id}/feed?since={result=get-post:$.created_time}&limit={nr_of_posts_to_return}"}]' \
https://graph.facebook.com/
In Graph Explorer, you have to change the HTTP method to Post, then add a new field called batch. Leave the URL blank so far. Paste this as batch value:
[{ "method":"GET","name":"get-post","relative_url":"​293088074081904_400071946716849?fields=created_time"},{"method":"GET","relative_url":"293088074081904/feed?since={result=get-post:$.created_time}&limit=1"}]
This works at least for me.
For others looking for a solution, it appears the 'since' done at the 'comment' and 'reply' levels are ignored. Which means this is not a solution for me.
The query Tobi provided will provide all the posts after the first 'since' but every comment and reply in those posts, regardless of that you set their 'since' to.
Further to this, if you wish to search for new comments , regardless of the age of the post, this fails as well. For example:remove the first 'since' and change to limit=1000 and only request comments as a fields using 'since' , this will return the last 1000 posts and all comments for all of those 1000.
That said, thank you Tobi for your time and showing me how to get everything I need in a single function call. I may experiment parsing the complete recordset every time. ( maybe too much traffic though!)

FQL/GRAPH Api Logic

I have a logic problem I can't seem to solve (might be possible).
Example:
I am inside 100 facebook groups
I need the 10 lastest posts of EACH group I am in.
That's pretty much it but I can't seem to find a way to do this without making a foreach loop calling the api over and over again, if I had a couple hundred more groups it would be impossible.
PS: I'm using FQL atm but am able to use graph, I've coded this in like 3 different ways but no success.
This is the farthest I could get:
SELECT actor_id,source_id FROM stream WHERE source_id IN (select gid from group_member where uid = me())
It only returns from one page, maybe there's no way to return all of this without a foreach asking for each groups 10 lastest messages.
There's no need to use FQL of batching. This can be done with a simple Graph API request IMHO:
GET /me/groups?fields=id,name,feed.fields(id,message).limit(10)
This will return 10 posts for each of your groups. In case there too much data to be returned, try setting the limit parameter for the base query as well:
GET /me/groups?fields=id,name,feed.fields(id,message).limit(10)&limit=20
Then, you'll get a next field in the result JSON. By calling the URL contained in this field, you'll get your next results. Do this until the result is empty, then you reached the end.
You can use batch calls, described here https://developers.facebook.com/docs/graph-api/making-multiple-requests/
Using batch requests, you can request upto 50 calls in one go. Note than batch request doesn't increase the rate limits, so if you make 50 requests in batch, it will be considered as 50 calls, and not one. However you will get the response in a shorter time.
If you think you're making too many calls, you should put some delay in between calls and avoid rate limiting.

Facebook FQL multiquery vs batch requests

I am developing a Facebook app and I need to fetch a lot of data from users. I am concerned about limits Facebook has. I can't send queries one by one, I would soon get error 4 "API Too Many Calls". So I am wondering what is a better approach, if grouping queries into multiquery or sending them in batch. Btw I will have to use multiqueries in batch request. So here is an example :)
Suppose that USER_A and USER_B are friends so I can access them both with 1 access token.
multiquery for USER_A:
"Q0":"SELECT post_id FROM stream WHERE source_id=<USER_A>",
"Q1":"SELECT id FROM comment WHERE post_id IN (SELECT post_id FROM #Q0)"
multiquery for USER_B
"Q2":"SELECT post_id FROM stream WHERE source_id=<USER_B>",
"Q3":"SELECT id FROM comment WHERE post_id IN (SELECT post_id FROM #Q2)"
So what is better approach, grouping these 2 multiqueries into 1 bigger multiquery or putting them into a batch request? In which case will Facebook count it as 1 API call? Facebook has some limit like 100M API calls per day. So if I send my 2 multiqueries in batch / bigger multiquery, will facebook count it as 1 API call?
EDIT: by batch request i mean this https://developers.facebook.com/docs/reference/api/batch/
I am not asking if I should use FQL or Graph API, I want to know if there is a difference between sending multiqueries in batch and sending them as 1 bigger multiquery..
If you can do everything with one access_token, in my experience, a large FQL multiquery returns the results more quickly than a batch request.
Also, looking at your example query, you could cut the number of queries down by combining these:
'User_A_Comments':
"SELECT id, post_id FROM comment WHERE post_id IN (SELECT post_id FROM
(SELECT post_id FROM stream WHERE source_id=<USER_A>) )"
The advantage that a batch request would give you is you could use multiple access tokens to get this information in one shot.

FQL query response randomly omits some results

I'm trying to make a simple "only status updates, nothing else" app (since FB doesn't seem too interested in this sort of thing). However, my queries seem to omit some results:
"select status_id,uid,time,message from status where uid in (select uid2 from friend where uid1= $my_id) ORDER BY time DESC LIMIT 25", for instance, when I last tested it, only returned 21 results (and yes, I've got enough friends who have made enough updates that there are definitely more than 25 historical statuses).
This is with the PHP API, by way of Heroku, but I've had similar issues for a while, going back to before FBML was deprecated, with basically the same query.
I haven't tested enough to determine absolutely that this is the case, but it seems only to return one status per user (if Bob posted six updates, it only returns his newest status and ignores the previous ones).
Is this a known issue/bug? Any workarounds?
facebook trades accuracy for performance it has been always the case , so it usually returns a limited number of results per query
It's probably because a user has not given access for a friends apps to access their posts. Apparently making an FQL request, facebook does the initial request, including your limit param. Then it filters the list removing anything that you don't have permissions to see. Hence receiving less than 25 results.