I'm building a Facebook application and I want it to be able to read all the user's statuses from the past year. When I make the API call, I can only retrieve the first 100 statuses no matter what I set the limit to.
Here's the URL I'm using to make the call:
https://graph.facebook.com/me/statuses?limit=100&access_token=...
When I set the limit lower, it shows fewer statuses (proving that the limit argument works). When I set the limit higher, it only gives me the first 100. When I use 'since', it still only gives me 100.
When I use the 'next' url it gives me, I see no data past the first 100 statuses.
I know it's possible to get much more than that because of applications such as My Year In Status
It does work. You must use both the limit and offset query parameters. The limit parameter sets the batch size. The offset parameter sets the position in the user's all-time status collection. Without specifying the offset parameter, Facebook defaults it to zero, which is why you keep seeing the same dataset.
For the 1st batch of 100 statuses, set limit to 100 and offset to 0.
For the 2nd batch of 100 statuses, set limit to 100 and offset to 100.
For the 3rd batch of 100 statuses, set limit to 100 and offset to 200.
For the 4th batch of 100 statuses, set limit to 100 and offset to 300.
And so on...
Keep iterating until you get an empty dataset:
{
"data": []
}
I've verified using the Graph API explorer that the pagination is not working as you have described. Log it as a bug with Facebook at: https://developers.facebook.com/bugs and post the bug # here.
EDIT
Per the bug closure, the 100 limit is By Design and you won't get more than that, meaning that Facebook has made a conscious business decision to limit the amount of data it has to store, process, and serve from the Graph API. It costs money to do so and since the API is free to use, I can't argue with them. However, if I was paying for it, then hell yes I kick and scream all the way down the road.
Paging and Since, Until parameters not working for me too.
But I found out, that I'm able get more then 100 statuses using Offset parameter.
Here is my code using Facebook C# SDK:
var fb = new FacebookClient(accessToken);
string connection = "statuses";
string urltemplate = "https://graph.facebook.com/me/{0}?limit={1}&offset={2}";
int limit = 25; //items on one page (Max is value 100)
int offset = 0;
var statuses = (IDictionary<string, object>)fb.Get(string.Format(urltemplate, connection, limit, offset));
while (statuses != null && statuses.ContainsKey("data") && ((Facebook.JsonArray)statuses["data"]).Count > 0)
{
var dataItems = (Facebook.JsonArray)statuses["data"];
foreach (Facebook.JsonObject item in dataItems)
{
//TODO: process item data
System.Diagnostics.Debug.WriteLine(item["message"]);
}
offset += limit;
statuses = (IDictionary<string, object>)fb.Get(string.Format(urltemplate, connection, limit, offset));
}
This worked as of a week ago, our best bet to get this fixed is to post on http://developers.facebook.com/bugs/155458081230560 so the facebook developers know how big of an issue this is.
Facebook closed the previous bug without really testing to see that it didn't work. I've made a new bug here: https://developers.facebook.com/bugs/242235199184053
Related
I have a table of posts. I would like to query these posts as pages. Because I would like to keep my endpoints stateless I would like to do this with offset and limit like this:
SELECT * FROM post LIMIT 50 OFFSET $1 ORDER BY id
Where $1 one would be the page number times the page size (50). The easy way to check if we have reached the end would be to see if we got 50 pages back. The problem of course is if the number of pages is divisible by 50, we can't be sure.
The way I have solved this until now is by simply fetching 51 posts per query with the page size still being 50. That way if the return query is less than 51, we have reached the end.
Unfortunately, this seems a very hacky way to do this. So I was wondering, is there some feature within pg-promise or postgresql that would indicate that I have reached the end of a table without resorting to tricks like this?
The simplest method with the lowest overhead I found:
You can request pageLimit+1 rows on every page request. In your controller you will check if rowsCount > pageLimit and will know that there is more data available. Of course, before returning the rows, you would need to remove the last element and send along the rows something like a hasNext boolean.
It is usually way cheaper for the DB to retrieve an extra row of data than count all rows or make an extra request for page+1 to check if it returns any rows.
Well there is no built in process for this directly. But you can count the rows and add that to the results. You could then even give the user the number of items or number of pages:
-- Item count
with pc(cnt) as (select count(*) from post)
select p.*, cnt
from post p
cross join pc
limit 50 offset $1;
-- page count
with pc(cnt) as (select count(*)/50 + ((count(*)%50)>0)::int from post)
select p.*, cnt
from post p
cross join pc
limit 50 offset $1;
Caution: The count function can be slow, and even when not it does add to response time. Is it worth the additional overhead? Only you and the user can answer that.
This method works well only in specific settings (SPA with caching of network requests and desire to make pagination feel faster with pre-fetching):
One every page, you make two requests: one for the current page data and one for the next page's data.
It works if you for example use a React Single-Page Application with react-query where the nextPage will not be refetched but reused when user opens it.
Otherwise, if the nextPage is not reused, it's worse than checking for a total number of rows to determine whether there are any rows left as you will make 2 requests for every page.
It will even make the user interface snappier as the transition to the next page will always be instant.
This method will work well if you have a lot of page transitions as the total number of calls equals numberOfPages+1, so if on average users go to 10 pages, numberOfPages+1=10+1 or just 10% overhead. But if your users usually do not go beyond the first page, it makes little sense as in this case numberOfPages+1=2 calls for a single page.
I am trying to build a Facebook application by using Facebook Graph API. I am trying to get the home feed of the application user for a particular time span. e.g. From 11-02-2014 to 15-07-2014. I have used since and until in Facebook Graph Explorer but it returned news feed of two consecutive days.
2011-01-01 is the wrong synthax to represent a time on Facebook. You should use Unix timestamps: 1418971177.
A time-paginated edge supports the following parameters:
until : A Unix timestamp or strtotime data value that points to the end of the range of time-based data.
since : A Unix timestamp or strtotime data value that points to the start of the range of time-based data.
limit : This is the number of individual objects that are returned in each page. A limit of 0 will return no results. Some
edges have an upper maximum on the limit value, for performance
reasons. We will return the correct pagination links if that happens.
Source: https://developers.facebook.com/docs/graph-api/using-graph-api/v2.2#
So, instead of:
/me/home?since=2011-01-01&until=2011-05-05
You must use:
/me/home?limit=25&until=1418971177 with until.
Or:
/me/home?limit=25&since=1418971128 with since.
Note that you can't use since and until together. You have to use the limit field to restrict the amount of results you get.
I am trying to fetch links that were posted on a public Facebook page sometime ago, in 2011 for example. Specifically, the Arabic CNN page on facebook: http://www.facebook.com/CNNArabic
Things I tried:
1- Graph API, a query like this:
CNNArabic/links?fields=id,name,link,created_time&limit=25&until=2012-05-15
2- FQL
SELECT link_id, url, created_time FROM link WHERE owner = 102581028206 and created_time < 1337085958 LIMIT 100
Both give an empty data set while there is data on the page on or before this date.
Other things I noticed:
1-If I changed the date to something like 2013-01-17 (which is yesterday), it works fine.
2-If I changed the date to something like 2012-12-17 (about a month ago), an empty data set is returned, however if I followed the next page links in the returned data set from the query in number 1 above until I pass by this date I actually get data.
I tried writing code that kept following the next page pointers until I reach the links on the date I want. However I need data much older (say in 2011) and the result set gets exhausted say 2 or 3 months earlier than now, in other words no more next links are returned so I actually can never reach that old data.
To cut this short:
is there a way I can query links that were posted on a public page before a specified date?
Querying the page's feed works in the Graph API:
/102581028206/feed?fields=id,link,name,created_time&limit=25&until=2012-05-15
This returns all the posts. There should be a way to filter this using field expansion, but I couldn't get the few things I tried to work.
You can get this filtered for links only with FQL on the stream table:
SELECT message, attachment, created_time FROM stream WHERE source_id = 102581028206
AND created_time < strtotime('2012-05-15') AND type=80 LIMIT 10
Links are type=80.
I see that we can only get 500 members of a group using the graph API.
and the doc says these are "the first 500 members",
Are these sorted by date signed up, or latest 500?
Is there any way I can further limit these to signed up in the last 24 hours/ 1 week?
Is the 500 limit there in using FQL also? (the docs don't specify that )
Is there any way I can further limit these to signed up in the last 24 hours/ 1 week using FQL?
i see that we can only get 500 members of a group using the graph API. and the doc says these are "the first 500 members",
are these sorted by date signed up, or latest 500,???
I’d say by date of joining the group, because otherwise calling them the “first” 500 would make little sense.
Is the 500 limit there in using FQL also? (the docs dont specify that )
From my tests there seems to be no such limit on the group_member table (just tried it for the FB developer group using Grapf API explorer, and my browser froze for about a minute loading the data).
is there any way i can further limit these to signed up in the last 24 hours/ 1 week using FQL?
No, there is no such info as signup date in the FQL table.
Can we get the number of updates since a particular timestamps using graph APIS? Number of updates might be number of new New feed or Messages.
depending on which api you are calling
When searching for public posts or posts on the user's News Feed, you
can page over the results by using the since, until and limit
parameters. since and until both accept a unix timestamp. When paging
back in time, you should use until in conjunction with limit where
until is the unixtime value of the created_time field in the last
object returned by your previous query. When paging forward in time
you should set since to be the unixtime value of the created_time
field in the first object returned by your previous query. Please
note, you can only search about 1 to 2 weeks back in the News Feed.
/me/feed?since=2+hours+ago&until=now "using strtotime in php"