For example, I want to parse all tweets of Microsoft .
Twitter has API - GET statuses/user_timeline. But how I can see, This method can only return up to 3,200 of a user’s most recent Tweets.
So, can I parse all tweets of some screen_name?
You might be a bit better with GET search/tweets, adding the query q=#Microsoft.
However, you will have problems as well:
You'll get all tweets mentioning #Microsoft, not only the ones at the user_timeline. You will have to filter afterwards
Although there is not limit in theory, as the one of 3200 of GET statuses/user_timeline, you probably wont be able to get all tweets from a user. By definition, Twitter API does not provide that kind of service. If you want all tweets you'll need to use a service like topsy (not free)
You'll have to use pagination since every query to GET search/tweets gives you a maximum of 100 tweets, and if you need to get more than 450*100 tweets (remember you'll get a mixture of tweets as pointed out in 1. above), you'll have to handle Twitter rate limits and launch your queries in windows of 15 minutes
Sorry there is not a simpler answer to your question... Hope it helps anyway.
(EDIT: 450*100 is assuming you use application-only authentication. If not, it is 180*100)
Related
I want to do a query like this:
search?q=KEY_NAME&type=page&fields=id,name,location&limit=500&offset=0
when I do this the first time the result is about 470 results, now I put offset to 471 and repeat the query
search?q=KEY_NAME&type=page&fields=id,name,location&limit=500&offset=471
and the result is empty.
Why? The key_name is a famous word like "fan" and I don't think that there are only 471 results on fb pages!
What is the problem?
Never use a limit that high, afaik a limit of 100 should be the maximum. Everything else may be buggy. If you use this API call, you get more than 500 with paging:
/search?pretty=0&fields=idmname,location&q=fan&type=page&limit=100
Don´t use "offset", always use the "next" link in the JSON document to get the next batch of results: https://developers.facebook.com/docs/graph-api/using-graph-api/v2.4#paging
The next 100 entries would be available with the following endpoint for me:
/search?pretty=0&fields=idmname,location&q=fan&type=page&limit=100&after=OTkZD
Please refer to this following blog post in which it says
https://developers.facebook.com/blog/post/478/
gist of it.
As such, when querying the following tables and connections, use time-based paging instead of “offset” to ensure you are getting back as many results as possible with each call. For these Graph API connections, use the “since” and “until” parameters
I'm looking to implement an API call where you can specify any combination of up to ~6000 ids to get data from the server. Trouble is it's quite likely that a request will contain a large number of id's - say around 4000. The query string would therefore be very long and possibly too long for the browser?
I wonder, what would be the best approach? I could use a POST but it doesn't really fit with REST - but then again I'm not too fussed about that. Is there a better way of doing this?
In this case, POST really is the solution. From a REST perspective and also from an optimization perspective, if you expect this call to be invoked multiple times with the same list of IDs, you may want to consider one POST call to create a server-side named/defined list and then for subsequent GET requests to reference the created list so that this data doesn't have to be repeated each and every time.
I have a logic problem I can't seem to solve (might be possible).
Example:
I am inside 100 facebook groups
I need the 10 lastest posts of EACH group I am in.
That's pretty much it but I can't seem to find a way to do this without making a foreach loop calling the api over and over again, if I had a couple hundred more groups it would be impossible.
PS: I'm using FQL atm but am able to use graph, I've coded this in like 3 different ways but no success.
This is the farthest I could get:
SELECT actor_id,source_id FROM stream WHERE source_id IN (select gid from group_member where uid = me())
It only returns from one page, maybe there's no way to return all of this without a foreach asking for each groups 10 lastest messages.
There's no need to use FQL of batching. This can be done with a simple Graph API request IMHO:
GET /me/groups?fields=id,name,feed.fields(id,message).limit(10)
This will return 10 posts for each of your groups. In case there too much data to be returned, try setting the limit parameter for the base query as well:
GET /me/groups?fields=id,name,feed.fields(id,message).limit(10)&limit=20
Then, you'll get a next field in the result JSON. By calling the URL contained in this field, you'll get your next results. Do this until the result is empty, then you reached the end.
You can use batch calls, described here https://developers.facebook.com/docs/graph-api/making-multiple-requests/
Using batch requests, you can request upto 50 calls in one go. Note than batch request doesn't increase the rate limits, so if you make 50 requests in batch, it will be considered as 50 calls, and not one. However you will get the response in a shorter time.
If you think you're making too many calls, you should put some delay in between calls and avoid rate limiting.
Explanation:
I am able to query the Google Core reporting APIv3 using the client library to get data on pageviews for specific URLs of a website I am working on. I want to get data(pageviews) for each day within a specified range. So far I am simply looping through the range, sending individual request to the API. in each request I am setting the same value for the start date and the end date.
Problem:
Obviously this gets the job done, BUT it is certainly not the best way to go about it. Because, assumming I want to get data for the past 3 months for each of about 2000 URIs. Then I will need 360000 number of requests and that value is well over the limit quota defined by Google.
Potential solution: So one way I thought of solving this issue is probably to send a request setting start-date and end-date to be a week apart but the API will return a sum of the values rather than the individual values.
main question: So is there a way to insist that these values should not be added up and returned as a sum but rather returned (as associative array or something like that) separately for each.
I hope the question is clear and that there is a solution! Thank you!
Very straightforward:
Metric: ga:pageview, Dimension: ga:date, Set a filter for your pagepath, and set a start-date and end-date.
Example:
https://www.googleapis.com/analytics/v3/data/ga?ids=ga%3Axxyyzz&dimensions=ga%3Adate&metrics=ga%3Apageviews&filters=ga%3Apagepath%3D%3D%2Ffaq.html&start-date=2013-06-27&end-date=2013-07-11&max-results=50
This will return the pageviews for that the faq.html& page for each day in the time-frame.
You should check out the QueryExplorer. Great tool to find out how to structure queries.
I am performing a rest call to facebooks search API using type=event
e.x.
search?fields=id,name,picture,owner,description,start_time,end_time,location,venue,updated_time,ticket_uri&q=concert&type=event
I have looked through the documentation and still have a few questions about specific pagination behavior of the event search API.
If I used a broad search term like "ma" and keep querying the pagination ['next'] URL would I cycle through all facebook events starting with "ma"? Does the pagination array give any indication when there are no more results to return?.
Do these searches include past events? If so is it possible to eliminate past events using the "since" parameter?
What is the maximum for the limit parameter?
Update:
As far as I can tell the number of pages you can get from a facebook search is limited to 500. This includes pages that can be accessed via pagination. In other words a query with limit >=500 will not return a pagination url, likewise a query with limit 250 will only return one pages worth of pagination.
You will "next page" until the count of results comes less then the limit
I'm not sure if that is possible using a simple Graph Request. Maybe using FQL
I don't know exactly. But i used a 2000 limit one day. And it worked.
Other doubts you can get answers testing your resquests with this tool
https://developers.facebook.com/tools/explorer/
I am also doing the same thing like you. I am collecting public post using graph search api.
When there are no results available or you reach max limit pagination section will not be there in response. So you can always check for paging is there in json response or not something like this.
NextResult = DeserJsonFBResponce.paging != null ? DeserJsonFBResponce.paging.next : string.Empty;
I am not so sure about this with events but for public post i am able to eliminate post using science and until parameters.
Maximum for the limit parameter is 2000 per get request.