Facebook Graph IP filtering - facebook

I've read few posts around about the limitations on 600 requests in 600 seconds that Facebook Graph Api sets on requests.
This question is about getting some clarification in the issue I'm facing.
I'm doing requests, quite simple to the FB Graph:
So, from my home I run:
curl https://graph.facebook.com/v2.0/?id=https://www.example.com/article/the-name-of-the-article/
(Having the trail slash is not trivial)
which gives me empty results:
{
"share": {
"comment_count": 0,
"share_count": 605
},
"og_object": {
"id": "XXXXX6ZZ70301002",
"description": "text",
"title": "title",
"type": "article",
"updated_time": "2019-03-09T00:15:06+0000"
},
"id": "https://www.example.com/article/the-name-of-the-article"
}
I took the url from js code in the website.
Instead, running the Scrapy crawler, on the same url, still from home network, gives me the same as above:
{
"share": {
"comment_count": 0,
"share_count": 605
},
"og_object": {
"id": "XXXXX6ZZ70301002",
"description": "text",
"title": "title",
"type": "article",
"updated_time": "2019-03-09T00:15:06+0000"
},
"id": "https://www.example.com/article/the-name-of-the-article"
}
Which is more than fine for now and the js-code-scraping system seems to be working. The results contain all the information from js calls to FB Graph.
Hands on server side, the crawler runs as expected, but having a closer look at the results, information coming from js code execution is not there.
I've checked the whole code, in other url which also fires js actions to provide html content and the code actually works fine.
Then, repeating the simple:
curl https://graph.facebook.com/v2.0/?id=https://www.example.com/article/the-name-of-the-article
this time from the server ip, it replies:
{
"error": {
"message": "(#4) Application request limit reached",
"type": "OAuthException",
"is_transient": true,
"code": 4,
"fbtrace_id": "ErXXXXZZrOn"
}
}
Regarding ip-blocks, the code wasn't able of delivering more than 600 requests. Actually it sent less than 10 requests to the graph api.
Obviously, the information coming from js requests to the Fb Graph Api from server side is missing.
I tried from different servers, from different providers, to check if there was a Ip filter on Cloud providers, but it seems that is not the case, as in every server the results are the same.
What is going on here?
Why the js requests do not get valid response data when they are fired from server ip addresses? (as it gives the error OAuthException:Application request limit reached also using the curl command)
Thanks for any clue

Related

Can someone explain, what does the webhook do in api.ai?

I am new to api.ai. I want to send data to the web server and receive it and then give it to the users? From the documentation that I read, I understood that I have to use a webhook. But I am not sure how will api.ai send and receive the data?
Can the webhook be developed in any language?
The webhook is a web service that you implement in any language and on any platform, with an HTTP (must be https for ghome) and JSON interface, that fullfils (in their lingo) a user intent.
API.AI matches a user utterance to an intent (which then suggests entity values and a response) and they pass these in the call to your web service. You do whatever processing you need - your domain logic - and then return a speech response for the user and optionally some API.AI contexts.
You can read more about it (and about slot filling fulfillment which is a little different) here.
You can visualize the working of a webhook like a block where data request comes in JSON format somewhat like this:
{
"id": "7aef9329-4a32-4d59-b661-8bf380a0f35b",
"timestamp": "2017-06-07T05:36:12.641Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "hi",
"action": "order.pizza",
"actionIncomplete": true,
"parameters": {
"address": "",
"crust": "",
"sauce": "",
"size": "",
"time": "",
"topping": "",
"type": ""
}
}
}
and another json file is returned to it according to the prescribed settings.

Facebook API not working sometime for some URL

I'm getting number of shares and comment of blog posts using facebook graph api for my blog, but sometimes it do not return shares and comment.
Following URL is not giving number of share and comments.
http://graph.facebook.com/?id=http://www.lehedonist.com/jeans-is-the-new-whiskey-2001
BUT it is working for this URL:
Example: http://graph.facebook.com/?id=http://www.lehedonist.com/spring-slam-fashion-war-ss16-1965
I somehow wonder why it's even working to request these URLs without an access token to be honest.
See
https://developers.facebook.com/docs/graph-api/reference/v2.5/url
It only gives a result if there no version info in the URL. To do this in the v2.5 way properly, you'd have to use
https://graph.facebook.com/v2.5/?access_token={access_token}&fields=id,og_object,share&id=http://www.lehedonist.com/spring-slam-fashion-war-ss16-1965
where {access_token} is an actual access token.
Results will look like
{
"id": "http://www.lehedonist.com/spring-slam-fashion-war-ss16-1965",
"og_object": {
"id": "987675524635070",
"description": "Not your usual fashion post, we decode styles best suited to your personality and what should you be sporting next summer",
"title": "Spring Summer 16 fashion: The clash of fashion personalities",
"type": "article",
"updated_time": "2015-10-08T11:24:32+0000",
"url": "http://www.lehedonist.com/spring-slam-fashion-war-ss16-1965"
},
"share": {
"comment_count": 2,
"share_count": 81
}
}

How can I expose statistics about resources through a RESTful API?

I'm building an API for my web app and have got as far as exposing all the resources my app uses, e.g. /users, /roles, /posts etc with no problem.
I'm now stuck on how to expose statistics about some of these resources in a RESTful way. It doesn't seem right to have a statistics resource, as GET /statistics/1 could be anything, and the results will likely change each request, as the stats are real-time, so it will not be cacheable.
Background:
For each of the /users in the system, the app periodically queries Steam's API for the /games they are playing, and the /servers they are playing it on, and stores this information along with a timestamp in the /states resource.
This information is aggregated to show a tally of the most popular games and servers on the /statistics/games/current-usage and statistics/servers/current-usage standard HTML pages. Illustrative screenshots: servers, games (taken at different times).
EDIT: Sample data for the basic resources
"state": {
"id": 292002,
"user_id": 135,
"game_id": 24663,
"server_id": 135,
"created_at":"2014-06-22 21:12:03"
},
"user": {
"id": 112,
"username": "ilumos",
"steam_id_64": "76561197970613738"
},
"server": {
"id": 135,
"application_id": 24663,
"name": null,
"address": "192.168.241.65",
"port": "0"
},
"game": {
"id": 24663,
"name": "DEFCON",
"steam_app_id": 1520
}
EDIT 2: Does REST permit endpoints that use a timestamp as the resource identifier? e.g:
GET /statistics/1403681498/games to get a response like this:
[
"game": {
"id": 123,
"name": "DEFCON",
"users": [
{
"id": 7654,
"username": "daryl",
"server": {
"id": 127,
"ip": "123.123.123.123",
"port": "27960"
}
},
{
"id": 135,
"username": "ilumos"
},
]
}
]
You have a variety of not-wholly-unreasonable options.
You can
include statistics with each response. Will all clients want
statistics? Are there a lot of statistics? Maybe something like GET /games?orderBy=numPlayers-&offset=0&limit=10 would work if all you're tracking is number of players.
have a /statistics/{statisticId} endpoint. This is not inherently unRESTful.
have a /games/{gameId}/statistics endpoint.
have a /statistics/games/{gameId} endpoint.
Really, there's no way for us to tell you what the best way is to implement this because we don't have enough information.
I'm going to go with creating a usage resosuce as all of these statistics will be the usage of other resources, either "right now" or at a historic point in time.
My URIs will look like this:
GET /usage/{resource-name}/{resource-id}
GET /usage/games/ collection of games in use right now (with user totals)
GET /usage/servers/ collection of servers in use right now
GET /usage/games/?timestamp=1234567890 collection of games in use at {timestamp}
GET /usage/games/1 usage of game with id 1 right now
GET /usage/games/1?timestamp=1234567890 usage of game with id 1 at {timestamp}
GET /usage/games/?user_id=123 usage of game with id 1 filtered to show only user with id 123
And in future I can extend the resource to for example return usage for electricity usage
GET /usage/phases/ collection of phases in use right now (with power draw totals)
GET /usage/phases/1 usage of phase with id 1 right now
GET /usage/phases/?timestamp=1234567890 collection of phases in use at {timestsamp} (with power draw totals)
Unless there's something inhernatly un-RESTful about this it seems to be the most fitting way of exposing this info.

How to get total number of tracks in a playlist using soundcloud json API

I am trying to paginate the track list of a set/playlist obtained via soundcloud JSON API. Pagination technique described here works fine if I want to generate tracklist of x number of tracks for next page only. What I am trying to do is to make a numbered pagination with multiple page links. I didn't find any parameter such as "track_count" which returns the total number of tracks for "/user/tracks". So, can anyone give me any insights on making a numbered pagination for a playlist when getting data via soundcloud JSON API? thanks
/users endpoint has "track_count" property in the returned representation of user:
$ curl "http://api.soundcloud.com/users/3207.json?client_id=YOUR_CLIENT_ID"
{
"id": 3207,
"permalink": "jwagener",
"username": "Johannes Wagener",
"uri": "http://api.soundcloud.com/users/3207",
"permalink_url": "http://soundcloud.com/jwagener",
"avatar_url": "http://i1.sndcdn.com/avatars-000001552142-pbw8yd-large.jpg?142a848",
"country": "Germany",
"full_name": "Johannes Wagener",
"city": "Berlin",
"description": "<b>Hacker at SoundCloud</b>\r\n\r\nSome of my recent Hacks:\r\n\r\nsoundiverse.com \r\nbrowse recordings with the FiRe app by artwork\r\n\r\ntopbillin.com \r\nfind people to follow on SoundCloud\r\n\r\nchatter.fm \r\nget your account hooked up with a voicebox\r\n\r\nrecbutton.com \r\nrecord straight to your soundcloud account",
"discogs_name": null,
"myspace_name": null,
"website": "http://johannes.wagener.cc",
"website_title": "johannes.wagener.cc",
"online": true,
"track_count": 12,
"playlist_count": 1,
"followers_count": 417,
"followings_count": 174,
"public_favorites_count": 26
}
This question is already old, but I hope this could help other people.
You could use either of this endpoint:
http://api.soundcloud.com/playlists/{playlist_id}?client_id={client_id}
https://api.soundcloud.com/playlists/{playlist_id}?oauth_token={oauth_token}
The 2nd API is undocumented, the first endpoint suddenly started to return 401 for no reason which is from the SoundCloud API documentation. I still provided the 1st endpoint as it's only not working at work, but behaves correctly at home IP address. I suggest if you'd use the 2nd API to generate and use a non-expiring token.
SoundCloud is not a reliable provider anymore as there's no app support for developers. You just have to figure things out by yourself.
On the JSON response, look for track_count. That'll give you the number of tracks in a playlist.

How to construct a permalink for objects returned by Facebook's new Graph API?

Facebook Graph API allows getting Facebook objects like posts, videos etc in JSON from. Here is an example of a post: (taken from the documentation):
{
"id": "719323658_129694190395214",
"from": {
"name": "Rabia Yalcinkaya",
"id": "719323658"
},
"message": "COK MUHTESEM!!!",
"picture": "http://external.ak.fbcdn.net/safe_image.php?d=41762b7121ee68754ebea48dee4568aa&w=130&h=130&url=http%3A%2F%2Fats.vimeo.com%2F192%2F882%2F19288238_200.jpg",
"link": "http://vimeo.com/5646785",
"source": "http://vimeo.com/moogaloop.swf?clip_id=5646785",
"name": "Watermelon carving",
"caption": "vimeo.com",
"description": "Me carving a watermelon.",
"icon": "http://static.ak.fbcdn.net/rsrc.php/z9XZ8/hash/976ulj6z.gif",
"type": "video",
"created_time": "2010-06-23T10:58:17+0000",
"updated_time": "2010-06-23T10:58:17+0000",
"likes": 1
},
How does one construct a url which allows you to link back to the original video (or other types) on Bret's wall on Facebook? something in the form of
http://www.facebook.com/<USER_ID>/video/<VIDEO_ID>
Thanks,
Boaz
in the actions field there is the link field (for each available action) which happens to contain the permalink to the given post. However be sure to inspect this before relying on it - it might change. I've only used it with "comment" and "like".
Apart from that you'd have to construct it manually. For users that have their username set (i.e. their profile is accessible via facebook.com/username, it works with:
http://facebook.com/USERID/posts/POSTID
For everything else:
http://www.facebook.com/permalink.php?id=USERID&v=wall&story_fbid=POSTID
Source: http://forum.developers.facebook.net/viewtopic.php?pid=316384
Note that the POSTID is in the form USERID_POSTID, so you'd have to strip the first part.