My final aim is to get the like count for a FaceBook PAGE. In short, I was doing
curl "https://fr-fr.facebook.com/PAGE/" | heavy-sed
Then, I found this page is much lighter, and saves bandwidth:
"https://fr-fr.facebook.com/PAGE/about"
Because of some bug, I have gone a bit further, and tried the more advanced method:
"https://graph.facebook.com/PAGE?access_token=secret|id&fields=fan_count"
All three methods end up in the same issue: the value returned by curl|sed always remain the same for a few hours (3 to 6h), even when the value is updated on classic browsers like Chrome or Firefox.
Many projects claim to update every 10s without issue. FB API states I should be allowed to do 200 requests per hour. Adding to curl -A with the same UA as my firefox does not help.
I do not personally have any Facebook account; so, even when I am using Firefox or Chrome, I am viewing a public page with anonymous access. Facebook should not be able to make any difference between curl and firefox when I provide the same UA.
I really need fresh values; 60 calls per hour is far below the official limit; one call every minute is enough for my project; but a 3h latency is not.
Curl is using IPv4 access, while Firefox is using IPv6; so, if Facebook is using a reverse proxy per IP, I would not be able to proove/detect it.
I do not understand why Firefox/Chrome can obtain a fresh and up to date value, while curl -A obtains a deprecated result.
Related
I'm using Facebook Ad Library API but I can't figure out how rate limit works.
On the application dashboard under "Application-Level Rate Limiting" the chart is always showing zero calls and when I make a new API call the header just shows this (the 'call_count' field is missing):
{
'total_time' : 0,
'total_cputime' : 0
}
I've read on a report by Mozilla developers that there is a rate limit (they were blocked a few times), someone can help?
Thanks
I was running into a similar issue and worked out the following solution:
You can see the current usage of your rate limit for the Ad Library API by looking at the "object_count_pct" in the header. API calls are blocked once it reaches a value of 100 (%).
Avoiding the limit is then possible by pausing calls at a certain percentage until the object_count_pct goes down again. You can also build your API calls so that they use the "estimated_time_to_regain_access" value to restart the calls after Facebook unblocks your app.
Unfortunately I could not find any documentation regarding the actual rate limit or at what speed the object_count_pct goes down once calls to the API are halted. The 200 calls per hour as stated in the official docs is definitively not accurate as I have managed to make over 5.000 calls in ca. 8 hours.
In golang, is there a way to pipe a variable to part of a web form?
For example, sending "123 Random St." to the Street address part of https://www.dominos.com/en/pages/order/#/locations/search/ and so on? I found pizza_party*, but the GUI used is no longer available, I have also found pizzadash**, but this uses a credit card where I want to use cash. I even found a list of golang ones, but the links that they use doesn't work anymore.***
Therefore, my goal is so: order a pizza in golang through the dominos website API!
NOTE: Please suggest a package or function with example!
NOTE: I do not want to make a web scraper/data getter.
NOTE: Your answer must work on at least one box of my linked website.
NOTE: I want to fill out links similar to the provided link from the linux command line.
*https://github.com/coryarcangel/Pizza-Party-0.1.b
**https://github.com/bhberson/pizzadash
***https://golanglibs.com/top?q=pizza
This is how you post any form values onto an online form. Provided you know the POST endpoint of the service.
func main():
resp, err := http.PostForm(targetPostUrlHere,
url.Values{"Service_Type": {"Delivery"},
"Address_Type_Select": {"House"},
"Street": {"123 E 24th St"},
"Address_Line_2": {"4D"},
"City": {"New York"},
"Region": {"NY"},
"Postal_Code": {"10027"}})
}
**Note: The field keys and values are guesstimates. You must inspect the actual key names expected in the form.
In your case, https://www.dominos.com/en/pages/order/ is an endpoint for the form page. Once the form is filled and submitted, the information is submitted using POST method akin to the code afore-mentioned to a dedicated CREATE endpoint (C in the CRUD), which normally can be found in the <form> html tag.
<form action="posttargetendpoint" method="POST">...</form>
Once the POST operation is successful, usually a web service would redirect you to another page. In your case, it is https://www.dominos.com/en/pages/order/#/section/Food/category/AllEntrees/
However, any good web service wouldn't expose the POST endpoint in the clear since it is the vulnerable point of attack. You're welcome to find out by inspect he Domino's page source and adjust the field values in the Go code accordingly.
Now to make a command line prompt to wrap around the PostForm code, I suggest you look into https://github.com/codegangsta/cli which is a very nice package for creating quick command line app.
I assume you mean pipe information originating from your backend to another site on behalf of a user?
The standard way of passing information between domains is via HTTP params, usually via a GET request, but this capability would need to be supported by established protocols the remote site. You can also use an iframe to embed the page of another site onto your page, however, you wouldn't be able to remotely interact, call JS code, or even query the page at all. Cross-domain security safeguards justifiably prohibit such capability, and generally speaking, interacting on behalf of the user via their browser is also restricted for security reasons.
However, if you're looking to emulate user behavior such as with a bot or web scraper from your own host or browser then that's a different story. There are tons of frameworks provide rich capability for interacting with a page. I'd recommend checking out Selenium, which acts as a virtual browser. There are also tons of libraries in Python for processing data from HTML and structured data. You might want to check out Beatiful Soup and Scrapy.
Hope this helps.
I'm after the impression uniques for our posts and ideally we would like one server call and it would bring back the last 100 posts with the reach (post_impressions_unique).
To get reach, I can currently do this using:
https://graph.facebook.com/nmemagazine/posts?fields=name,message,full_picture,created_time,shares,likes.limit(1).summary(true),comments.limit(1).summary(true)&access_token=USER_TOKEN
and getting the post ID, then calling: https://graph.facebook.com/v2.5/POST_ID/insights/post_impressions_unique?access_token=USER_TOKEN
Is there a way where I can make one call to get all recent posts and their reach in one API call?
You could simply call
https://graph.facebook.com/nmemagazine/posts?fields=name,message,full_picture,created_time,shares,likes.limit(1).summary(true),comments.limit(1).summary(true),insights&access_token=USER_TOKEN
NOTE: extra field "insights".
However, as far as I can tell you cannot limit the insights to just Post_impressions_Unique so it would bring back some data you do not necessarily want/need. I.e. I don't think you can perform a search/filter on a "second level"/child/nested type.
To give an update on this:
In facebook Graph API 2.10, you can (and have to) specify the metrics you are interested in via insights.metric(...). Possible metrics are documented here:
https://developers.facebook.com/docs/graph-api/reference/v2.10/insights#availmetrics
For your example:
curl -i -X GET "https://graph.facebook.com/nmemagazine/posts?fields=name,message,full_picture,created_time,shares,likes.limit(1).summary(true),comments.limit(1).summary(true),insights.metric(post_impressions_unique)&access_token=USER_TOKEN"
Suddenly simple Facebook Graph API calls started to return an empty result. The node I'm calling is a simple search page by name:
`https://graph.facebook.com/search?type=page&fields=profile_picture,username,id,name,likes,category,talking_about_count&access_token=<APP_TOKEN>&limit=10&q=stackoverflow`
It just started to return an empty result set when called from my server:
{"data":[]}
When I'm calling it from my personal computer browser, it works just fine.
I don't see any notification in Facebook Manage Apps page. What can be the reason? Is there any way I can see any error in this Graph API result?
UPDATE: suddenly it started working again, after it was down for at least 2 hours. Is there any way I can make these calls through client side JS? Any way to avoid to exposure of my APP_TOKEN?
UPDATE2: It stopped working again, and although I haven't identified the reason, I did figure when replacing the with a , it does work. The documentation actually says to use an APP_TOKEN.
I understand that it's possible to use oembed to get embed code for private files, but I'm running into "403 forbidden" when I try it.
http://soundcloud.com/albert-albala-1/audio-recording-on-monday/s-VzgUg
is accessible in a browser, but
curl -iL http://soundcloud.com/oembed?url=http%3A//soundcloud.com/albert-albala-1/audio-recording-on-monday/s-VzgUg&format=json&auto_play=false
yields a HTTP/1.1 403 Forbidden error. The same call with a public sound works fine:
curl -iL http://soundcloud.com/oembed?url=http%3A//soundcloud.com/osmconcerts/rosemarie-landryv2&format=json&auto_play=false
Related question: Rendering SoundCloud widget for a private track using PHP API, although that one uses PHP. I'm trying to get this to work on the command line.
It's possible to embed private tracks using a secret token if the track's settings allow embedding.
Embedding is allowed by default, but if you're receiving a 403 with a valid secret token, you've probably disallowed public embedding for the track.
You can enable this through the web UI on your track's edit page under Settings -> Advanced -> Widget Settings -> Click to enable -> For everybody:
Just had this problem myself, and the track/sound settings have changed layout/structure since pje's answer from 30/09/2013.
Currently, under Permissions, you need to make sure that 'Display Embed code' is turned on - despite the fact that you may not want this to happen! Doing so should mean a 200 status code is returned along with the embed metadata.
Essentially what this is doing is allowing the track to be publicly embedded on other sites (despite the track being private in terms of not being listed on the main account page).