Retrieving Facebook reports in R - facebook

Note: this is an updated post
In Facebook I have scheduled report reports to be sent to my e-mail.
Unfortunately reports are not coming as attachment but rather as a link e.g. below
https://www.facebook.com/ads/manage/download_report.php?act=1119884838046520&report_run_id=6086936413420&format=csv&source=email_v2
Please help me to construct the URI path to retrieve the report.
Current state
I am able to get properties of the report:
RCurl::getURL("https://graph.facebook.com/v2.9/6087139848020?access_token=EAAX****")
results into
$ id : chr "6087139848020"
$ account_id : chr "1119884838046520"
$ schedule_id : chr "6085730147820"
$ time_ref : num 1.49e+09
$ time_completed : num 1.49e+09
$ async_status : chr "Job Completed"
$ async_percent_completion: num 100
$ friendly_name : chr "*****"
$ date_start : chr "2017-05-03"
$ date_stop : chr "2017-05-03"
So everything seems OK
Problem
However by trying to access the /report_ID/insights as suggested FB API reference/ad-report-run/:
RCurl::getURL("https://graph.facebook.com/v2.9/6087139848020/insights?access_token=E***")
I get an error message:
(#100) Tried accessing nonexisting summary field (results)
Any ideas how to get the body of Facebook report distributed by e-mail?
Help will be strongly appreciated.
PS Special thanks to user #phwd for suggestions

The solution is
curl -G \
-d 'report_run_id=6087450841820' \
-d 'format=csv' \
'https://www.facebook.com/ads/ads_insights/export_report/'
Source of info: FB docs for asynchronous reports

Related

GRIB1 to GRIB2 eccodes conversion failing due to paramId = 0

So what I am trying is converting a GRIB1 file to GRIB2 containing mostly wind data.
Usually you can just change the edition to 2 and the eccodes lib does everything else.
Now the issue is that my GRIB1 file has only paramId=0 and shortName=unknown messages.
What the hell am I supposed to do? When I load it into a viewer (e.g. PredictWind Offshore) it displays just fine. Any ides on how I can convert this to GRIB2 without knowing the messages? Am I missing something?
What I already tried:
❯ grib_set -s edition=2 e1.grib e2.grib
ECCODES ERROR : concept: no match for paramId=0
ECCODES ERROR : Please check the Parameter Database 'https://apps.ecmwf.int/codes/grib/param-db/?id=0'
ECCODES ERROR : concept: input handle edition=2
ECCODES ERROR : grib_set_values[0] edition (type=long) failed: Concept no match
What my GRIB1 (e1.grib) looks like:
❯ grib_dump e1.grib | egrep 'paramId|shortName'
...
shortName = unknown;
paramId = 0;
...
❯ grib_dump dsv1.grib | egrep 'paramId' | wc -l
679 # all unknown and 0
Edit:
For future readers:
I ended up not using eccodes at all. My solution now involves using PyNIO which worked great for me. It is able to read GRIB1.
For further info contact me directly.

How can i get the Passing/Failing status of a Github Action Workflow?

I have been looking at the GitHub REST API and i have been trying to find out where I can find the endpoint to get the status of a workflow in my actions. The only way i can tell if it is passing or failing is by downloading the badge.svg.
You can use workflow run api :
GET https://api.github.com/repos/[owner]/[repo]/actions/workflows/[workflowID]/runs
[workflowID] can also be the filename, in the following example ci.yml :
https://api.github.com/repos/bertrandmartel/tableau-scraping/actions/workflows/ci.yml/runs
Then you can get the first run using curl and jq :
curl -s "https://api.github.com/repos/bertrandmartel/tableau-scraping/actions/workflows/ci.yml/runs" | \
jq -r '.workflow_runs[0].status'
output:
completed

jfrog cli artifact search by filename pattern

I want to search for a filename pattern across entire JFrog ARM
without knowing the explicit repository name in the JFrog cli.
jfrog rt s "reponame/*pattern*"
is giving the results as expected in a specific repo.
But I have repo1, repo2, repo3, ... so on.
How do I search using wildcard for reponame, below is not working.
jfrog rt s "*/*pattern*"
Basically I want the jfrog cli equlivalent of the curl GET request search
"https://server/artifactory/api/search/artifact?name=*pattern*"
This is not for cli client, but an alternative way to get desired feature. Spent some time looking at API here:
https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API
I recommend to scroll down that page slowly and read in entirety as a lof of possible commands, syntax is excellent, I executed a few searches and they searched all local repositories. No need to recursively search 1 by 1. Command syntax:
export url="http://url/to/articatory"
curl --noproxy '*' -x GET "$url/api/search/artifact?name=log4j*"
Read link above for more granular search options/syntax.
How I set it up:
alias artpost='curl -X POST "http://url/artifactory/api/search/aql" -T - -u admin:password'
Some example usage:
echo 'items.find({"name": {"$match" : "log4j*"}})' | artpost
echo 'items.find({"$and" : [{"created" : {"$gt" : "2017-06-12"}},{"name": {"$nmatch" : "*surefire*"}}]})' | artpost

Google sites liberation: Bad Request Invalid request URI

I keep getting
Jul 24, 2014 11:53:15 AM com.google.sites.liberation.export.ContinuousContentFeed?$FeedIterator? catchException WARNING: Error retrieving response from query. com.google.gdata.util.InvalidEntryException?: Bad Request Invalid request URI
at com.google.gdata.client.http.HttpGDataRequest.handleErrorResponse(HttpGDataRequest.java:558) at com.google.gdata.client.http.GoogleGDataRequest.handleErrorResponse(GoogleGDataRequest.java:543) at com.google.gdata.client.http.HttpGDataRequest.checkResponse(HttpGDataRequest.java:536) at com.google.gdata.client.http.HttpGDataRequest.execute(HttpGDataRequest.java:515)
I used (you can use this test account i provide the password here below :) ).
java -jar google-sites-liberation-1.0.4.jar -w https://sites.google.com/site/hmmtestgsl/ -u hmmtestgsl#gmail.com -p aaabbbcccddd -f ./backup
and I changed the main in MANIFEST.MF to: Main-Class: com.google.sites.liberation.export.Main so that I can run it from command line.
Using
-w hmmtestgsl
instead of
-w https://sites.google.com/site/hmmtestgsl/
fixed it.

how to improve this perl/bash one-liner to deserialize json data

I have a little bash program that calls a webservice that returns JSON data.
I have written the webservice program myself, I have complete control over its data sources, and thus I can trust the data that is returned.
Now I want to do something with the data.
The data is a simple, short key-value structure without nesting, and looks like this:
{
"asciifile" : "../tmp/data_20120720_105746-01580.txt",
"excelfile" : "../tmp/data_01580-20120720_105746.xlsx",
"from" : "Jun 19, 2012",
"msg" : "some info message, for the admin",
"outfile" : "data--recent.txt",
"outfile_excel" : "data--recent.txt.xlsx",
"resolution" : "std"
"to" : "Jul 20, 2012",
"url_comment" : "another info message, for the screen/user",
"url_outfile" : "http://www.example.com/path/tmp_cached_files/data--recent.txt",
"url_outfile_excel" : "http://www.example.com/path/tmp_cached_files/data--recent.txt.xlsx",
}
Now I am using this one-liner to deserialize the json structure returned to perl code. See last line of this snippet:
#!/bin/bash
cmd=$(curl_or_wget_call_to_webservice)
output=$(eval $cmd)
outfile_excel=$(echo "$output"| json_xs -f json -t dumper | tee | perl -n0777 -E 'eval "%h=%{$_}"; warn $# if $#; say $h{outfile_excel}')
For example, I'm not sure why I came up with the %{$_} construct. Is there a better way to do this? Is there a shorter way or a safer way to write the last line?
SE Editors: if you wish, you may move this post to the codereview stackexchange site, but I don't have an account there.
Edit: After revisiting the post after 8 months, I'd like to add that these days I use this one liner for getting the name of my github repos:
wget --quiet --auth-no-challenge --user knbknb --password secret -O -
https://api.github.com/user/repos |
perl -MJSON -n0777 -E '$r = decode_json($_); map {say $_->{name}} #$r' -
Perl can decode JSON itself, so the next should give some idea, using LWP::Simple to get some json data.
perl -MLWP::Simple -MJSON \
-e '$ref = decode_json(get("http://your.url/to/webservice")); print $ref->{outfile_excel}'
The $ref contains a perl structure of all JSON data, print out as you want it..
There is jshon. You could simply call something like
curl http://somehere.tld/data.json | jshon -e url_outfile_excel
Which would print the value for the given key.
By the way. Having control over the webservice doesn't make the input trustworthy. Be careful when calling eval.