I'm new to Github API v3 (rest) & v4 (graphql).
I have created a PAT (personal access token) and using it to fetch data from github - organization private repositories.
While able to get other users data (event, commits, issues etc.) in my organization with V3 api (REST),
I can't get information when using V4 api (Graphql) - I get info only when querying my own user, for other users I get zeros.
The graphql query I use (token is set in the header and some user in my org):
query {
user (login: "<user>") {
contributionsCollection(from: "2020-01-01T00:00:00.000Z", to: "2020-06-25T23:59:59.999Z", organizationID: "<my_org_id>") {
totalCommitContributions
totalIssueContributions
totalPullRequestContributions
totalPullRequestReviewContributions
totalRepositoriesWithContributedCommits
totalRepositoriesWithContributedIssues
totalRepositoriesWithContributedPullRequestReviews
totalRepositoriesWithContributedPullRequests
}
}
}
and the zeroed response (when querying myself I get non-zero values):
{
"data": {
"user": {
"contributionsCollection": {
"totalCommitContributions": 0,
"totalIssueContributions": 0,
"totalPullRequestContributions": 0,
"totalPullRequestReviewContributions": 0,
"totalRepositoriesWithContributedCommits": 0,
"totalRepositoriesWithContributedIssues": 0,
"totalRepositoriesWithContributedPullRequestReviews": 0,
"totalRepositoriesWithContributedPullRequests": 0
}
}
}
}
While in REST api:
curl -H "Authorization: token <TOKEN>" -X GET 'https://api.github.com/users/<user>/events?page=1&per_page=10'
I get information (on the specified dates above)
What am I missing?
Start with this:
#!/bin/bash
query=" \
{ \
\"query\": \"query \
{ \
repository(owner: \\\"MyOrg\\\", name: \\\"some-repo-name\\\") {\
issues(last: 50, states: OPEN, labels: \\\"some label\\\") {\
edges {\
node {\
title\
body\
url\
createdAt\
lastEditedAt\
}\
}\
}\
}\
} \" \
} \
"
curl -H "Authorization: bearer XX-your-PAT-goes-here-XX" \
-X POST \
-d "${query%%$'\n*'}" \
https://api.github.com/graphql |
jq --compact-output '.[] | .repository.issues.edges[] | .node' |
while read -r object; do
echo "================================"
# determine last edit date and time
lastEditedAt=$(prettify_date $(echo "$object" | jq '.lastEditedAt'))
createdAt=$(prettify_date $(echo "$object" | jq '.createdAt'))
echo "${createdAt} -- ${lastEditedAt}"
[[ ${lastEditedAt} == nul ]] &&
reallyLastEditedAt="${createdAt}" ||
reallyLastEditedAt="${lastEditedAt}"
title=$(echo "$object" | jq '.title')
content="$(echo "$object" | jq '.body' | sed -e 's/^"//' -e 's/"$//')"
echo ${content} | sed 's!\\r\\n!\n!g'
done
Related
I'm trying to get Insight of an Ad-Account by filtering using multiple specific campaigns, I was able to filter with single specific campaigns
Here is the code which I tried for single specific campaigns
https://graph.facebook.com/v12.0/act_YOUR_ACCOUNT_ID/insights?fields=actions,reach,impressions,clicks,cpc,spend&filtering=[{field: "campaign.id",operator:"CONTAIN", value: '123456789'}}]
You have 2 options:
You can build a list of campaigns by your specific condition and then you use IN operator instead of CONTAIN operator like this:
https://graph.facebook.com/v12.0/act_YOUR_ACCOUNT_ID/insights?fields=actions,reach,impressions,clicks,cpc,spend&filtering=[{field: "campaign.id",operator:"IN", value: ['id1', 'id2', 'id3']}]
You can try to use batch request like this (documentation here and here):
curl \
-F 'access_token=<ACCESS_TOKEN>' \
-F 'batch=[ \
{ \
"method": "GET", \
"relative_url": "v12.0/act_YOUR_ACCOUNT_ID/insights?fields=impressions,spend,ad_id,adset_id&filtering=[{field: "campaign.id",operator:"CONTAIN", value: '123456789'}]" \
}, \
{ \
"method": "GET", \
"relative_url": "v12.0/act_YOUR_ACCOUNT_ID/insights?fields=impressions,spend,ad_id,adset_id&filtering=[{field: "campaign.id",operator:"CONTAIN", value: '222222222'}]" \
}, \
]' \
'https://graph.facebook.com'
I want to query a REST API with HTTPie. I am usuale to do so with curl, with which I am able to specify maxKeys and startAfterFilename e.g.
curl --location --request GET -G \
"https://some.thing.some.where/data/v1/datasets/mydataset/versions/2/files" \
-d maxKeys=100 \
-d startAfterFilename=YYYMMDD_HHMMSS.file \
--header "Authorization: verylongtoken"
How can I use those -d options in HTTPie?
In your case the command looks like this:
http -F https://some.thing.some.where/data/v1/datasets/mydataset/versions/2/files \
Authorization:verylongtoken \
startAfterFilename=="YYYMMDD_HHMMSS.file" \
maxKeys=="100"
Although, there is a bunch of methods to pass some data with httpie. For example
http POST http://example.com/posts/3 \
Origin:example.com \ # : HTTP headers
name="John Doe" \ # = string
q=="search" \ # == URL parameters (?q=search)
age:=29 \ # := for non-strings
list:='[1,3,4]' \ # := json
file#file.bin \ # # attach file
token=#token.txt \ # =# read from file (text)
user:=#user.json # :=# read from file (json)
Or, in the case of forms
http --form POST example.com \
name="John Smith" \
cv=#document.txt
I have a json file like below
[
{
"field": {
"empID": "sapid",
"location": "India",
}
},
{
"field": {
"empID": "sapid",
"location": "India",
}
},
{
"field": {
"empID": "sapid",
"location": "India",
}
}
{
"field": {
"empID": "sapid",
"location": "India",
}
},
{
"field": {
"empID": "sapid",
"location": "India",
}
}
{
"field": {
"empID": "sapid",
"location": "India",
}
}
.... upto 1 million
]
I have to use this json as an input for a rest request For example
curl <REST Server URL with temp.json as input> "Content-Type: application/json" -d #temp.json
My server will not accept 1 million json object at a time.
I am looking for an approach where i have to extract the first 500 objects from the main json and send it in one rest query and then next 500 object in second rest query and so on.
Can you please suggest how can i achieve this by jq?
There's an intrinsic tradeoff here between space and time efficiency. In the following, the focus is on the latter.
Assuming that each call to curl must send a JSON array, a time-efficient solution can be constructed along the following lines:
< array.json jq -c '
def batch($n): length as $l | range(0;length;$n) as $i | .[$i: $i+n];
batch(500)
' | while read -r json
do
echo "$json" | curl -X POST -H "Content-Type: application/json" -d -# ....
done
Here .... signifies additional appropriate curl arguments.
GNU parallel
You might also want to consider using GNU parallel, e.g.:
< array.json jq -c '
def batch($n):
length as $l
| range(0;length;$n) as $i
| .[$i: $i+n];
batch(500)
' | parallel --pipe -N1 curl -X POST -H "Content-Type: application/json" -d #- ....
You have not shared any HW information of the system you are running this on. At the minimum you need to do some sort of multiprocessing to make this faster instead of running (1000000/500) curl requests altogether.
One way, would be to use GNU xargs which has a built-in to run number of parallel instances of a given process using the -P flag and number of lines of input to read from at any time with the L flag.
To start with you can do something like below to instruct curl to run on 500 lines at a time and invoke 20 such invocations in parallel. So at a given tick, approximately (500 *20) lines of input are processed. You can tune the numbers depending on your HW capability both on the host and the server side.
xargs -L 500 -P 20 curl -X POST -H "Content-Type: application/json" http://sample-url -d #- < <(jq -c 'range(0;length;500) as $i | .[$i: $i+500]' json)
Modified jq filter to pack the JSON payload as an array of objects (credit peak's answer). The earlier version jq -c '.[]' json might not work as the individual chunk of lines passed at a time doesn't represent a valid JSON.
Note: Not tested due to performance constraints.
Assuming you have this formatting, splitting can be done by unpacking the array and saving the desired number of objects to separate files, e.g.:
<input.json jq -c '.[]' | split -l500
Which creates xaa with the first 500 objects, xab with the next 500 objects, etc. If you want to repackage the objects in an array, use the -s option to jq, e.g.: jq -s . xaa.
If you want to do this from the shell, you could use jq to split your JSON and pass it to xargs to call curl for each object returned.
jq -c '.[]' temp.json | xargs -I {} curl <REST Server URL with temp.json as input> "Content-Type: application/json" -d '{}'
This will send one curl request for each object. However, if you e.g. want to send the first 500 objects in a single curl request, you can specify a subarray in the jq filter. To send all of your JSON objects you will then somehow need to repeat the command, as afaik jq has no built-in way to split the input into chunks of objects.
jq -c '.[0:500]' temp.json | xargs -I {} curl <REST Server URL with temp.json as input> "Content-Type: application/json" -d '{}'
jq -c '.[500:1000]' temp.json | xargs -I {} curl <REST Server URL with temp.json as input> "Content-Type: application/json" -d '{}'
jq -c '.[1000:1500]' temp.json | xargs -I {} curl <REST Server URL with temp.json as input> "Content-Type: application/json" -d '{}'
[...]
I'd like to see the 'config' details as shown by the command of:
kubectl config view
However this shows the entire config details of all contexts, how can I filter it (or perhaps there is another command), to view the config details of the CURRENT context?
kubectl config view --minify displays only the current context
use the below command to get the full config including certificates
kubectl config view --minify --flatten
The cloud-native way to do this is to use the JSON output of the command, then filter it with jq:
kubectl config view -o json | jq '. as $o
| ."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name) as $context
| $o.clusters[] | select(.name == $context.context.cluster) as $cluster
| $o.users[] | select(.name == $context.context.user) as $user
| {"current-context-name": $current_context_name, context: $context, cluster: $cluster, user: $user}'
{
"current-context-name": "docker-for-desktop",
"context": {
"name": "docker-for-desktop",
"context": {
"cluster": "docker-for-desktop-cluster",
"user": "docker-for-desktop"
}
},
"cluster": {
"name": "docker-for-desktop-cluster",
"cluster": {
"server": "https://localhost:6443",
"insecure-skip-tls-verify": true
}
},
"user": {
"name": "docker-for-desktop",
"user": {
"client-certificate-data": "REDACTED",
"client-key-data": "REDACTED"
}
}
}
This answer helped me figure out some of the jq bits.
The bash/kubectl with a little bit of jq, for any context equivalent:
exec >/tmp/output &&
CONTEXT_NAME=kubernetes-admin#kubernetes \
CONTEXT_CLUSTER=$(kubectl config view -o=jsonpath="{.contexts[?(#.name==\"${CONTEXT_NAME}\")].context.cluster}") \
CONTEXT_USER=$(kubectl config view -o=jsonpath="{.contexts[?(#.name==\"${CONTEXT_NAME}\")].context.user}") && \
echo "[" && \
kubectl config view -o=json | jq -j --arg CONTEXT_NAME "$CONTEXT_NAME" '.contexts[] | select(.name==$CONTEXT_NAME)' && \
echo "," && \
kubectl config view -o=json | jq -j --arg CONTEXT_CLUSTER "$CONTEXT_CLUSTER" '.clusters[] | select(.name==$CONTEXT_CLUSTER)' && \
echo "," && \
kubectl config view -o=json | jq -j --arg CONTEXT_USER "$CONTEXT_USER" '.users[] | select(.name==$CONTEXT_USER)' && \
echo -e "\n]\n" && \
exec >/dev/tty && \
cat /tmp/output | jq && \
rm -rf /tmp/output
You can use the command kubectl config view --minify to get current context only.
It is handy to use --help to get the options what you could have for kubectl operations.
kubectl config view --help
Thanks to this tutorial: https://www.twilio.com/docs/sip-trunking/api/trunks#action-create I am able to CRUD create, read, update and delete trunks on my Twilio account.
To create a new trunk I do it like so:
curl -XPOST https://trunking.twilio.com/v1/Trunks \
-d "FriendlyName=MyTrunk" \
-u '{twilio account sid}:{twilio auth token}'
and this is the response I get when creating a new trunk:
{
"trunks": [
{
"sid": "TKfa1e5a85f63bfc475c2c753c0f289932",
"account_sid": "ACxxx",
....
....
"date_updated": "2015-09-02T23:23:11Z",
"url": "https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932",
"links": {
"origination_urls": "https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932/OriginationUrls",
"credential_lists": "https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932/CredentialLists",
"ip_access_control_lists": "https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932/IpAccessControlLists",
"phone_numbers": "https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932/PhoneNumbers"
}
}],
"meta": {
"page": 0,
"page_size": 50,
... more
}
}
What I am interested from the response is:
"links": {
"origination_urls": "https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932/OriginationUrls",
Now if I perform a get command on that link like:
curl -G "https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932/OriginationUrls" -u '{twilio account sid}:{twilio auth token}'
I get back this:
{
"meta":
{
"page": 0,
"page_size": 50,
"first_page_url":
....
},
"origination_urls": []
}
Now my goal is to update the origination_urls. So using the same approach I used to update a trunk I have tried:
curl -XPOST https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932/OriginationUrls \
-d "origination_urls=sip:200#somedomain.com" \
-u '{twilio account sid}:{twilio auth token}'
But that fails. I have also tried:
curl -XPOST https://trunking.twilio.com/v1/Trunks/TKfa1e5a85f63bfc475c2c753c0f289932/OriginationUrls \
-d "origination_urls=['someUrl']" \
-u '{twilio account sid}:{twilio auth token}'
and that fails too. How can I update the origination_urls?
I was missing to add Priority, FriendlyName, SipUrl, Weight and Enabled on my post request. I finally got it to work by doing:
curl -XPOST "https://trunking.twilio.com/v1/Trunks/TKfae10...../OriginationUrls" -d "Priority=10" -d "FriendlyName=Org1" -d "Enabled=true" -d "SipUrl=sip:test#domain.com" -d "Weight=10" -u '{twilio account sid}:{twilio auth token}'