How to retrieve details from Activity Logs from rundeck - tableau-api

I am trying to create a dashboard for my Rundeck
I want to Have no of jobs executed, passed ,Failed, Terminated which is available in Activity log in Rundeck tool. But that cannot be downloaded in excel sheet..Here is the Dashboard Format , i am Looking
I want to download the data on basis of day, weekly and monthly and then use tableau to create a dashboard. let me know
or is their anyway i can integrate my rundeck and tableau directly and create a similar dashboard

You can get all those data from API, (for example getting execution info):
#!/bin/sh
# protocol
protocol="http"
# basic rundeck info
rdeck_host="localhost"
rdeck_port="4440"
rdeck_api="36"
rdeck_token="nMp0uRsNUAH0w17JXxoDPqoBq6V4mB2O"
# specific api call info
execution="2"
# get the job forecast
curl -s --location --request GET "$protocol://$rdeck_host:$rdeck_port/api/$rdeck_api/execution/$execution" \
--header "Accept: application/json" \
--header "X-Rundeck-Auth-Token: $rdeck_token" | jq
Output:
{
"id": 2,
"href": "http://localhost:4440/api/36/execution/2",
"permalink": "http://localhost:4440/project/ProjectEXAMPLE/execution/show/2",
"status": "succeeded",
"project": "ProjectEXAMPLE",
"executionType": "user",
"user": "admin",
"date-started": {
"unixtime": 1606853136420,
"date": "2020-12-01T20:05:36Z"
},
"date-ended": {
"unixtime": 1606853136878,
"date": "2020-12-01T20:05:36Z"
},
"job": {
"id": "900a885b-576c-4c8c-a118-4bd1bf775c7b",
"averageDuration": 458,
"name": "Test",
"group": "",
"project": "ProjectEXAMPLE",
"description": "",
"href": "http://localhost:4440/api/36/job/900a885b-576c-4c8c-a118-4bd1bf775c7b",
"permalink": "http://localhost:4440/project/ProjectEXAMPLE/job/show/900a885b-576c-4c8c-a118-4bd1bf775c7b"
},
"description": "echo $LANG [... 2 steps]",
"argstring": null,
"serverUUID": "bd2d85ef-f9f9-4805-b51c-a4bbcf5875e3",
"successfulNodes": [
"localhost"
]
}
And if you want to filter you can use any JSON parser tool, on my example I have used JQ (jq -r .status to get the status of execution):
#!/bin/sh
# protocol
protocol="http"
# basic rundeck info
rdeck_host="localhost"
rdeck_port="4440"
rdeck_api="36"
rdeck_token="nMp0uRsNUAH0w17JXxoDPqoBq6V4mB2O"
# specific api call info
execution="2"
# get the job forecast
curl -s --location --request GET "$protocol://$rdeck_host:$rdeck_port/api/$rdeck_api/execution/$execution" \
--header "Accept: application/json" \
--header "X-Rundeck-Auth-Token: $rdeck_token" | jq -r .status
Output:
succeeded
So, checking the Tableau documentation, you can send your results via Tableau API. In fact, you can use Server API Client (python) to do this, maybe is a good idea to use some Rundeck job to send that data to Tableau via Tableau API on any python script step.

Related

How to get raw content directly from api.github.com (or raw.githubusercontent.com)

First of all, please take note of the new API changes:
https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param/
The problem seems to be that I have to exchange a github personal access token for a temporary token, in order to read from raw.githubusercontent.com.
I have this request info:
set -e
export github_personal_access_token=a8f464fdxxxxxxxxxxxxxxxxxxxxxxfb89e6be
export file_url="https://api.github.com/repos/oresoftware/live-mutex/contents/package.json?ref=master"
mkdir tmp && cd tmp
curl -H "Authorization: token $github_personal_access_token" "$file_url" 2> err.log > output.json
the output.json looks like:
{
"name": "package.json",
"path": "package.json",
"sha": "6a2d55983bb641ff217d822d8e60dbb6c8f85ea3",
"size": 1343,
"url": "https://api.github.com/repos/ORESoftware/live-mutex/contents/package.json?ref=master",
"html_url": "https://github.com/ORESoftware/live-mutex/blob/master/package.json",
"git_url": "https://api.github.com/repos/ORESoftware/live-mutex/git/blobs/6a2d55983bb641ff217d822d8e60dbb6c8f85ea3",
"download_url": "https://raw.githubusercontent.com/ORESoftware/live-mutex/master/package.json",
"type": "file",
"content": "ewogICJuYW1lIjogImxpdmUtbXV0ZXgiLAogICJ2ZXJzaW9uIjogIjAuMi4y\nNCIsCiAgImRlc2NyaXB0aW9uIjogIlNpbXBsZSBtdXRleCB0aGF0IHVzZXMg\nYSBUQ1Agc2VydmVyOyB1c2VmdWwgaWYgeW91IGNhbm5vdCBpbnN0YWxsIFJl\nZGlzLCBldGMuIiwKICAibWFpbiI6ICJkaXN0L21haW4uanMiLAogICJ0eXBp\nbmdzIjogImRpc3QvbWFpbi5kLnRzIiwKICAidHlwZXMiOiAiZGlzdC9tYWlu\nLmQudHMiLAogICJiaW4iOiB7CiAgICAibG14X2FjcXVpcmVfbG9jayI6ICJh\nc3NldHMvY2xpL2FjcXVpcmUuanMiLAogICAgImxteF9yZWxlYXNlX2xvY2si\nOiAiYXNzZXRzL2NsaS9yZWxlYXNlLmpzIiwKICAgICJsbXhfaW5zcGVjdF9i\ncm9rZXIiOiAiYXNzZXRzL2NsaS9pbnNwZWN0LmpzIiwKICAgICJsbXhfbGF1\nbmNoX2Jyb2tlciI6ICJhc3NldHMvY2xpL3N0YXJ0LXNlcnZlci5qcyIsCiAg\nICAibG14X3N0YXJ0X3NlcnZlciI6ICJhc3NldHMvY2xpL3N0YXJ0LXNlcnZl\nci5qcyIsCiAgICAibG14X2xzIjogImFzc2V0cy9jbGkvbHMuanMiLAogICAg\nImxteCI6ICJhc3NldHMvbG14LnNoIgogIH0sCiAgInNjcmlwdHMiOiB7CiAg\nICAidGVzdCI6ICIuL3NjcmlwdHMvdGVzdC5zaCIsCiAgICAicG9zdGluc3Rh\nbGwiOiAiLi9hc3NldHMvcG9zdGluc3RhbGwuc2giCiAgfSwKICAicjJnIjog\newogICAgInRlc3QiOiAiLi90ZXN0L3NldHVwLXRlc3Quc2ggJiYgc3VtYW4g\nLS1kZWZhdWx0IgogIH0sCiAgInJlcG9zaXRvcnkiOiB7CiAgICAidHlwZSI6\nICJnaXQiLAogICAgInVybCI6ICJnaXQraHR0cHM6Ly9naXRodWIuY29tL09S\nRVNvZnR3YXJlL2xpdmUtbXV0ZXguZ2l0IgogIH0sCiAgImF1dGhvciI6ICJP\nbGVnemFuZHIgVkQiLAogICJsaWNlbnNlIjogIk1JVCIsCiAgImJ1Z3MiOiB7\nCiAgICAidXJsIjogImh0dHBzOi8vZ2l0aHViLmNvbS9PUkVTb2Z0d2FyZS9s\naXZlLW11dGV4L2lzc3VlcyIKICB9LAogICJob21lcGFnZSI6ICJodHRwczov\nL2dpdGh1Yi5jb20vT1JFU29mdHdhcmUvbGl2ZS1tdXRleCNyZWFkbWUiLAog\nICJkZXBlbmRlbmNpZXMiOiB7CiAgICAiQG9yZXNvZnR3YXJlL2pzb24tc3Ry\nZWFtLXBhcnNlciI6ICIwLjAuMTI0IiwKICAgICJAb3Jlc29mdHdhcmUvbGlu\na2VkLXF1ZXVlIjogIjAuMS4xMDYiLAogICAgImNoYWxrIjogIl4yLjQuMiIs\nCiAgICAidGNwLXBpbmciOiAiXjAuMS4xIiwKICAgICJ1dWlkIjogIl4zLjMu\nMiIKICB9LAogICJkZXZEZXBlbmRlbmNpZXMiOiB7CiAgICAiQHR5cGVzL25v\nZGUiOiAiXjEwLjEuMiIsCiAgICAiQHR5cGVzL3RjcC1waW5nIjogIl4wLjEu\nMCIsCiAgICAiQHR5cGVzL3V1aWQiOiAiXjMuNC4zIgogIH0KfQo=\n",
"encoding": "base64",
"_links": {
"self": "https://api.github.com/repos/ORESoftware/live-mutex/contents/package.json?ref=master",
"git": "https://api.github.com/repos/ORESoftware/live-mutex/git/blobs/6a2d55983bb641ff217d822d8e60dbb6c8f85ea3",
"html": "https://github.com/ORESoftware/live-mutex/blob/master/package.json"
}
}
but I just want the raw file content, not the metadata. The metadata does give me a link to the raw content:
https://raw.githubusercontent.com/ORESoftware/live-mutex/master/package.json
but for private repos, it requires an access token. So is there an easier way to do this other than this?
curl -H "Authorization: token $github_personal_access_token" "$file_url" |
jq -r '.content' | base64 -d > output.json
like I said, the biggest problem is I don't have a valid access_token in hand, and I can get an access token to download the file from the download_url, but that requires extra scripting steps. Looking for a single command. AKA, I don't want to have to install jq in a docker image if possible.
GitHub supports different media types to indicate what the client wishes to accept. In your case, you can get the raw file like this:
curl -H "Accept: application/vnd.github.v3.raw" \
-H "Authorization: token $github_personal_access_token" \
"$file_url" 2> err.log > output.json

How to update jenkins credentials via the REST API?

I found this to create credentials in Jenkins using a REST API call, how to update the credentials with a given id using the REST API calls?
curl -X POST 'http://jenkins:8080/credentials/store/system/domain/_/createCredentials' --data-urlencode 'json={
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": "apicredentials",
"username": "apicredentials",
"password": "P#$$W0rd",
"description": "apicredentials",
"stapler-class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
}
}'
Example with xml file:
$ cat credential.xml
<com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl>
<scope>GLOBAL</scope>
<id>apicredentials</id>
<username>manu</username>
<password>bar1</password>
</com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl>
$ curl -X POST -H content-type:application/xml -d #credential.xml 'http://USER:KEY#JENKINS_URL:8080/credentials/store/system/domain/_/credential/apicredentials/config.xml'

curl PUT command removes the reviewers from pull-request while trying to edit the description

I am using a curl PUT command to update the pull-request description. It is updating the description but removing the existing (already added) reviewers.
The curl command I am using to update the pull-request description:
curl -s -X PUT -u '<username>':'<password>' -H "Content-Type: application/json" -v -d '{"version": 1, "description": "'"should go together"'"}' https://{bitbucket.com}/rest/api/1.0/projects/{project_key}/repos/{repo_slug}/pull-requests/{pull_request_id}
Unfortunately you need also to pass the reviewers in the curl command:
curl -s --user USER:PASS --data #- --header 'Content-Type:application/json' --request PUT BITBUCKET-SERVER/rest/api/1.0/projects/PROJECT/repos/REPO/pull-requests/PULL-REQUEST <<EOF
{
"version": VERSION,
"description": "DESCRIPTION",
"reviewers": [
{
"user": {
"name": "REVIEWER-1"
},
"user": {
"name": "REVIEWER-2"
}
}
]
}
EOF
The Bitbucket API documentation says:
Note: the reviewers list may be updated using this resource. However
the author and participants list may not.

Trying to find Column value based filtering in HBase REST API

HI I am trying to build REST APIs for exposing data that resides in HBase. For simplicity I am using built in HBase REST API.I am following documentation from https://www.cloudera.com/documentation/enterprise/latest/topics/admin_hbase_rest_api.html . I have created one API for search utility which uses Rowkey. But I am stuck at that the my remaining API's require search based on column values. The documentation suggests steps but I am not able to use it.And there are no other sources available. I found there are filtering options in HBase Java Client like singlecolumnvalue filter with utilities like substringComaparator. Like these is there any way i can apply filters in HBase REST APIs?
The link you pasted show how to use scanner:
curl -vi -X PUT \
-H "Accept: text/xml" \
-H "Content-Type:text/xml" \
-d #filter.txt \
"http://example.com:20550/users/scanner/"
#stelcheck collected usage of some filters here. So if you want to use SingleColumnValueFilter with hbase rest api, your filter.txt will be like:
<Scanner batch="100">
<filter>
{
"type": "SingleColumnValueFilter",
"op": "EQUAL",
"family": "Y2Y=",
"qualifier": "cQ==",
"latestVersion": true,
"comparator": {
"type": "BinaryComparator",
"value": "dmFsdWU5"
}
}
</filter>
</Scanner>
This example is to find the cell with value value9 in column cf:q.
Update:
Add example to parse filter content in command line.
If don't want to use file content as data, just parse the content in command line.
For the example above, it will be :
curl -s -i -X PUT -H "Accept: text/xml" -H "Content-Type: text/xml" -d '<Scanner batch="100"><filter>{"type": "SingleColumnValueFilter", "op": "EQUAL", "family": "Y2Y=", "qualifier": "cQ==", "latestVersion": true, "comparator": { "type": "BinaryComparator", "value": "dmFsdWU5" } }</filter></Scanner>' "http://example.com:20550/users/scanner/"

get build status through github api

The GitHub API provides a lot of functionality, but is there a way to retrieve the build status for a commit? The GitHub UI provides information from the CI system we have configured, but I can't see this information exposed through the API?
You can access the status for a particular ref
GET https://api.github.com/repos/:owner/:repo/commits/:ref/statuses
For the value of :ref, you can use a SHA, a branch name, or a tag name.
It doesn't provide status directly, but offers you to create a status
That means the CI can have a final build step which publishes the status to GitHub repo that way.
POST /repos/:owner/:repo/statuses/:sha
For example:
{
"state": "success",
"target_url": "https://example.com/build/status",
"description": "The build succeeded!",
"context": "continuous-integration/jenkins"
}
(and that, for a given SHA1)
See for instance "Github Commit Status API with Bamboo from Atlassian", where:
${bamboo.buildResultsUrl} is the GitHub commit SHA1:
<xxx> is a placeholder value, which can be replaced by a value, or a variable ${var} as shown here.
Add those to your plan as Script.
complete.sh:
# specs and cukes results are stored in JUnit format under test-reports
if (grep 'failures="[^0]"' test-reports/* || \
grep 'errors="[^0]"' test-reports/*); then
curl -H "Authorization: token <MY_TOKEN>" --request POST \
--data '{"state": "failure", "description": "Failed!", \
"target_url": "${bamboo.buildResultsUrl}"}' \
https://api.github.com/repos/<USER>/<REPO>/statuses/${bamboo.repository.revision.number} > /dev/null
else
curl -H "Authorization: token <MY_TOKEN>" --request POST \
--data '{"state": "success", "description": "Success!", \
"target_url": "${bamboo.buildResultsUrl}"}' \
https://api.github.com/repos/<USER>/<REPO>/statuses \
/${bamboo.repository.revision.number} > /dev/null
fi
pending.sh:
curl -H "Authorization: token <MY_TOKEN>" --request POST \
--data '{"state": "pending", "description": "Build is running", \
"target_url": "${bamboo.buildResultsUrl}"}' \
https://api.github.com/repos/<USER>/<REPO>/statuses/${bamboo.repository.revision.number} > /dev/null