Create jobs and execute them in jenkins using REST - rest

I am trying to create a WCF REST client that will communicate to Jenkins and create a job from an XML file and then build the job. My understanding is that you can do that with Jenkins.
Can some one please provide some commands that you can type on a browser's address bar to create and build jobs? ie: http:localhost/jenkins/createItem?name=TESTJOB something along those lines.

Usually, when parsing through the documentation, it can take one or two days. It is helpful to be able to access code or curl commands to get you up and running in one hour. That is my objective with a lot of third party software.
See the post at http://scottizu.wordpress.com/2014/04/30/getting-started-with-the-jenkins-api/ which lists several of the curl commands. You will have to replace my.jenkins.com (ie JENKINS_HOST) with the your own url.
To create a job, for instance, try:
curl -X POST -H "Content-Type:application/xml" -d "<project><builders/><publishers/><buildWrappers/></project>" "http://JENKINS_HOST/createItem?name=AA_TEST_JOB2"
This uses a generic config. You can also download a config from a manually created job and then use that as a template.
curl "http://JENKINS_HOST/job/MY_JOB_NAME/config.xml" > config.xml
curl -X POST -H "Content-Type:application/xml" -d #config.xml "http://JENKINS_HOST/createItem?name=AA_TEST_JOB3"
To execute the job (and set string parameters), use:
curl "http://JENKINS_HOST/job/MY_JOB_NAME/build"
curl "http://JENKINS_HOST/job/MY_JOB_NAME/buildWithParameters?PARAMETER0=VALUE0&PARAMETER1=VALUE1"

See the Jenkins API Wiki page (including the comments at the end). You can fill in the gaps using the documentation provided by Jenkins itself; for example, http://JENKINS_HOST/api will give you the URL for creating a job and http://JENKINS_HOST/job/JOBNAME/api will give you the URL to trigger a build.
I highly recommend avoiding the custom creation of job configuration XML files and looking at something like the Job DSL plugin instead. This gives you a nice Groovy-based DSL to create jobs programmatically - much more concise and less error-prone.

Thanks to a GIST - https://gist.github.com/stuart-warren/7786892
Check if job exists
curl -XGET 'http://jenkins/checkJobName?value=yourJobFolderName' --user user.name:YourAPIToken
With folder plugin
curl -s -XPOST 'http://jenkins/job/FolderName/createItem?name=yourJobName' --data-binary #config.xml -H "Content-Type:text/xml" --user user.name:YourAPIToken
Without folder plugin
curl -s -XPOST 'http://jenkins/createItem?name=yourJobName' --data-binary #config.xml -H "Content-Type:text/xml" --user user.name:YourAPIToken
Create folder
curl -XPOST 'http://jenkins/createItem?name=FolderName&mode=com.cloudbees.hudson.plugins.folder.Folder&from=&json=%7B%22name%22%3A%22FolderName%22%2C%22mode%22%3A%22com.cloudbees.hudson.plugins.folder.Folder%22%2C%22from%22%3A%22%22%2C%22Submit%22%3A%22OK%22%7D&Submit=OK' --user user.name:YourAPIToken -H "Content-Type:application/x-www-form-urlencoded"

If you want to create a job into a view given the view exists.
curl -X POST -H "Content-Type:application/xml" -d #build.xml "http://jenkins_host/view/viewName/createItem?name=itemName"
the build.xml filetemplate could be found in the root directory of a job's workspace
if you want to create a view:
curl -X POST -H "Content-Type:application/xml" -d #view.xml "http://jenkins_host/createView?name=viewName"
the content of the file view.xml could be:
<?xml version="1.0" encoding="UTF-8"?>
<hudson.model.ListView>
<name>viewName</name>
<filterExecutors>false</filterExecutors>
<filterQueue>false</filterQueue>
<properties class="hudson.model.View$PropertyList"/>
<jobNames>
<comparator class="hudson.util.CaseInsensitiveComparator"/>
</jobNames>
<jobFilters/>
<columns>
<hudson.views.StatusColumn/>
<hudson.views.WeatherColumn/>
<hudson.views.JobColumn/>
<hudson.views.LastSuccessColumn/>
<hudson.views.LastFailureColumn/>
<hudson.views.LastDurationColumn/>
<hudson.views.BuildButtonColumn/>
</columns>
</hudson.model.ListView>
and to check if a view exists:
curl -X POST -H "Content-Type:application/xml" "http://jenkins_host/checkViewName?value=viewName"
to check if a job exists:
curl -X POST -H "Content-Type:application/xml" "http://jenkins_host/checkJobName?value=jobName"

To create a job:
curl -X POST -H "Content-Type:application/xml" -d "<project><builders/><publishers/><buildWrappers/></project>" -u username: API_Token http://JENKINS_HOST/createItem?name=AA_TEST_JOB2
To build a job:
curl -X POST -u username:API_TOKEN http://JENKINS_HOST/job/MY_JOB_NAME/build

In case you need to make the same HTTP calls using the Python requests library, instead of CURL...
Download a job config:
import requests
auth = ("username", "api_token")
url = "http://" + JENKINS_HOST + "/job/" + JOB_NAME + "/config.xml"
response = requests.get(url, auth=auth)
open('config.xml', 'wt').write(response.text)
Create a new job using same config:
url = "http://" + JENKINS_HOST + "/createItem?name=" + NEW_JOB_NAME
headers = {'content-type': 'text/xml'}
data = response.text
response = requests.post(url, auth=auth, headers=headers, data=data)
Omit auth parameter when not needed.

Related

Add existing user to existing group apache ranger

im looking at apache Ranger rest API to add an existing internal user/users to an existing internal group.
I have been looking at the docs and cant seem to find something useful,is there an API for that?
Try following api call:-
curl -ivk -u admin:admin -H "Content-Type: application/json" -d '{"id":13,"createDate":"2020-12-23T07:55:04Z","updateDate":"2020-12-23T07:55:04Z","owner":"rangerusersync","updatedBy":"rangerusersync","name":"atlas","password":"*****","description":"atlas - add from Unix box","groupIdList":[6,59,4,131,133],"groupNameList":["atlas","hadoop","shadow"],"status":0,"isVisible":1,"userSource":1,"userRoleList":["ROLE_USER"],"otherAttributes":"{\"full_name\":\"atlas\",\"original_name\":\"atlas\"}"}' -X PUT https://RANGER_HOST:6182/service/xusers/users
Its little bit big but if user is already added to ranger then run following api to get the id information for users:-
curl -ivk -u admin:admin -H "Accept : application/json" -X GET https://RANGER_HOST:6182/service/xusers/users
Once you have IDs for all users, you can run following curl api to get the json formatted data which you can use to modify and then use PUT method in first API I mentioned:-
curl -ivk -u admin:admin -H "Accept: application/json" -X GET https://RANGER_HOST:6182/service/xusers/users/13
Above Curl api should return something like following:-
{"id":13,"createDate":"2020-12-23T07:55:04Z","updateDate":"2020-12-23T16:45:14Z","owner":"rangerusersync","updatedBy":"admin","name":"atlas","password":"*****","description":"atlas - add from Unix box","groupIdList":[133,6],"groupNameList":["apitest","atlas","hadoop","shadow","ssb"],"status":0,"isVisible":1,"userSource":1,"userRoleList":["ROLE_USER"],"otherAttributes":"{\"full_name\":\"atlas\",\"original_name\":\"atlas\"}"}
you have to modify "groupIdList":[133,6] from the above output copy entire output and pass it with PUT method as shown in the first api call mentioned above.

Create Bitbucket Pull Request Comment via REST

I want to create a bitbucket comment after finishing my bamboo build. Unfortunately my rest call didnt work. ALso Im getting no response.
Am I missing something?
curl -u "USER:PASSWORD" -X POST -d '{"body": "Your Message to Comment"}' "http://URL/2.0/repositories/OWNER/REPO/pullrequests/ID/comments"
Execute the following command:
curl -u "USER:PASSWORD" --request POST --data '{"text": "YOUR-COMMENT"}' "BITBUCKET-SERVER/rest/api/1.0/projects/PROJECT/repos/REPO/pullrequests/ID/comments"

How to execute a jar-packaged scala program via Apache Livy on Spark that responds with a result directly to a client request?

What I intend to achieve is having a Scala Spark program (in a jar) receive a POST message from a client e.g. curl, take some argument values, do some Spark processing and then return a result value to the calling client.
From the Apache Livy documentation available I cannot find a way how I can invoke a compiled and packaged Spark program from a client (e.g. curl) via Livy in an interactive i.e. session mode. Such a request/reply scenario via Livy can be done with Scala code passed in plain text to the Spark shell. But how can I do it with a Scala class in a packaged jar?
curl -k --user "admin:mypassword" -v \
-H "Content-Type: application/json" -X POST \
-d #Curl-KindSpark_ScalaCode01.json \
"https://myHDI-Spark-Clustername.azurehdinsight.net/livy/sessions/0/statements" \
-H "X-Requested-By: admin"
Instead of Scala source code as data (-d #Curl-KindSpark_ScalaCode01.json) I would rather pass the path and filename of the jar-file and a ClassName and Argument values. But how?
Make a uber jar of your Spark app with sbt-assemby plugin.
Upload jar file from the previous step to your HDFS cluster:
hdfs dfs -put /home/hduser/PiNumber.jar /user/hduser
Execute your job via livy:
curl -X POST -d '{"conf": {"kind": "spark" , "jars": "hdfs://localhost:8020/user/hduser/PiNumber.jar"}}' -H "Content-Type: application/json" -H "X-Requested-By: user" localhost:8998/sessions
check it:
curl localhost/sessions/0/statements/3:
{"id":3,"state":"available","output":{"status":"ok","execution_count":3,"data":{"text/plain":"Pi
is roughly 3.14256"}}}
p.s.
Spark Livy API for Scala/Java requires using an uber jar file. sbt-assembly doesn't make fat jar instantly, it annoys me.
Usually, I use Python API of Livy for smoke tests and tweaking.
Sanity checks with Python:
curl localhost:sessions/0/statements -X POST -H 'Content-Type: application/json' -d '{"code":"print(\"Sanity check for Livy\")"}'
You can put more complicated logic to field code.
BTW, it's a way in which popular notebooks for Spark works - sending the source code to cluster via Livy.
Thx, I will try this out. In the meanwhile I found another solution:
$ curl -k --user "admin:" -v -H "Content-Type: application/json" -X POST -d #Curl-KindSpark_BrandSampleModel_SessionSetup.json "https://mycluster.azurehdinsight.net/livy/sessions
with a JSON file containing
{
"kind": "spark",
"jars": ["adl://skylytics1.azuredatalakestore.net/skylytics11/azuresparklivy_2.11-0.1.jar"]
}
and with the uploaded jar in the Azure Data Lake Gen1 account containing the Scala object and then post the statement
$ curl -k --user "admin:myPassword" -v -H "Content-Type: application/json" -X POST -d #Curl-KindSpark_BrandSampleModel_CodeSubmit.json "https://mycluster.azurehdinsight.net/livy/sessions/4/statements" -H "X-Requested-By: admin"
with the content
{
"code": "import AzureSparkLivy_GL01._; val brandModelSamplesFirstModel = AzureSparkLivyFunction.SampleModelOfBrand(sc, \"Honda\"); brandModelSamplesFirstModel"
}.
So I told Livy to start an interactive Spark session and load the specified jar and passed some code to invoke a member of the object in the jar. It works. Will check your advice too.

Atlassian Confluence : Updating parent container of an attachment

I'd like to move an attachment to a different space (i.e, update the parent container of an attachment in a Confluence page) using Cloud REST API and "curl" command. I looked up REST API endpoint URL and found this:
PUT /rest/api/content/{id}/child/attachment/{attachmentId}
Could somebody provide an example for a correctly constructed JSON input to achieve this? I tried the one below to move an attachment called "test.jpeg" with ID "att000" and current version 5 to parent with ID 1234, but it fails:
curl -v -u 'admin:password' -X PUT -H 'Content-Type:application/json' -d '{"id":"att000","type":"attachment","title":"test.jpeg","version":{"number":5,"minorEdit":false},"ancestors":[{"id":1234}]' -H ‘X-Atlassian-Token:access-token' https://test.atlassian.net/wiki/rest/api/content/170234/child/attachment | python -m json.tool
Given below is the error message
< HTTP/1.1 415 Unsupported Media Type
.
.
.
No JSON object could be decoded
Thank you for your time!
I believe in your curl example you are uploading it under space which is not possible. Attachments in Confluence needs to be under content. Take a look at following example:
curl -v -S -u admin:admin -X POST -H "X-Atlassian-Token: no-check" -F "file=#myfile.txt" -F "comment=this is my file" "http://localhost:8080/confluence/rest/api/content/3604482/child/attachment" | python -mjson.tool
You have to set the container property in the request object.
Specify "container":{"id":"123456","type":"attachment"}
curl -v -u 'admin:password' -X PUT -H 'Content-Type:application/json' -d '{"id":"att000","type":"attachment","title":"test.jpeg","version":{"number":5,"minorEdit":false},"container":{"id":"123456","type":"attachment"}, "ancestors":[{"id":1234}]' -H ‘X-Atlassian-Token:access-token' https://test.atlassian.net/wiki/rest/api/content/170234/child/attachment | python -m json.tool
id => id of the page where attachment is being moved.

NextCloud file tagging through WebDAV script

I am using NextCloud 11 to store my personal files, and I use the simple curl script from the documentation in order to upload files to my NextCloud drive:
curl -u user:pw -T test.pdf "http://localhost/nextcloud/remote.php/dav/files/user/test/test.pdf"
Moreover, I would like to directly add some tags to the uploaded files. However, in the official documentation, they just show how files can be uploaded, deleted and moved through the WebDAV interface.
Does anybody have a hint how I could tag a file remotely?
I have posted the same question in the official NextCloud community forum, but I did not receive a response yet. In case I receive a response, I will post it here.
POST https://yournextcloud.com/index.php/api/v1/files/path/to/file
Payload is JSON:
{"tags": ["tag1", "tag2"]}
You will need to authenticate yourself using Basic Auth
Edit: The API can only be called from inside Nextcloud because the CSRF token is required.
For the record, after a bit of digging I found https://doc.owncloud.com/server/latest/developer_manual/webdav_api/tags.html which does the job for nextcloud as well. In a nutshell:
Get file id for a given file:
curl --silent -u user:password -X PROPFIND -H "Content-Type: text/xml" \
--data-binary "#file-propfind.xml" https://nextcloud/remote.php/webdav/file' | xmllint --format -
with a file-propfind.xml in your directory containing something like
<?xml version="1.0" encoding="utf-8" ?>
<a:propfind xmlns:a="DAV:" xmlns:oc="http://owncloud.org/ns">
<a:prop>
<oc:fileid/>
</a:prop>
</a:propfind>
Then get list of tags for this file using
curl --silent -u user:password -X PROPFIND -H "Content-Type: text/xml" \
--data-binary "#tags-propfind.xml" https://nextcloud/remote.php/dav/systemtags-relations/files/<FILEID>" | xmllint --format -
where FILEID is the number you got as oc:fileid in the previous response and tags-propfind.xml a file containing something like
<?xml version="1.0" encoding="utf-8" ?>
<a:propfind xmlns:a="DAV:" xmlns:oc="http://owncloud.org/ns">
<a:prop>
<oc:display-name/>
<oc:user-visible/>
<oc:user-assignable/>
<oc:id/>
</a:prop>
</a:propfind>
This is for tag reading, but the API document also explain how to add a tag in the same fashion.