How to pass API parameters to GCP cloud build triggers - rest

I have a large set of GCP Cloud Build Triggers that I invoke via a Cloud scheduler, all running fine.
Now I want to invoke these triggers by an external API call and pass them dynamic parameters that vary in values and number of parameters.
I was able to start a trigger by running an API request but any JSON parameters in the API request that I sent were ignored.
Google talks about substitution parameters at https://cloud.google.com/cloud-build/docs/configuring-builds/substitute-variable-values. I define these variables in the cloudbuild.yaml file, however they were not propagated into my shell script from the API request.
I don't any errors with authentication or authorization, so security may not be an issue.
Is my idea supported at all or do I need to resort to another solution such as running a GKE cluster with containers that would expose its API (a very heavy-boxing solution).

We do something similar -- we migrated from Jenkins to GCB but for some people we still need a nicer "UI" to start builds / pass variables.
I got scripts from here and modified them to our own needs: https://medium.com/#nieldw/put-your-build-triggers-into-source-control-with-the-cloud-build-api-ed0c18d6fcac
Here is their REST API: https://cloud.google.com/cloud-build/docs/api/reference/rest/v1/projects.triggers/run
For the script below, keep in mind you need the trigger-id of what you want to run. (you can also get this by parsing the output of another REST API.)
TRIGGER_ID=1
# we need to specify ATLEAST the branch name or commit id (check after)
BRANCH_OR_SHA=$2
# check if branch_name or commit_sha
if [[ $BRANCH_OR_SHA =~ [0-9a-f]{5,40} ]]; then
# is COMMIT_HASH
COMMIT_SHA=$BRANCH_OR_SHA
BRANCH_OR_SHA="\"commitSha\": \"$COMMIT_SHA\""
else
# is BRANCH_NAME
BRANCH_OR_SHA="\"branchName\": \"$BRANCH_OR_SHA\""
fi
# This is the request we send to google so it knows what to build
# Here we're overriding some variables that we have already set in the default 'cloudbuild.yaml' file of the repo
cat <<EOF > request.json
{
"projectId": "$PROJECT_ID",
$BRANCH_OR_SHA,
"substitutions": {
"_MY_VAR_1": "my_value",
"_MY_VAR_2": "my_value_2"
}
}
EOF
# our curl post, we send 'request.json' with info, add our Token, and set the trigger_id
curl -X POST -T request.json -H "Authorization: Bearer $(gcloud config config-helper \
--format='value(credential.access_token)')" \
https://cloudbuild.googleapis.com/v1/projects/"$PROJECT_ID"/triggers/"$TRIGGER_ID":run

Related

How to Use Command Line Parameters in JMeter

I'm using Jmeter for testing APIs and I want to parametrize the project's path from the terminal and then I want to use this parameter in JMeter.
The parameter that I've sent via Command Line :
./jmeter -n -t your_script.jmx -Jurl=abcdef.com
The parameter that I've used in User Defined Variables :
${__P(url)}
But when I run my automation in JMeter, my test scripts are not going to URL that's been defined. When I check the request body, I see POST https://1 as URL.
Please see the attached photos.
https://mylifebox.com/shr/3df5bb35-cf43-4488-b20b-5c2d59656212&language=en
Let's start clean:
In the User Defined Variables configure the variable with the name of url and the value of ${__P(url,)}
In the HTTP Request sampler (or even better HTTP Request Defaults) put ${url} into "Server Name or IP" field:
Run your test in command-line non-GUI mode like:
jmeter -n -t your_script.jmx -Jurl=abcdef.com -f -l result.jtl
mind this -f argument which tells JMeter to overwrite the existing results file (it might be the case you're looking into "old" results where the url property value was starting with 1)
That's it, you should see the HTTP Request sampler making a call to abcdef.com in the .jtl results file. And if you change this url parameter - you will see the impact in the .jtl results file:
Put ${__P(url)} inside Server Name field in HTTP Request.
Domain name or IP address of the web server, e.g. www.example.com. [Do not include the http:// prefix.] Note: If the "Host" header is defined in a Header Manager, then this will be used as the virtual host name.
Don't use User Defined Variables

Unable to import test results to jira via rest api

i'm using the following curl command to import the output.xml file into jira test execution key and receiving error as below. I'm sure the test execution key is existing in jira and the project id is also correct. Any pointers?
curl -H "Content-Type: multipart/form-data" -u userid:pass -F "file=#output.xml" "https://server/rest/raven/latest/import/execution/robot?projectKey=PROJKEY+and+testExecKey=TESTEXNKEY" -o error.txt
The error i receive is as below
The User "userid" does not have permission to create issues
Why does it try to create new issue while the issue already exists? And why does it say the user doesn't have access when the access is there?
You probably mean Xray add-on and you probably use the same request per their documentation. The problem seems to be with your parameter syntax. It should be .../robot/?projectKey=PROJKEY&testExecKey=TESTEXNKEY (i.e. & instead of +and+).
Plus I would explicitly specify it's a POST request: curl -X POST ....
But their error message is not clear, anyway. I don't have Xray available right now, but if you keep having troubles, I would recommend checking with their support.

Spring Boot Admin: example for filtering notifications

I want to use the notification filtering function in Spring Boot Admin (see http://codecentric.github.io/spring-boot-admin/current/#filtering-notifications ) but don't see how the filtering rules mentioned in the documentation can be configured or with what HTTP requests exactly they can be added/removed. Does someone have an example for such a configuration? (e.g. excluding applications with certain name patterns from notification or disabling all notifications; curl command for a HTTP request activating such a rule?)
Example code from the documentation:
#Bean
public FilteringNotifier filteringNotifier() {
CompositeNotifier delegate = new CompositeNotifier(this.otherNotifiers.getIfAvailable(Collections::emptyList));
return new FilteringNotifier(delegate, this.repository);
}
This does not seem to configure any specific rules.
In https://github.com/codecentric/spring-boot-admin/tree/master/spring-boot-admin-samples/spring-boot-admin-sample-servlet I also do not see any code that configures specific filters.
Is this sufficient or is additional configuration code necessary?
I did some further investigation and found out how it works - maybe this helps someone. The REST requests for notification filtering are processed by the following code: NotificationController.java
From the code, one can see that the following requests are possible:
# filter notifications for application xxx for the next 10000 milliseconds
# (ttl is optional)
curl -d "applicationName=xxx&ttl=10000" -X POST http(s)://.../notifications/filters
# filter notifications for instanceId yyy for the next 10000 milliseconds
# (ttl is optional)
curl -d "instanceId=yyy&ttl=10000" -X POST http(s)://.../notifications/filters
# Get all added notification filters with their ids
curl http(s)://.../notifications/filters
# Delete notification filter with id <id>
curl -X DELETE http(s)://.../notifications/filters/<id>
Authentication has to be added if necessary. No other changes need to be made to the Spring Boot Admin application for this to work, it is sufficient to register the FilteringNotifier.

How to remove a Github Environment

My question is about cleaning up the "Environments" tab on a Github repository.
I previously deployed via Heroku, using automatic deployment from two separate Github branches (one for staging, one for production).
This created a tab "Environments" on the repository, in which both Heroku environments were shown - exactly as intended.
Once I started to dive into Heroku pipelines, I have now configured the app to be promoted to production from staging, so the production environment no longer auto-deploys from a branch.
The Environments tab on my Github repo has no way to remove the environment that I no longer use. I can't seem to find any place on Github or Heroku to make Github "forget" this deployment environment.
I hope my question is clear enough; if I can elaborate on anything, please let me know.
I made a little webpage/script too, to automate the process (I don't have Python installed, and I didn't see that someone else had already made a script), and this is online and putting your info will do the process automatically.
Stackblitz - Github Deployments deleter
Edit 18/07/2020: I copied the script from Stackblitz to a local snippet code in here too, just in case Stackblitz disappears:
// RECOMMENDED: Disconnect HEROKU from Github before doing this (though not strictly necessary, I think).
//See https://stackoverflow.com/a/61272173/6569950 for more info.
// PARAMETERS
const TOKEN = ""; // MUST BE `repo_deployments` authorized
const REPO = "your-repo"; // e.g. "monorepo"
const USER_OR_ORG = "your-name"; // e.g. "your-name"
// GLOBAL VARS
const URL = `https://api.github.com/repos/${USER_OR_ORG}/${REPO}/deployments`;
const AUTH_HEADER = `token ${TOKEN}`;
// UTILITY FUNCTIONS
const getAllDeployments = () =>
fetch(`${URL}`, {
headers: {
authorization: AUTH_HEADER
}
}).then(val => val.json());
const makeDeploymentInactive = id =>
fetch(`${URL}/${id}/statuses`, {
method: "POST",
body: JSON.stringify({
state: "inactive"
}),
headers: {
"Content-Type": "application/json",
Accept: "application/vnd.github.ant-man-preview+json",
authorization: AUTH_HEADER
}
}).then(() => id);
const deleteDeployment = id =>
fetch(`${URL}/${id}`, {
method: "DELETE",
headers: {
authorization: AUTH_HEADER
}
}).then(() => id);
// MAIN
getAllDeployments()
.catch(console.error)
.then(res => {
console.log(`${res.length} deployments found`);
return res;
})
.then(val => val.map(({
id
}) => id))
.then(ids => Promise.all(ids.map(id => makeDeploymentInactive(id))))
.then(res => {
console.log(`${res.length} deployments marked as "inactive"`);
return res;
})
.then(ids => Promise.all(ids.map(id => deleteDeployment(id))))
.then(res => {
console.log(`${res.length} deployments deleted`);
return res;
})
.then(finalResult => {
const appDiv = document.getElementById("app");
appDiv.innerHTML = `
<h1>CLEANUP RESULT</h1>
<br>
Removed Deployments: ${finalResult.length}
<br>
<br>Ids:<br>
${JSON.stringify(finalResult)}
<br><br><br><br><br><br>
<p>(Open up the console)</p>
`;
});
h1,
h2 {
font-family: Lato;
}
<div id="app">
<h1>Github Deployment's Cleaner</h1>
<p> You need to put the parameters in!</p>
</div>
There doesn't seem to be UI for it, but you can do it using the GitHub API.
You should probably disconnect GitHub and Heroku before doing this.
First, go to your GitHub account settings, then developer settings, then personal access tokens. Create a new token that has repo_deployments allowed. After it's generated, save the hexadecimal token, you'll need it for the upcoming API requests.
For these examples I'll assume that your username is $aaaa and your repo name is $bbbb and your access token is $tttt. Replace these with your actual username and repo name and access token. Or just use shell variables to store the actual values which will let you paste the code blocks directly.
First, list all the deployments on your repo:
curl https://api.github.com/repos/$aaaa/$bbbb/deployments
Each deployment has an id integer. Note it, and replace $iiii in the upcoming code blocks with that ID. Or create another shell variable for it.
Now you have to create an "inactive" status for that deployment:
curl https://api.github.com/repos/$aaaa/$bbbb/deployments/$iiii/statuses -X POST -d '{"state":"inactive"}' -H 'accept: application/vnd.github.ant-man-preview+json' -H "authorization: token $tttt"
And now you can delete the deployment forever:
curl https://api.github.com/repos/$aaaa/$bbbb/deployments/$iiii -X DELETE -H "authorization: token $tttt"
If you have multiple deployments, send the first request to see all the deployments that remain, and then you can delete those too if you want.
After you delete all the deployments, the environments button on the GitHub repo will disappear.
Information sourced from the GitHub deployments documentation and the GitHub oauth documentation. This worked for me.
This will not answer the OP question, I thought it would at first, but it didn't behave as I had expected. Therefore, I'm adding this answer as a community wiki.
GitHub seems to have two notions of "Environments", the one the OP means are "public environments", but GitHub also seems to have some kind of "private environments".
I'm adding my experience as an answer below because it is really confusing.
You can access "private environments" through "Settings > Environments". (E.g: https://github.com/UnlyEd/next-right-now/settings/environments)
You can then delete each environment. It'll prompt a confirm dialog. Upon confirm, the environment will be destroyed.
I deleted the "staging" and "production" environments.
But the public environments still continue to exist, alongside all their deployments. (and this is not what the OP wants)
Public environments still contains "staging" and "production".
I've made an interactive python script which can delete specific environments by name (which was my issue) or all of your deployments. Check it out and let me know if it works for you: https://github.com/VishalRamesh50/Github-Environment-Cleaner.
This will also actually delete all of your deployments even if you have more than 30 unlike the other scripts here because it goes through the paginated response data from Github's API and doesn't just use the first page.
Based on Cadence's answer, I built the following bash script. Just set the appropriate parameters and let it run.
The token requires repo_deployment OAuth scope.
env=asd
token=asd
repo=asd
user=asd
for id in $(curl -u $user:$token https://api.github.com/repos/$user/$repo/deployments\?environment\=$env | jq ".[].id"); do
curl -X POST -u $user:$token -d '{"state":"inactive"}' -H 'accept: application/vnd.github.ant-man-preview+json' https://api.github.com/repos/$user/$repo/deployments/$id/statuses
curl -X DELETE -u $user:$token https://api.github.com/repos/$user/$repo/deployments/$id
done
I Expect Your already got the answer for your question This answer is for those who wan't to Remove The Environments Tab from There GitHib Repo
As I Understand
If You Want to Remove the Environments Tab Shown on The Right Side of The Screen
If You Want to Remove Specific Environments
Remove Environments Tab
To Remove Entire Tab
On Your Repo Page Click on Settings Button
Uncheck the Mark on Environments
It Will Remove The Environments Tab from the Right Side
Remove Specific Environments
For Removing Specific Environment
Click on SettingsClick On EnvironmentsHere You Can Remove a Specific Environment
Reply from GitHub for Removing of Environments from Private Repo
Hi Deekshith,
Thanks for writing in!
As it turns out, the Environments feature is an enterprise-only feature for private repositories, however, they still get created as a side effect of Deployments in private repositories.
https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment
This means that though you'd be able to create Deployments in a private repository, you won't be able to manage (e.g delete, add rules e.t.c) the deployment Environments. I'm afraid this is currently the expected behaviour. Our engineers are still discussing how best to help users who'd want to delete environments in non-enterprise private repositories, such as in your case. We’ll be sure to update you as soon as there’s any news to share.
Sorry that we could not be of more help with this -- please let us know if you have any questions at all!
Regards,
Peter
GitHub Support
I've built an online tool to help out removing deployments
I ran into the same problem and found #spersico's code very handy, but needed a bit more tooling/feedback. I iterated on #spersico code to add a bit of frontend.
Same as #spersico's version, all calls are made client side (and are visible in your console/network logs).
Project is opensource on Github and has a hosted version on Netlify which can be used instantly:
https://github-deployment-cleaner.netlify.app/
Unfortunately, it seems that the 'Deployments' dashboard is currently in beta, which means that they may not have a feature yet.
Read here.
I've just used this Python script: 5 minutes and I removed my unwanted environment:
https://github.com/VishalRamesh50/Github-Environment-Cleaner
There is a discussion in GitHub Community: https://github.community/t/how-to-remove-the-environment-tab/10584/10?u=aerendir
Please, vote up the feature.
Using GitHub CLI:
org=':org:'
repo=':repo:'
env=':env:'
gh api "repos/${org}/${repo}/deployments?environment=${env}" \
| jq -r ".[].id" \
| xargs -n 1 -I % sh -c "
gh api -X POST -F state=inactive repos/${org}/${repo}/deployments/%/statuses
gh api -X DELETE repos/${org}/${repo}/deployments/%
"
I don't know if this was posted already but I definitely didn't want do delete my 40+ deployments manually so I created following script, feel free to use it too :)
#!/bin/bash
REPO=<your GH name>/<your project name>
TOKEN=<your personal access token>
# https://starkandwayne.com/blog/bash-for-loop-over-json-array-using-jq/
for deployment in $(curl https://api.github.com/repos/$REPO/deployments | jq -r '.[] | #base64'); do
DEPLOYMENT_ID=$(echo "$deployment" | base64 --decode | jq -r '.id')
echo "$DEPLOYMENT_ID"
curl "https://api.github.com/repos/$REPO/deployments/$DEPLOYMENT_ID/statuses" \
-X POST \
-d '{"state":"inactive"}' \
-H 'accept: application/vnd.github.ant-man-preview+json' \
-H "authorization: token $TOKEN"
done
for deployment in $(curl https://api.github.com/repos/$REPO/deployments | jq -r '.[] | #base64'); do
DEPLOYMENT_ID=$(echo "$deployment" | base64 --decode | jq -r '.id')
curl "https://api.github.com/repos/$REPO/deployments/$DEPLOYMENT_ID" \
-X DELETE \
-H "authorization: token $TOKEN"
done
You can remove the environment from a repository if its public. But in case of private repositories, either you have to make it public or use the github API. Both works, but here is my approach for deleting the environments.
I created an npm package (here) for the same. Just get the github access token, with repo_deployments scope enabled.
Now run npx delete-github-environment and select the environment that you want to delete. If everything went right, your environment will be deleted.
PS: Here's my github repo - (github), feel free to contribute to the code.
To remove a Github Environment go to Settings -> Environments -> and click the trashcan icon next to the Environment you want to delete (see picture below).
More can be read about it from the Github official documentation:
Deleting an Environment

Copy graphite dashboard to another graphite dashboard

I have a production graphite dashboard. I've saved some graphs under the tag abc so that you can access it using http://prod-graphite.com/dashboard/abc.
I've another dashboard for staging hosted on different server. Let's say the URL is http://staging-graphite.com/dashboard/.
I want to copy all the graphs of prod /abc to staging as I don't want to go through the trouble of creating 20 graphs again. I've tried the Copy Dashboard feature provided by graphite but it is not working. Nothing happens when I enter the prod URL. any help?
GET/POST http://your.graphite.host/dashboard/load/YOUR_DASHBOARD_NAME - gives you dump of specified dashboard. It returns json with state as root object, that holds dashboards' structure.
POST http://your.graphite.host/dashboard/save/NEW_DASHBOARD_NAME - lets you save data as new dashboard. Requires state parameter with dashboards' structure.
Oneliner, fetchs dump, prepares body, save:
curl -o- http://graphite.host/dashboard/load/DASH_NAME | \
python -c "import json,sys,urllib;o=json.load(sys.stdin);print('state=%s' % urllib.quote(json.dumps(o['state'])));" | \
curl -X POST http://graphite.host/dashboard/save/COPY_OF_DASH_NAME -d #-