I have been working on an installation of OKD on an air-gapped environment. The first major step has been mirroring the OKD images so that they can be moved over to the new environment and pulled locally. I've been following a combination of the OpenShift documentation and this article, as well as this resource for getting my certificates set up. I have been making slow but consistent progress.
However, I am now having trouble when attempting to actually mirror the files using
oc adm -a ${LOCAL_SECRET_JSON} release mirror \
--from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE} \
--to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
--to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}
I get the following, encouraging response:
info: Mirroring 120 images to host.okd-registry.dns:5000/ocp4/openshift4 ...
followed by blobs: and manifests: lines, and finally the line
stats: shared=0 unique=7 size=105.3MiB ratio=1.00
I then get about 50 lines stating
error: unable to retrieve source image quay.io/openshift-release-dev/ocp-v4.0-art-dev manifest
sha256:{some value}: unauthorized: access to the requested resource is not authorized
I have a quay account but I am not sure if that is required even after my research, and if it is, where or how I would log into it. I have attempted doing so using oc login followed by various addresses within the release structure, but if this is the solution, I may be using the wrong arguments as I have not been able to find any instructions on doing this.
I have also tried the command with sudo. I doubt that is an issue but I tried it anyway.
I suppose the issue could be with my certificates, but I am not sure how to determine if this is the case.
Any guidance or suggestions would be much appreciated.
It has been determined that the OKD documentation is inaccurate at the time that I am posting this answer, and was instructing readers to pull from the OCP image repository rather than the OKD repository, which apparently requires additional credentials. A bug has been logged and the documentation will hopefully be updated soon.
The correct environment variables and full command to mirror the images are as follows:
LOCAL_REGISTRY=localhost:5000 (or your local domain name and port for the registry)
LOCAL_REPOSITORY=okd
LOCAL_SECRET_JSON=<full path to your pull secret>
OCP_RELEASE=4.5.0-0.okd-2020-10-15-235428
PRODUCT_REPO=openshift
RELEASE_NAME=okd
ARCHITECTURE=not-used-in-okd
oc adm -a ${LOCAL_SECRET_JSON} release mirror \
--from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE} \
--to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
--to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE} --dry-run
I have a large set of GCP Cloud Build Triggers that I invoke via a Cloud scheduler, all running fine.
Now I want to invoke these triggers by an external API call and pass them dynamic parameters that vary in values and number of parameters.
I was able to start a trigger by running an API request but any JSON parameters in the API request that I sent were ignored.
Google talks about substitution parameters at https://cloud.google.com/cloud-build/docs/configuring-builds/substitute-variable-values. I define these variables in the cloudbuild.yaml file, however they were not propagated into my shell script from the API request.
I don't any errors with authentication or authorization, so security may not be an issue.
Is my idea supported at all or do I need to resort to another solution such as running a GKE cluster with containers that would expose its API (a very heavy-boxing solution).
We do something similar -- we migrated from Jenkins to GCB but for some people we still need a nicer "UI" to start builds / pass variables.
I got scripts from here and modified them to our own needs: https://medium.com/#nieldw/put-your-build-triggers-into-source-control-with-the-cloud-build-api-ed0c18d6fcac
Here is their REST API: https://cloud.google.com/cloud-build/docs/api/reference/rest/v1/projects.triggers/run
For the script below, keep in mind you need the trigger-id of what you want to run. (you can also get this by parsing the output of another REST API.)
TRIGGER_ID=1
# we need to specify ATLEAST the branch name or commit id (check after)
BRANCH_OR_SHA=$2
# check if branch_name or commit_sha
if [[ $BRANCH_OR_SHA =~ [0-9a-f]{5,40} ]]; then
# is COMMIT_HASH
COMMIT_SHA=$BRANCH_OR_SHA
BRANCH_OR_SHA="\"commitSha\": \"$COMMIT_SHA\""
else
# is BRANCH_NAME
BRANCH_OR_SHA="\"branchName\": \"$BRANCH_OR_SHA\""
fi
# This is the request we send to google so it knows what to build
# Here we're overriding some variables that we have already set in the default 'cloudbuild.yaml' file of the repo
cat <<EOF > request.json
{
"projectId": "$PROJECT_ID",
$BRANCH_OR_SHA,
"substitutions": {
"_MY_VAR_1": "my_value",
"_MY_VAR_2": "my_value_2"
}
}
EOF
# our curl post, we send 'request.json' with info, add our Token, and set the trigger_id
curl -X POST -T request.json -H "Authorization: Bearer $(gcloud config config-helper \
--format='value(credential.access_token)')" \
https://cloudbuild.googleapis.com/v1/projects/"$PROJECT_ID"/triggers/"$TRIGGER_ID":run
i'm using the following curl command to import the output.xml file into jira test execution key and receiving error as below. I'm sure the test execution key is existing in jira and the project id is also correct. Any pointers?
curl -H "Content-Type: multipart/form-data" -u userid:pass -F "file=#output.xml" "https://server/rest/raven/latest/import/execution/robot?projectKey=PROJKEY+and+testExecKey=TESTEXNKEY" -o error.txt
The error i receive is as below
The User "userid" does not have permission to create issues
Why does it try to create new issue while the issue already exists? And why does it say the user doesn't have access when the access is there?
You probably mean Xray add-on and you probably use the same request per their documentation. The problem seems to be with your parameter syntax. It should be .../robot/?projectKey=PROJKEY&testExecKey=TESTEXNKEY (i.e. & instead of +and+).
Plus I would explicitly specify it's a POST request: curl -X POST ....
But their error message is not clear, anyway. I don't have Xray available right now, but if you keep having troubles, I would recommend checking with their support.
My question is about cleaning up the "Environments" tab on a Github repository.
I previously deployed via Heroku, using automatic deployment from two separate Github branches (one for staging, one for production).
This created a tab "Environments" on the repository, in which both Heroku environments were shown - exactly as intended.
Once I started to dive into Heroku pipelines, I have now configured the app to be promoted to production from staging, so the production environment no longer auto-deploys from a branch.
The Environments tab on my Github repo has no way to remove the environment that I no longer use. I can't seem to find any place on Github or Heroku to make Github "forget" this deployment environment.
I hope my question is clear enough; if I can elaborate on anything, please let me know.
I made a little webpage/script too, to automate the process (I don't have Python installed, and I didn't see that someone else had already made a script), and this is online and putting your info will do the process automatically.
Stackblitz - Github Deployments deleter
Edit 18/07/2020: I copied the script from Stackblitz to a local snippet code in here too, just in case Stackblitz disappears:
// RECOMMENDED: Disconnect HEROKU from Github before doing this (though not strictly necessary, I think).
//See https://stackoverflow.com/a/61272173/6569950 for more info.
// PARAMETERS
const TOKEN = ""; // MUST BE `repo_deployments` authorized
const REPO = "your-repo"; // e.g. "monorepo"
const USER_OR_ORG = "your-name"; // e.g. "your-name"
// GLOBAL VARS
const URL = `https://api.github.com/repos/${USER_OR_ORG}/${REPO}/deployments`;
const AUTH_HEADER = `token ${TOKEN}`;
// UTILITY FUNCTIONS
const getAllDeployments = () =>
fetch(`${URL}`, {
headers: {
authorization: AUTH_HEADER
}
}).then(val => val.json());
const makeDeploymentInactive = id =>
fetch(`${URL}/${id}/statuses`, {
method: "POST",
body: JSON.stringify({
state: "inactive"
}),
headers: {
"Content-Type": "application/json",
Accept: "application/vnd.github.ant-man-preview+json",
authorization: AUTH_HEADER
}
}).then(() => id);
const deleteDeployment = id =>
fetch(`${URL}/${id}`, {
method: "DELETE",
headers: {
authorization: AUTH_HEADER
}
}).then(() => id);
// MAIN
getAllDeployments()
.catch(console.error)
.then(res => {
console.log(`${res.length} deployments found`);
return res;
})
.then(val => val.map(({
id
}) => id))
.then(ids => Promise.all(ids.map(id => makeDeploymentInactive(id))))
.then(res => {
console.log(`${res.length} deployments marked as "inactive"`);
return res;
})
.then(ids => Promise.all(ids.map(id => deleteDeployment(id))))
.then(res => {
console.log(`${res.length} deployments deleted`);
return res;
})
.then(finalResult => {
const appDiv = document.getElementById("app");
appDiv.innerHTML = `
<h1>CLEANUP RESULT</h1>
<br>
Removed Deployments: ${finalResult.length}
<br>
<br>Ids:<br>
${JSON.stringify(finalResult)}
<br><br><br><br><br><br>
<p>(Open up the console)</p>
`;
});
h1,
h2 {
font-family: Lato;
}
<div id="app">
<h1>Github Deployment's Cleaner</h1>
<p> You need to put the parameters in!</p>
</div>
There doesn't seem to be UI for it, but you can do it using the GitHub API.
You should probably disconnect GitHub and Heroku before doing this.
First, go to your GitHub account settings, then developer settings, then personal access tokens. Create a new token that has repo_deployments allowed. After it's generated, save the hexadecimal token, you'll need it for the upcoming API requests.
For these examples I'll assume that your username is $aaaa and your repo name is $bbbb and your access token is $tttt. Replace these with your actual username and repo name and access token. Or just use shell variables to store the actual values which will let you paste the code blocks directly.
First, list all the deployments on your repo:
curl https://api.github.com/repos/$aaaa/$bbbb/deployments
Each deployment has an id integer. Note it, and replace $iiii in the upcoming code blocks with that ID. Or create another shell variable for it.
Now you have to create an "inactive" status for that deployment:
curl https://api.github.com/repos/$aaaa/$bbbb/deployments/$iiii/statuses -X POST -d '{"state":"inactive"}' -H 'accept: application/vnd.github.ant-man-preview+json' -H "authorization: token $tttt"
And now you can delete the deployment forever:
curl https://api.github.com/repos/$aaaa/$bbbb/deployments/$iiii -X DELETE -H "authorization: token $tttt"
If you have multiple deployments, send the first request to see all the deployments that remain, and then you can delete those too if you want.
After you delete all the deployments, the environments button on the GitHub repo will disappear.
Information sourced from the GitHub deployments documentation and the GitHub oauth documentation. This worked for me.
This will not answer the OP question, I thought it would at first, but it didn't behave as I had expected. Therefore, I'm adding this answer as a community wiki.
GitHub seems to have two notions of "Environments", the one the OP means are "public environments", but GitHub also seems to have some kind of "private environments".
I'm adding my experience as an answer below because it is really confusing.
You can access "private environments" through "Settings > Environments". (E.g: https://github.com/UnlyEd/next-right-now/settings/environments)
You can then delete each environment. It'll prompt a confirm dialog. Upon confirm, the environment will be destroyed.
I deleted the "staging" and "production" environments.
But the public environments still continue to exist, alongside all their deployments. (and this is not what the OP wants)
Public environments still contains "staging" and "production".
I've made an interactive python script which can delete specific environments by name (which was my issue) or all of your deployments. Check it out and let me know if it works for you: https://github.com/VishalRamesh50/Github-Environment-Cleaner.
This will also actually delete all of your deployments even if you have more than 30 unlike the other scripts here because it goes through the paginated response data from Github's API and doesn't just use the first page.
Based on Cadence's answer, I built the following bash script. Just set the appropriate parameters and let it run.
The token requires repo_deployment OAuth scope.
env=asd
token=asd
repo=asd
user=asd
for id in $(curl -u $user:$token https://api.github.com/repos/$user/$repo/deployments\?environment\=$env | jq ".[].id"); do
curl -X POST -u $user:$token -d '{"state":"inactive"}' -H 'accept: application/vnd.github.ant-man-preview+json' https://api.github.com/repos/$user/$repo/deployments/$id/statuses
curl -X DELETE -u $user:$token https://api.github.com/repos/$user/$repo/deployments/$id
done
I Expect Your already got the answer for your question This answer is for those who wan't to Remove The Environments Tab from There GitHib Repo
As I Understand
If You Want to Remove the Environments Tab Shown on The Right Side of The Screen
If You Want to Remove Specific Environments
Remove Environments Tab
To Remove Entire Tab
On Your Repo Page Click on Settings Button
Uncheck the Mark on Environments
It Will Remove The Environments Tab from the Right Side
Remove Specific Environments
For Removing Specific Environment
Click on SettingsClick On EnvironmentsHere You Can Remove a Specific Environment
Reply from GitHub for Removing of Environments from Private Repo
Hi Deekshith,
Thanks for writing in!
As it turns out, the Environments feature is an enterprise-only feature for private repositories, however, they still get created as a side effect of Deployments in private repositories.
https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment
This means that though you'd be able to create Deployments in a private repository, you won't be able to manage (e.g delete, add rules e.t.c) the deployment Environments. I'm afraid this is currently the expected behaviour. Our engineers are still discussing how best to help users who'd want to delete environments in non-enterprise private repositories, such as in your case. We’ll be sure to update you as soon as there’s any news to share.
Sorry that we could not be of more help with this -- please let us know if you have any questions at all!
Regards,
Peter
GitHub Support
I've built an online tool to help out removing deployments
I ran into the same problem and found #spersico's code very handy, but needed a bit more tooling/feedback. I iterated on #spersico code to add a bit of frontend.
Same as #spersico's version, all calls are made client side (and are visible in your console/network logs).
Project is opensource on Github and has a hosted version on Netlify which can be used instantly:
https://github-deployment-cleaner.netlify.app/
Unfortunately, it seems that the 'Deployments' dashboard is currently in beta, which means that they may not have a feature yet.
Read here.
I've just used this Python script: 5 minutes and I removed my unwanted environment:
https://github.com/VishalRamesh50/Github-Environment-Cleaner
There is a discussion in GitHub Community: https://github.community/t/how-to-remove-the-environment-tab/10584/10?u=aerendir
Please, vote up the feature.
Using GitHub CLI:
org=':org:'
repo=':repo:'
env=':env:'
gh api "repos/${org}/${repo}/deployments?environment=${env}" \
| jq -r ".[].id" \
| xargs -n 1 -I % sh -c "
gh api -X POST -F state=inactive repos/${org}/${repo}/deployments/%/statuses
gh api -X DELETE repos/${org}/${repo}/deployments/%
"
I don't know if this was posted already but I definitely didn't want do delete my 40+ deployments manually so I created following script, feel free to use it too :)
#!/bin/bash
REPO=<your GH name>/<your project name>
TOKEN=<your personal access token>
# https://starkandwayne.com/blog/bash-for-loop-over-json-array-using-jq/
for deployment in $(curl https://api.github.com/repos/$REPO/deployments | jq -r '.[] | #base64'); do
DEPLOYMENT_ID=$(echo "$deployment" | base64 --decode | jq -r '.id')
echo "$DEPLOYMENT_ID"
curl "https://api.github.com/repos/$REPO/deployments/$DEPLOYMENT_ID/statuses" \
-X POST \
-d '{"state":"inactive"}' \
-H 'accept: application/vnd.github.ant-man-preview+json' \
-H "authorization: token $TOKEN"
done
for deployment in $(curl https://api.github.com/repos/$REPO/deployments | jq -r '.[] | #base64'); do
DEPLOYMENT_ID=$(echo "$deployment" | base64 --decode | jq -r '.id')
curl "https://api.github.com/repos/$REPO/deployments/$DEPLOYMENT_ID" \
-X DELETE \
-H "authorization: token $TOKEN"
done
You can remove the environment from a repository if its public. But in case of private repositories, either you have to make it public or use the github API. Both works, but here is my approach for deleting the environments.
I created an npm package (here) for the same. Just get the github access token, with repo_deployments scope enabled.
Now run npx delete-github-environment and select the environment that you want to delete. If everything went right, your environment will be deleted.
PS: Here's my github repo - (github), feel free to contribute to the code.
To remove a Github Environment go to Settings -> Environments -> and click the trashcan icon next to the Environment you want to delete (see picture below).
More can be read about it from the Github official documentation:
Deleting an Environment
I am running bitbucket server v4.14. I want to be able to get diff of any two commits for a repo. I saw this posted at https://bitbucket.org/site/master/issues/4779/ability-to-diff-between-any-two-commits
However the same does not work, probably cause the version I am running is older. The compare page directly takes me to diff across branches. I do not want any UI element, just a URL is fine.
Thanks
This feature is not available to Bitbucket 4.14.
You can get this data using the REST API. Try to execute the following command:
curl -s --user USER:PASS --request GET https://BITBUCKET-SERVER/rest/api/1.0/projects/PROJECT/repos/REPOSITORY/commits?since=SINCE-COMMIT\&until=UNTIL-COMMIT | jq --raw-output '.values[] | .displayId+ " " + .author.name'
AFAIK there is only rest api endpoint. There is a since parameter that you can pass. See the docs for more details.