I'm working on project, where I have two Git Repos:
Repo1 - DevRepo, Rep02- TestRepo
My Scenario is:
Whenever a commit or a PR happens on the Repo1:
Step1: Immediately Repo2 should be triggered
Step2: Once Step1 is success, Repo1 should be triggered.
Basically Repo1 should build only if Repo2 is run and it turns success.
Could someone please help me how I can set this up, much appreciated:
which .travis.yml file should I configure to meet my scenario.
exact configuration steps that I can write in my .travis.yml file
Answer: I got this working:
Trigger your dependent build: Repo2 by using the travis api:
Create a trigger_build.sh file and add this code:
`
body='{
"request": {
"branch":"master"
}}'
curl -s -X POST \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-H "Travis-API-Version: 3" \
-H "Authorization: token ..git_hub_login_token.." \
-d "$body" \
https://api.travis-ci.org/repo/xxxx%2Fyyyy/requests
#The 15s sleep is to allow Travis to trigger the dependent build:
sleep 15`
Create a new or a separate get_build_status.sh file. Poll for the status of the dependent build. Based on the status of the Repo2 build, we will either continue building Repo1 or stop building Repo1:
# Polling for the Repo2 build status
# Setting a maximum time for the Repo2 build to run, however once we get the status to passed/failed, we will return to the Repo1 build run
`i=1
max=300
while [ $i -lt $max ]
do
echo "--------------------------------------------"
echo "Polling for the tests run build status..."`
curl -i -H "Accept: application/vnd.travis-ci.2+json" "https://api.travis-ci.org/repos/xxxx/yyyy/builds" > test.json
LATEST_STATE=$(grep -o '"state":.[a-z\"]*' test.json | head -1)
#LATEST_ID=$(grep -o '"id":.[0-9]*' test.json | head -1 | grep ':.[0-9]*')
get_state_value=${LATEST_STATE#*:}
STATE="${get_state_value//\"}"
if [ $STATE == "passed" ]
then
echo "TESTS RUN... $STATE :-) "
break #As soon as the Repo2 run pass, we break and return back to the Repo1 build run
elif [ $STATE == "failed" ]
then
echo "TESTS RUN... $STATE :-("
echo "Stop building elements"
exit 1 #As soon as the Repo2 run fail, we stop building Repo1
fi
true $(( i++ ))
sleep 1 #This 1s is required to poll the build status for every second
done
Your .travis.yml configuration:
script:
- chmod 777 ./trigger_build.sh
- chmod 777 ./get_build_status.sh
- ./trigger_build.sh
- ./get_build_status.sh
I've now built a CI service for this purpose. It lives here: https://github.com/cesium-ml/dependent_build_server
The above code basically does the following:
Wait for GitHub to let it know when a PR has changed
Use the Travis-CI API to trigger a build of the project, and insert two environment variables to describe which version of the code to build.
Listen for Travis-CI to report back, via webhook, the result of the build.
Use the GitHub API to report the new status to the PR.
Note: You have to verify the payloads that come from GitHub and Travis-CI. This is trivial for GitHub, but slightly harder for Travis-CI which uses proper key signing.
GitHub:
import hmac
def verify_signature(payload, signature, secret):
expected = 'sha1=' + hmac.new(secret.encode('ascii'),
payload, 'sha1').hexdigest()
return hmac.compare_digest(signature, expected)
Travis-CI:
from OpenSSL import crypto
def verify_signature(payload, signature):
# Get this by querying the Travis-CI config API
public_key = crypto.load_publickey(crypto.FILETYPE_PEM, pubkey)
certificate = crypto.X509()
certificate.set_pubkey(public_key)
try:
crypto.verify(certificate, signature, payload, 'sha1')
return True
except crypto.Error:
return False
Related
I created a workflow file and the workflow is not showing up in the GitHub actions tab
name: AZ Deploy Workflow
on:
workflow_dispatch:
inputs:
deploy-environment:
description: 'Environment to deploy to'
required: true
default: 'dev'
image-tag:
description: 'Docker tag to deploy'
required: true
default: 'latest'
any ideas what's the issue.
Earlier I have added other workflow into the .github directory that shows in actions with no issue even if a put a empty file it does but not this one
Make sure your workflow is located in the default branch.
To run a workflow manually, the workflow must be configured to run on the workflow_dispatch event. To trigger the workflow_dispatch event, your workflow must be in the default branch. For more information about configuring the workflow_dispatch event, see "Events that trigger workflows".
For more details, see the Manually running a workflow and about the default branch.
UPD:
Testing it from a branch other than from the main branch can be done using the GitHub REST API:
curl \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer <YOUR-TOKEN>"\
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/OWNER/REPO/actions/workflows/WORKFLOW_ID/dispatches \
-d '{"ref":"test-branch","inputs":{"deploy-environment":"dev","image-tag":"latest"}}'
You can replace WORKFLOW_ID with the workflow file name. For example, you could use main.yaml.
ref - your branch name.
For more details visit the official docs: Create a workflow dispatch event.
In Github Actions is the following scenario possible
WORKFLOW1 workflow1.yaml ---> output x --> WORKFLOW2 workflow2.yaml
WORKFLOW1 Calculates a value x WORKFLOW2 Uses the calculated value x e.g (a +x) = total
which is output from previous WORKFLOW1
P.S The first run workflow1 should not be called again as the value of x may change.
Any tips would be highly appreciated
Thanks
I have tried reusable workflows but it seems to run the workflow which calculates a new value of x and runs the entire workflow 1 again which we don't want we just need the output of wf1 to be used in wf2
Thanks
An option could be to call the workflow 2 from the workflow 1 using a dispatch event.
The workflow 2 trigger implementation would look like this:
on:
workflow_dispatch:
inputs:
input_name:
description: "description"
required: true
You could send a dispatch event from the workflow 1 by using the Github API:
On the official Github Documentation, there is a service to create a workflow dispatch event
Here is a curl example:
- run: |
curl \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer <YOUR-TOKEN>"\ # if needed
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/OWNER/REPO/actions/workflows/WORKFLOW_ID/dispatches \
-d '{"ref":"main","inputs":{"input_name":"input_value"}}'
You can also find more references about this in this article.
Is there a way to use the github-cli or api to view the inputs of an action while it is running?
I want to allow Github actions to run concurrently. The resources they will manage are determined by the input stack_name. I want to make sure two pipelines cannot run at the same time with the same stack_name input. If this happens, then I want one of the pipeline actions to fail and stop immediately.
I am also taking the input and turning it into an environmental variable for one of my jobs. After the job finishes, the values are available in the logs and I can grep through the following output to get a pipelines stack_name:
$ gh run view $running_pipeline_id --repo=$GITHUB_SERVER_URL/$GITHUB_REPOSITORY --log
....
env-check env-check 2022-03-22T17:06:30.2615395Z STACK_NAME: foo
However, this is not available while a job is running and I instead get this error:
run 1234567890 is still in progress; logs will be available when it is complete
Here is my current attempt at a code block that can achieve this. I could also use suggestions on how to make better gh run list and/or gh run view calls that can avoid using grep and awk. Clean json output I can parse with jq is preferable.
set +e
running_pipeline_ids=$(gh run list --workflow=$SLEEVE --repo=$GITHUB_SERVER_URL/$GITHUB_REPOSITORY \
| grep 'in_progress' \
| awk '{print $((NF-2))}' \
| grep -v $GITHUB_RUN_ID)
set -e
for running_pipeline_id in $running_pipeline_ids; do
# get the stack name for all other running pipelines
running_pipeline_stack_name=$(gh run view $running_pipeline_id --repo=$GITHUB_SERVER_URL/$GITHUB_REPOSITORY --log \
| grep 'STACK_NAME:' | head -n 1 \
| awk -F "STACK_NAME:" '{print $2}' | awk '{print $1}')
# fail if we detect another pipeline running against the same stack
if [ "$running_pipeline_stack_name" == "$STACK_NAME" ]; then
echo "ERROR: concurrent pipeline detected. $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$running_pipeline_id"
echo "Please try again after the running pipeline has completed."
exit 1
fi
done
Perhaps you could use the concurrency feature of GitHub Actions?
Now you cannot directly bake this into an action, but if it's possible for you to extract your action into a reusable workflow then you could make use of the concurrency feature.
It would look something like this:
# ./github/workflows/partial.yaml
on:
workflow_call:
inputs:
stack-name:
description: "name of the stack"
required: true
type: string
jobs:
greet:
runs-on: ubuntu-latest
concurrency:
group: ${{ inputs.stack-name }}
cancel-in-progress: true
steps:
- uses: my/other-action
with:
stack_name: ${{ inputs.stack-name }}
And then where you're using it:
jobs:
test:
uses: my/app-repo/.github/workflows/partial.yml#main
with:
stack-name: 'my-stack'
I am trying to trigger the workflow_dispatch action for a GitHub workflow via REST but I am getting a "not found" error.
My question is similar to this one but the difference is that I am still getting the "not found" error even though the header indicates I am authenticated (the rate limit has increased to 5,000).
Here's my script:
#!/bin/bash
# https://docs.github.com/en/rest/reference/actions#create-a-workflow-dispatch-event
OWNER='myGithubOrganization'
REPO='myRepo'
WORKFLOW_ID='main.yml'
POST_URL="https://api.github.com/repos/$OWNER/$REPO/actions/workflows/$WORKFLOW_ID/dispatches"
echo "Calling $POST_URL"
GITHUB_PERSONAL_ACCESS_TOKEN=$(echo "$PLATFORM_VARIABLES" | base64 --decode | jq '.GITHUB_PERSONAL_ACCESS_TOKEN' --raw-output)
# -i to include headers.
curl \
-i \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $GITHUB_PERSONAL_ACCESS_TOKEN" \
$POST_URL \
-d '{"ref":"ref"}'
In the headers, I see the rate limit has increased to 5,000, so I know I am logged in.
The personal access token has the following permissions:
repo
workflow
admin:org_hook
The personal access token is for a machine user.
In the repo settings, under "Collaborators and teams", the machine user account has the "Read" role.
What more do I need to do to trigger the workflow?
The machine user needs to have write access, not read access.
This is true even if the workflow does something like run CI tests and does not write any code.
I have written a shell script to create a pull request with Github Enterprise API.
#!/bin/sh
CURRENT_BRANCH=$(git branch --show-current)
git stash save
git checkout master
git pull
UUID=$(uuidgen)
git checkout -b "${UUID}"
npm version patch
git add package.json
git commit -m "#; update version script test"
git push origin "${UUID}"
curl -u [MY_USER_NAME]:[MY_TOKEN] https://github.[MY_ORGANIZATION].com/api/v3/user
curl \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
https://github.[MY_ORGANIZATION].com/api/v3/repos/Modules/[MY_REPO]/pulls \
-d "{'head':'${UUID}','base':'master'}"
The log is displayed as below.
It fail to create the pull request with the error
"message": "Must authenticate to access this API."
However, there seems no issue with the authentication.
// omit some useless log here
remote:
To github.microstrategy.com:Modules/mstr-web-hierarchy.git
* [new branch] 23CFA0E3-33D1-489E-9E55-D38F92BB1B99 -> 23CFA0E3-33D1-489E-9E55-D38F92BB1B99
{
"login": "shizhang",
"id": 1191,
// some useless properties here
{
"message": "Must authenticate to access this API.",
"documentation_url": "https://docs.github.com/enterprise/3.0/rest"
}
Your second curl does not seem to include any authentication information.
Try adding the same -u [MY_USER_NAME]:[MY_TOKEN] as you used in the first curl.
In other words, the first curl success does not mean other curls would benefit from an authenticated session.
Problems parsing JSON
Try with a simpler JSON, just for testing.
And pay attention to quotes: '${UUID}' would not be expanded, and would remain exactly as '${UUID}'.
See "How to include environment variable in bash line CURL?"