How to create a release tag using .gitlab-ci.yml file - tags

I neet to run jobB only when jobA passes. I have to create a release tag only when my test stage passes successfully (I have added a code for that to happen here). My repository has 'README.md' file already. I am just checking its existence in my test stage. So, my test stage will always pass. Please let me know how do I write a code to create a release tag. A tag can be for example v1.1
stages:
- build
- test
- release
jobA:
stage: test
script:
- test -e README.md && exit 0
jobB:
stage: release
when: on_success
script:
# code for creating a release tag

In addition of Greg Woz's answer, in which GITLAB_API_TOKEN (a PAT -- PErsonal Access Token) is used, you now can also use a dedicated private project token:
See GitLab 13.9 (February 2021)
Support PRIVATE-TOKEN to create releases using the Release-CLI
In this milestone, we added the ability to use the release-cli with a PRIVATE-TOKEN as defined in the Create a release API documentation.
This enables overriding the user that creates a release, and supports automation by allowing connection of a project-level PRIVATE-TOKEN or by using a bot user account’s PRIVATE-TOKEN.
See Documentation and Issue.

It may not answer fully for your question but hopefully it helps. Perhaps try the below. My solution assumes you have package.json with version as it's the most popular case - but there is nothing wrong if you use any other way to define version:
version:
stage: version
only:
- master
script:
- VERSION_NUMBER=$(node -p "require('./package.json').version")
- "curl --header \"PRIVATE-TOKEN: $GITLAB_API_TOKEN\" --request POST \"https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/repository/tags?tag_name=v${VERSION_NUMBER}&ref=$CI_COMMIT_SHA&message=Tagged%20with%20version%20v${VERSION_NUMBER}\""
Then you can use:
publish:
stage: publish
only:
- tags
script:
- npm version --no-git-tag-version ${CI_COMMIT_TAG:1}
- npm publish --access public
for example in a next step.

Gitlab 13.2 introduced release keyword in .gitlab-ci.yml. There's no need anymore to create a "jobB" job, you can add keyword to main job as long as it can access release-cli:
stages:
- build
- test
- release
jobA:
stage: test
image: registry.gitlab.com/gitlab-org/release-cli:latest
script:
- test -e README.md && exit 0
release:
tag_name: $TAG
description: './path/to/CHANGELOG.md'
GitLab Docs has some examples to dynamically provide $TAG and description.

Probably the dependencies mechanism is a right tool for doing that. So, you can use README.MD (or test results folder) as an artifact in jobA and add the dependency on jobA in the jobB
jobB:
....
dependencies:
- jobA
If the jobA fail it won't provide dependent artifact for jobB. Thus, it also will fail.
(Edit)
Have you checked Protected tags gitlab settings? (Settings -> Repository -> Protected tags)
Default settings are:
By default, protected tags are designed to:
Prevent tag creation by everybody except Maintainers
Prevent anyone from updating the tag
Prevent anyone from deleting the tag

Related

How to share Azure Devops pipeline between multiple repos?

So, I have the following situation:
20 git repositories with a microservice in each
A repo with a template pipeline for the standard build process
Each of the 20 repos defines its own pipeline that uses this template with some parameters
On a PR build for any of the 20 repos, it will run its own pipeline as a build validation.
That's all working fine.
But now, I want to add an additional Optional Check to each of the 20 repos which would run a code analysis tool (eg. sonarqube) as part of the PR.
I don't want to add this to the main pipeline as I want it to appear in the PR as a separate optional check which can be skipped or toggled between optional/required.
The only way that I can find to achieve this is to add a CodeAnalysis.yml to each of the 20 repos and create 20 associated pipelines, which is an overhead I'd rather not deal with.
Is there a way that you can have a single pipeline that can be referenced as an optional check in all of these repos?
According to the docs, it should be possible for the shared pipeline to dynamically fetch the code from the right repo using something like this:
- checkout: git://ProjectName/$(Build.Repository.Name)#$(Build.SourceBranch)
But when I try this, the PR is unable to queue the pipeline (unhelpfully, it doesn't give a reason why).
Is there a solution to this?
You need to use Templates to design a shared template to run the code scanning. Essentially templates are reusable yaml files that you can pass parameters to to customise options.
For example, for your use case you could have the template for code scanning and add an existing job onto your pipelines to extend this template, pass any optional parameters you need (such as the repo to check out) and you can add in conditions to decide when to run the code scanning check
I know what you mean, this is not possible (at least not in the PR interface). Because when you press 'Queue' button in the PR build section, there won't even be a popup to select parameters, it will just choose the default value.
- checkout: git://ProjectName/$(Build.Repository.Name)#$(Build.SourceBranch)
This is also not possible, because runtime variables are not accepted here, they will be read directly as string types.
One suggestion is that you can specify parameters manually in pipeline page and then run the pipeline after setting the parameters
The reason for this is:
1, The checkout section is expanded before everything happens in pipeline run.
2, The Queue button on PR page didn't provide pop-out window to select parameters.
Your pipeline definition should be like this:
trigger:
- none
parameters:
- name: ProjectName
default: xxx
- name: RepoName
default: xxx
- name: BranchName
default: xxx
pool:
vmImage: windows-latest
steps:
- checkout: git://${{parameters.ProjectName}}/${{parameters.RepoName}}#${{parameters.BranchName}}
- script: |
dir
displayName: 'Run a multi-line script'
Manually select the parameters every time:

With yaml pipelines, is there a way to select an environment parameter from a dynamic list of all environments?

We've been migrating some of our manual deployment processes from Octopus to Azure DevOps Yaml pipelines. One of the QoL changes we're sorely missing is to be able to select the environment from a drop-down list/ auto-complete field as we could in Octopus.
Is there a way to achieve this? Currently, the only way I can think of doing it is to have a repo with a .yaml template file updated with a list of new environments as part of our provisioning process... Which seems less than ideal.
If you are going to trigger the pipeline manually then you can make use of Runtime parameters in the Azure DevOps pipeline.
For Example:
In order to make OS image name selectable from a list of choices, you can use the following snippet.
parameters:
- name: EnvName
displayName: EnvName
type: string
default: A
values:
- A
- B
- C
- D
- E
- F
trigger: none # trigger is explicitly set to none
jobs:
- job: build
displayName: build
steps:
- script: echo building $(Build.BuildNumber) with ${{ parameters.EnvName }}
Documentation about runtime parameters are here.
The downside to this is that the trigger: None limits you that the pipeline can only be manually triggered. Not sure how this works with other trigger options.

Approval & Checks on Azure Repos Git are ignored

I placed a checks configuration on my git repository.
Then I added a yaml pipeline to the same directory.
If I run the pipeline all checks or approvals are ignored independent of the type as if they don't exist.
In the log of the job I can see that the repositoy is accessed (default checkout of self).
Do checks on resources with type repository get only executed under special circumstances?
Am I missing something?
I want to force the pipeline to extend a dedicated template regardless which environment is used.
here the pipeline source, that I expected to fail:
trigger:
- main
- users/*/*
- features/*
pool:
vmImage: ubuntu-latest
steps:
- bash: echo "hello world"
displayName: echo something
I checked the same checks on the agent pool and that works as expected. It fails the pipeline if that does not extend the template.
template check configuration

How to download build artifact from other versions (runs) published build artifact?

My pipeline publishes two different build artifacts when all its tests have passed - stage: publish_pipeline_as_build.
One of my tests needs to use the build that was made in the current run, of the current version.
But additionally, I need to get the build artifact of the previous version, in order to run some compatibility tests.
How do I download the build artifact from that other pipeline run?
I know the build artifact name (from runtime script), but how would I find that?
I tried playing around with azure-cli az pipelines runs artifact list. It requies a --run-id and actually my script won’t have that.
So far I kind of managed, assuming the response of az pipelines runs list retuns the latest match to the query first:
az pipelines runs list --project PROJNAME --query "[?sourceBranch=='refs/heads/releases/R21.3.0.2']" | jq '.[0]'
I currently seem to run out of Ideas.
Perhaps just some confused/frustrated questions that pop up:
How can I find that specific build artifact name's latest version and download it?
How are pipeline tasks fed with runtime generated values?
Is this so ridiculously difficult when doing it in Azure DevOps, or am I just going the wrong way?
The job I'm trying to get there with:
jobs:
- job: test_session_integration
dependsOn: easysales_Build
steps:
- template: ./utils/cache_yarn_and_install.yml
- template: ./utils/update_webdriver.yml
- template: ./utils/download_artifact.yml
parameters:
artifact: easysales_$(Build.BuildId)_build
path: $(System.DefaultWorkingDirectory)/dist
# current release name as output
- template: ./utils/get_release_name.yml
# previous release name, branch and build name output
- template: ./utils/get_prev_release.yml
# clone prev version manually - can't use output variables as task input
# (BTW: why? that is super inconvenient, is there really no way?)
- bash: |
git clone --depth 1 -b $(get_prev_release.BRANCH_NAME) \
"https://${REPO_USERNAME}:${REPO_TOKEN}#dev.azure.com/organisation/PROJECTNAME/_git/frontend-app" \
./reference
workingDirectory: $(System.DefaultWorkingDirectory)
env:
REPO_TOKEN: $(GIT_AUTH_TOKEN)
REPO_USERNAME: $(GIT_AUTH_USERNAME)
name: clone_reference_branch
Any clues?
I'd be glad for any rubber ducking hints or clues on how I would be able to achieve what I need.
I'm new to Azure DevOps and currently struggle to find orientation in the vast but also quite in many places bits and pieces documentation Microsoft offers to me. It's fine, but frankly I struggle quite a bit with it. Is it just me having this problem?
All stages and full YAML on pastebin
The main template with the stages (Expanded templates made with "download full YAML"):
stages:
- stage: install_prepare
displayName: install & prepare
jobs:
- template: az_templates/install_hls_lib_build_job.yml
- stage: test_and_build
displayName: test and build projects
dependsOn: install_prepare
jobs:
- template: az_templates/build_projects_jobs.yml
- template: az_templates/test_session_integration_job.yml
- stage: publish_pipeline_as_build
displayName: Publish finished project artifacts as builds
dependsOn: test_and_build
jobs:
- template: az_templates/build_artifact_publish_jobs.yml
I by now found a solution. Perhaps not a definitive one, but should sort of work:
In my library variable group I added a azure devops personal access token with read access to the necessary groups as ADO_PAT_TOKEN
I Sign in with azure-cli with that token
I get the latest run id with azure cli
- bash: |
az devops configure --defaults organization=$(System.CollectionUri)
az devops configure --defaults project=$(System.TeamProject)
echo "$AZ_DO_TOKEN" | az devops login
AZ_QUERY="[?sourceBranch=='refs/heads/$(prev_release.BRANCH_NAME)'] | [0].id"
ID=$(az pipelines runs list --query-order FinishTimeDesc --query "$AZ_QUERY")
echo "##vso[task.setvariable variable=ID;isOutput=true]$ID"
env:
AZ_DO_TOKEN: $(ADO_PAT_TOKEN)
name: prev_build_run
I then download the artifact with azure-cli and the queried run id
- bash: |
az pipelines runs artifact download \
--artifact-name 'easysales_$(prev_release.PREV_RELEASE_VERSION)' \
--run-id $(prev_build_run.ID) \
--path '$(System.DefaultWorkingDirectory)/reference/dist/easySales'
workingDirectory: $(System.DefaultWorkingDirectory)
name: download_prev_release_build_artifact
This roughly seems to work for me now... finally 😉
Missing
The personal access token I added to the secrets may work, but AFAIS these tokens can not be created with no expiry date further than one year in the future.
That is not ideal, since I don't want my pipeline to stop working and perhaps no one around knows how to fix this.
Perhaps someone knows how I can use azure CLI within the current pipeline without authentication?
Given it accesses only the current organization and project?
Or does anyone see a more elegant solution to my admittedly clumsy solution.

How to version build artifacts using GitHub Actions?

My use case is I want to have a unique version number for artifacts per each build/run. With current tools like CircleCI, Travis, etc. there is a build number available which is basically a counter that always goes up. So, I can create version strings like 0.1.0-27. This counter is increased each time even for the same commit.
How can I do something similar with GitHub Actions? Github actions only offer GITHUB_SHA and GITHUB_REF.
GitHub Actions now has a unique number and ID for a run/build in the github context.
github.run_id : A unique number for each workflow run within a repository. This number does not change if you re-run the workflow run.
github.run_number : A unique number for each run of a particular workflow in a repository. This number begins at 1 for the workflow's first run, and increments with each new run. This number does not change if you re-run the workflow run.
github.run_attempt : A unique number for each attempt of a particular workflow run in a repository. This number begins at 1 for the workflow run's first attempt, and increments with each re-run.
ref: https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#github-context
You can reference them in workflows like this:
- name: Output Run ID
run: echo ${{ github.run_id }}
- name: Output Run Number
run: echo ${{ github.run_number }}
- name: Output Run Attempt
run: echo ${{ github.run_attempt }}
I had the same problem and have just created an action to generate sequential build numbers. Use it like
- uses: einaregilsson/build-number#v1
with:
token: ${{secrets.github_token}}
In steps after that you'll have a BUILD_NUMBER environment variable. See more info about using the same build number for different jobs and more at https://github.com/einaregilsson/build-number/
UPDATE: There is now a $GITHUB_RUN_NUMBER variable built into GitHub Actions, so this approach is not needed anymore.
If you want a constant integer increment (1,2,3,4,5), I haven't found anything in the docs that you could use as such increment which is aware of how many times that particular action ran. There's two solutions I can think of:
Maintaining state on the repo: for example with a count.build file that uses the workflow ID and you increment it on build. This is my least favourite solution of the two because it adds other complexities, like it will itself trigger a push event. You could store this file somewhere else like S3 or in a Gist.
Using the Date: if you're not worried about sequence on the integer increment you could just use the current data and time, for example 0.1.0-201903031310 for Today at 13:10.
Regardless if you have Actions Beta Access, I would definitely feed this back to GitHub.
Hope it helps.
You can use GitVersion to generate incrementing versions from tags in Git. The PR at https://github.com/GitTools/GitVersion/pull/1787 has some details, but basically you can define this job:
- uses: actions/checkout#v1
- name: Get Git Version
uses: docker://gittools/gitversion:5.0.2-beta1-34-linux-debian-9-netcoreapp2.1
with:
args: /github/workspace /nofetch /exec /bin/sh /execargs "-c \"echo $GitVersion_MajorMinorPatch > /github/workspace/version.txt\""