Ive got a requirement to update the bitbucket pipelines yaml file by adding a task definition to the pipelines.branches.develop set of steps (adding it to be the first step, while keeping any existing steps) and if the develop branch pipeline doesn't exist, create it with the specified task definition as a step.
yq command im using the add to the develop pipeline is below:
yq '.pipelines.branches.develop = [{"step":"*My-Step"}] + .pipelines.branches.develop' bitbucket-pipelines.yml
This works but its enclosing the '*My-Step' in quotes and I cant work out how to remove them without causing the command to fail.
I have the same problem with the '&' character for my addition to the definition steps.
Could someone provide some guidance, or documentation.
Branch "1850" was merged into the branch "dev". And pipeline trigger when there is merge to branch "dev". When release is success or fail, there is a stage will send email to let user know the status of realease. And from email i cant tell which one it is. So how can i get the branch name of "1850" variable.
Update to Leo Liu-MSFT's answer:
Thanks for helping. so i make change and these error occurs. please explain why so.
How can i get the branch name that is merged into target branch that trigger the pipeline?
You could use the predefined variables System.PullRequest.SourceBranch, which return the branch that is being reviewed in a pull request.
As test, I add a inline powershell task to output this variable:
Write-Host "The merged target branch is:-" $(System.PullRequest.SourceBranch)
Then we could get the value is refs/heads/1805.
We could use a command line task to parse that path to get the branch name:
set BranchName=$(System.PullRequest.SourceBranch)
for %%a in (%BranchName%) do echo %%~nxa
Update:
As I test, it works fine on my side with following code:
Write-Host "##vso[task.setvariable variable=SourceBranchName;]$(System.PullRequest.SourceBranch)"
Remove the isOutput=true.
Then, I could get the variable SourceBranchName by the command line task:
echo $(SourceBranchName)
According to your error, it seems the error comes from your powershell task itself or the agent. So, please try to use hosted agent with Inline powershell task to check if you still have this issue.
Hope this helps.
I am finding using AZDO Release pipeline variables maddening in Powershell steps.
I am running an Azure PowerShell step to return a primary key value. It is 2 lines…
$primarykey = (Get-AzRelayKey -ResourceGroupName ${env:az-resourcegroupname} -Namespace ${env:az-relaynamespace} -HybridConnection ${env:serviceBus.primaryRelay.ConnectionName} -Name ${env:serviceBus.primaryRelay.KeyName} | Select-Object -ExpandProperty PrimaryKey)
Write-Host "##vso[task.setvariable variable=serviceBus.primaryRelay.Key]$primarykey"
In my pipeline I have a mix of variable names, some I have complete control over (the az- prefixed ones) and others I don’t (the ones starting serviceBus.)
The reason I have no control over the latter is that they are used for a later File Transform step that navigates an appsettings.json file to find/replace values, and its unable to be changed (for example serviceBus.primaryRelay.ConnectionName is a value that is changed in the JSON and the file transform step specifies to navigate the JSON structure, it has to be separated with a period . )
When this script runs it always complains about the -HybridConnection value being empty. This is because the variable has a period in it.
I’ve tried everything I can think of to retrieve that value in the code.
Are they suggesting here that a variable with a period isn’t workable in Powershell in AZDO release pipelines? I’m completely lost.
I have found the answer by looking under the Release Pipelines "Initialize Job" log. It appears to substitute the period . with a dash -
The log revealed this...
[SERVICEBUS_PRIMARYRELAY_CONNECTIONNAME] --> [dev-sbrelay]
I am using team city with git. I am trying to take different action on the basis of the branch name from build is getting triggered. Build gets triggered from a branch in which a change gets checked in.
I have branch name like US12345, F12345, DE12345 and I want to perform different tests on these different builds.
For this I have added a command line step and try to run a custom script (Windwos batch script). Here is the script:
#ECHO OFF
echo "BRANCH Name: %teamcity.build.branch%"
SET BranchName="%teamcity.build.branch%"
if /I %%BranchName:~0,1%%==F (echo Working on a feature branch && <take action 1>)
if /I %%BranchName:~0,2%%==US (echo Working on a story branch && <take action 2>)
if /I %%BranchName:~0,2%%==DE (echo Working on a defect branch && <take action 3>)
if /I %%BranchName:~0,2%%==QA (echo Working on a QA1 branch && <take action 4>)
In the above script, first I am assigning value of current branch name to a varialbe "BranchName".
Then in the first if condition, I am taking out first character of that variable and check if that character is equal to F. If that is true then I want to perform some set to tasks.
In the second if condition, I am taking out first two characters of "BranchName" and check if it is equal to US? If it is then I perform another different set of actions.
In the same way I want to do in third and fourth if condition.
Now the problem is, I am not able to get first few characters of a variable in the custom script because teamcity treats everything inside a "%%" as a reference parameter and we have to add those parameters in the build configuration.
Has anyone worked in these use case? Any help would be greatly appreciated.
If you want to pass % to TeamCity, you should escape it with other %, i.e. for % it must be %%, for %% it must be %%%%
You are cramming all your different branch builds into a single configuration, then using a bat file to trigger what gets build...that sounds complicated, and also a good choice for VSC trigger in TeamCity. I can a suited approach:
Create one build configuration, then create a VCS trigger for your desired branch name to trigger the build.
Copy that configuration N-times, one for each branch you want to build, then reconfigure the VSC trigger to have the correct branch name.
If you want to keep doing what you are doing, use PoSh instead of BAT script. Add a PowerShell step, change the script to be "Source Code" and enter this under "Script source":
$branch = "%teamcity.build.branch%"
if ($branch.StartsWith('F')) {
echo "Working on a feature branch $branch"
}
elseif ($branch.StartsWith('US')) {
echo "Working on a feature branch $branch"
}
elseif ($branch.StartsWith('DE')) {
echo "Working on a feature branch $branch"
}
elseif ($branch.StartsWith('QA')) {
echo "Working on a feature branch $branch"
}
I'm looking into the possibility of using Capistrano as a generic deploy solution. By "generic", I mean not-rails. I'm not happy with the quality of the documentation I'm finding, though, granted, I'm not looking at the ones that presume you are deploying rails. So I'll just try to hack up something based on a few examples, but there are a couple of problems I'm facing right from the start.
My problem is that cap deploy doesn't have enough information to do anything. Importantly, it is missing the tag for the version I want to deploy, and this has to be passed on the command line.
The other problem is how I specify my git repository. Our git server is accessed by SSH on the user's account, but I don't know how to change deploy.rb to use the user's id as part of the scm URL.
So, how do I accomplish these things?
Example
I want to deploy the result of the first sprint of the second release. That's tagged in the git repository as r2s1. Also, let's say user "johndoe" gets the task of deploying the system. To access the repository, he has to use the URL johndoe#gitsrv.domain:app. So the remote URL for the repository depends on the user id.
The command lines to get the desired files would be these:
git clone johndoe#gitsrv.domain:app
cd app
git checkout r2s1
Update: For Capistrano 3, see scieslak's answer below.
Has jarrad has said, capistrano-ash is a good basic set of helper modules to deploy other project types, though it's not required as at the end of the day. It's just a scripting language and most tasks are done with the system commands and end up becoming almost shell script like.
To pass in parameters, you can set the -s flag when running cap to give you a key value pair. First create a task like this.
desc "Parameter Testing"
task :parameter do
puts "Parameter test #{branch} #{tag}"
end
Then start your task like so.
cap test:parameter -s branch=master -s tag=1.0.0
For the last part. I would recommend setting up passwordless access using ssh keys to your server. But if you want to take it from the current logged in user. You can do something like this.
desc "Parameter Testing"
task :parameter do
system("whoami", user)
puts "Parameter test #{user} #{branch} #{tag}"
end
UPDATE: Edited to work with the latest versions of Capistrano. The configuration array is no longer available.
Global Parameters: See comments Use set :branch, fetch(:branch, 'a-default-value') to use parameters globally. (And pass them with -S instead.)
Update. Regarding passing parameters to Capistrano 3 task only.
I know this question is quite old but still pops up first on Google when searching for passing parameters to Capistrano task. Unfortunately, the fantastic answer provided by Jamie Sutherland is no longer valid with Capistrano 3. Before you waste your time trying it out except the results to be like below:
cap test:parameter -s branch=master
outputs :
cap aborted!
OptionParser::AmbiguousOption: ambiguous option: -s
OptionParser::InvalidOption: invalid option: s
and
cap test:parameter -S branch=master
outputs:
invalid option: -S
The valid answers for Capistrano 3 provided by #senz and Brad Dwyer you can find by clicking this gold link:
Capistrano 3 pulling command line arguments
For completeness see the code below to find out about two option you have.
1st option:
You can iterate tasks with the key and value as you do with regular hashes:
desc "This task accepts optional parameters"
task :task_with_params, :first_param, :second_param do |task_name, parameter|
run_locally do
puts "Task name: #{task_name}"
puts "First parameter: #{parameter[:first_param]}"
puts "Second parameter: #{parameter[:second_param]}"
end
end
Make sure there is no space between parameters when you call cap:
cap production task_with_params[one,two]
2nd option:
While you call any task, you can assign environmental variables and then call them from the code:
set :first_param, ENV['first_env'] || 'first default'
set :second_param, ENV['second_env'] || 'second default'
desc "This task accepts optional parameters"
task :task_with_env_params do
run_locally do
puts "First parameter: #{fetch(:first_param)}"
puts "Second parameter: #{fetch(:second_param)}"
end
end
To assign environmental variables, call cap like bellow:
cap production task_with_env_params first_env=one second_env=two
Hope that will save you some time.
I'd suggest to use ENV variables.
Somethings like this (command):
$ GIT_REPO="johndoe#gitsrv.domain:app" GIT_BRANCH="r2s1" cap testing
Cap config:
#deploy.rb:
task :testing, :roles => :app do
puts ENV['GIT_REPO']
puts ENV['GIT_BRANCH']
end
And take a look at the https://github.com/capistrano/capistrano/wiki/2.x-Multistage-Extension, may be this approach will be useful for you as well.
As Jamie already showed, you can pass parameters to tasks with the -s flag. I want to show you how you additionally can use a default value.
If you want to work with default values, you have to use fetch instead of ||= or checking for nil:
namespace :logs do
task :tail do
file = fetch(:file, 'production') # sets 'production' as default value
puts "I would use #{file}.log now"
end
end
You can either run this task by (uses the default value production for file)
$ cap logs:tail
or (uses the value cron for file
$ cap logs:tail -s file=cron
Check out capistrano-ash for a library that helps with non-rails deployment. I use it to deploy a PyroCMS app and it works great.
Here is a snippet from my Capfile for that project:
# deploy from git repo
set :repository, "git#git.mygitserver.com:mygitrepo.git"
# tells cap to use git
set :scm, :git
I'm not sure I understand the last two parts of the question. Provide some more detail and I'd be happy to help.
EDIT after example given:
set :repository, "#{scm_user}#gitsrv.domain:app"
Then each person with deploy priveledges can add the following to their local ~/.caprc file:
set :scm_user, 'someuser'