I have a nightly build that is triggered by another checkin build. The trigger from checkin build is predicated by the success of that build. In other words, got_revision for the nightly build will always point to the last passing checkin build.
I'd like to skip the nightly build if got_revision is the same as last build. What would the master config look like?
Thanks in advance.
See: http://docs.buildbot.net/current/manual/cfg-schedulers.html
Example:
c['schedulers'].append(
schedulers.Nightly(name='BeforeWork',
branch=`default`,
builderNames=['builder1'],
dayOfWeek=0,
hour=[6,8],
minute=23,
onlyIfChanged=True))
onlyIfChanged=True should do the trick => makes sure to only perform the build when someone has committed code in the interim.
Following code can be used for doStepIf callback. It will skip a run if the last build was same revision and passed. This could be extended to search for all previous builds instead of the last run. Also, in my original question I asked for got_revision. In doStepIf it's best not to use got_revision because not all builds will result in valid got_revision.
def do_step_if(step):
cur_status = step.build.build_status
prev_status = cur_status.getPreviousBuild()
# this is the first build
if prev_status==None:
return True
# never skip if this is a forced run
if cur_status.getProperty("revision")==None or cur_status.getProperty("revision")=="":
return True
# got_revision won't be set if update step wasn't run, so better to use revision
if prev_status.getResults()==SUCCESS and prev_status.getProperty("revision")==cur_status.getProperty("revision"):
return False
return True
Related
I have this structure of projects (folders) in git repository:
/src
/src/Sample.Backend.Common
/src/Sample.Backend.Common.Tests
/src/Sample.Backend.Common.Domain
/src/Sample.Backend.Common.Domain.Tests
/src/Sample.Backend.Pricing.Abstractions
/src/Sample.Backend.Pricing.Domain
/src/Sample.Backend.Pricing.Domain.Tests
/src/Sample.Backend.Pricing.Persistence
/src/Sample.Backend.Pricing.Persistence.Tests
/src/Sample.Backend.Accounting.Abstractions
/src/Sample.Backend.Accounting.Domain
/src/Sample.Backend.Accounting.Domain.Tests
/src/Sample.Backend.Accounting.Persistence
/src/Sample.Backend.Accounting.Persistence.Tests
/src/Sample.Backend.Api
/src/Sample.Common
/src/Sample.Frontend.Common
/src/Sample.Frontend.Web
/src/Sample.Tests.Common
(The sample is simplified, in real there are much more projects/folders.)
I want different pipelines for different parts. For example a pipeline to be triggered whenever any file is commited in master branch in any Backend project. Something like this:
trigger:
branches:
include:
- master
paths:
include:
- src/Sample.Backend.*
- src/Sample.Common
- src/Sample.Tests.Common
The problem is, that filter src/Sample.Backend.* is not working. I have to add exact name of each Backend folder to get it working. I could use exclude but I have the same problem - there are many other projects and I would have to name them all.
I found that wildcards are not supported: https://github.com/MicrosoftDocs/azure-devops-docs/issues/397#issuecomment-422958966
Is there any other way to achieve the same result?
Does Azure YAML pipelne support wildcards in path filter in trigger?
This is a known request on our main forum for product:
Support wildcards (*) in Trigger > Path Filters
This feature has not yet been implemented; you could add your comment and vote this on user voice.
As a workaround for us, we add an inline PowerShell task as the first task to execute the git command line git diff HEAD HEAD~ --name-only then get the modified file names and filter the files name in the latest submit, and use Logging Command to sets variables which are then referenced in custom conditions in the next steps in the build pipeline:
and(succeeded(), eq(variables['CustomVar'], 'True'))
Our inline PowerShell script:
cd $(System.DefaultWorkingDirectory)
$editedFiles = git diff HEAD HEAD~ --name-only
echo "$($editedFiles.Length) files modified:"
$editedFiles | ForEach-Object {
echo $_
Switch -Wildcard ($_ ) {
'XXXX/Src/Sample.Backend.*' {
Write-Host ("##vso[task.setvariable variable=CustomVar]True")
}
'XXXX/Src/Sample.Common*' {
Write-Host ("##vso[task.setvariable variable=CustomVar]True")}
'XXXX/Src/Sample.Tests.Common' {
Write-Host ("##vso[task.setvariable variable=CustomVar]True")}
}
}
Then add the condition for all remaining tasks:
In this case, if the changed files do not meet our filters, then all remaining tasks will be skipped.
UPDATE: 09/09/2021
This is possible now as it is written here
Wild cards can be used when specifying inclusion and exclusion branches for CI or PR triggers in a pipeline YAML file. However, they cannot be used when specifying path filters. For instance, you cannot include all paths that match src/app//myapp*. This has been pointed out as an inconvenience by several customers. This update fills this gap. Now, you can use wild card characters (, *, or ?) when specifying path filters.
Note: documentation seems to be not updated yet.
Old answer:
No this is not possible at the moment. You have even feature request here and I would recommend to upvote it. (I already did this) Rick in above mentioned topic shared his idea how to overcome the issue:
I currently achieve this by having 3 files:
azure-pipelines.yml ( This calls some python on each commit )
azure-pipelines.py (This checks for changed folders and has some parameters to ignore certain folders, then calls the API directly)
azure-pipelines-trigger.yml ( This is called by the python based on the changed folders )
It works well enough, but it is unfortunate for the need to go through these loops.
But it needs an extra work.
This feature will roll out over the next two to three weeks according to the latest release notes
Update on this.
It took a few weeks but the change mentioned by pavlo in the comments above finally got rolled out and path triggers are now supported in YAML.
Getsolution.exe will give New data available or no new data available, if new data available then next jobs should be executed else nothing should be executed. How should i do it? (i am working on classic editor)
example: i have set of tasks, consider 4 tasks:
task-1: builds the solution
task-2: runs the Getstatus.exe which get the status of data available or no data available
task-3: i should be able to use the above task and make a condition/use some api query and to proceed to publish an artifact if data is available if not cleanly break out of the task and stop the build. it Shouldn't proceed to publish artifact or move to the next available task
task-4:publish artifact
First what you need is to set a variable in your task where you run Getstatus.exe:
and then set condition in next tasks:
If you set doThing to different valu than Yes you will get this:
How to refer previous task and stop the build in azure devops if there is no new data to publish an artifact
Since we need to execute different task based on the different results of Getstatus.exe running, we need set the condition based on the result of Getstatus.exe running.
To resolve this, just like the Krzysztof Madej said, we could set variable(s) based on the return value of Getstatus.exe in the inline powershell task:
$dataAvailable= $(The value of the `Getstatus.exe`）
if ($dataAvailable -eq "True")
{
Write-Host ("##vso[task.setvariable variable=Status]Yes")
}
elseif ($dataAvailable -eq "False")
{
Write-Host ("##vso[task.setvariable variable=Status]No")
}
Then set the different condition for next task:
You could check the document Specify conditions for some more details.
I created a new build with Azure Pipelines (Azure DevOps) and it worked really well.
Usually, you use $(Rev:.r) to get the revision in the build. Unfortunately, it seems the variable isn't replaced/set in the build steps. The only place where you can use it is the name: property in the YAML document.
Now I set it in the name and extract it in some PowerShell, which isn't necessary if you can get it via an environment variable.
How do I get the Revision (like $(Rev)) in the new builds (outside of the name: property in the YAML document)?
(The Build Agents running on-premise, inside Docker - but this shouldn't affect the things above)
You can't get the revision number without parsing, it is not stored as a separate field somewhere or in an environment variable.
The $(Rev:.r) portion instructs Azure DevOps to come up with the first number that makes the build number unique (and, in that specific example, put a dot in front of it).
Like you said, the only way is to use PowerShell script to get the value:
$buildNumber = $Env:BUILD_BUILDNUMBER
$revision= $buildNumber.Substring($buildNumber.LastIndexOf('.') + 1)
Edit:
You can install the Get Revision Number extension that does it.
Another possible solution to the above problem could be to use counter expression for ex: we difine the variable and use it in a task to build nuget package.
variables:
counterVar: $[counter($(versionVariable),0)]
.......
- task: CmdLine#2
inputs:
script: >
nuget pack ClassLibrary1/ClassLibrary1.csproj
-OutputDirectory $(Build.ArtifactStagingDirectory)
-NonInteractive
-Properties Configuration=release
-Version $(versionVariable).$(counterVar)
-Verbosity Detailed
-IncludeReferencedProjects
Here versionVariable is a custome variable defined in pipelines->variables.And the seed value is 0(2nd param to counter).
It works as below
Let's assume the versionVariable is 1.19
Build Run 1 counterVar will be 0.
Build Run 2 counterVar will be 1.
Now say we change the versionVariable to 1.20
Build Run 3 counterVar will be 0.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops
Check the counter expression in above link it reset its value for diff prefix.
P.S. Benefit of using counter over $(Rev:r) is that it can start from 0 unlike $(Rev:r)
I have a Jenkins build pipeline created using workflow plugin. At the beginning the pipeline runs a gulp build inside of the docker container and then archives test results using the following code
step([$class: 'JUnitResultArchiver', testResults: 'build/test-results/*.xml'])
In the following steps I package up the artifacts and ship them to the binary repository.
When unit tests are not passing Jenkins understands the build is unstable and marks it yellow. However it still continues with subsequent steps in the pipeline. Is there any way make the pipeline stop when unit tests are failing?
the JUnitResultArchiver will cause this condition to be true when the build is unstable:
currentBuild.result != null.
If I remember correctly it sets it to UNSTABLE, but it is enough to check that is different than null.
So you could do something like
step([$class: 'JUnitResultArchiver', testResults: 'build/test-results/*.xml'])
if (currentBuild.result == null) {
//contintue with your pipeline
} else {
//notify that the build is unstable. //or just do nothing
}
There is nothing to do at Jenkins side but at Gulp side. The call to gulp CLI needs to return a proper error value to have the sh step failing correctly.
Jenkins just interprets what the shell is returning, so you juts need to make Gulp to return a fail when tests fail (see this blog post, it seems to achieve exactly that).
I need to skip a build step when building some branches.
More exactly, I want to execute a ShellCommand step only if the script to be ran is present on the source tree.
I tried:
ShellCommand(command=["myscript"],
workdir="path/to",
doStepIf=(lambda step: os.path.isfile("path/to/myscript")))
but the step is never executed.
def doesMyCriticalFileExist(step):
if step.getProperty("myCriticalFileExists"):
return True
return False
<factory>.addStep(SetProperty(command='[ -f /path/to/myscript ] && ls -1 /path/to/myscript || exit 0', property='myCriticalFileExists')))
<factory>.addStep(ShellCommand(command=["myscript"], workdir="path/to", doStepIf=doesMyCriticalFileExist))
The better thing to do is to set a property in a previous step and then check the property in your doStepif method. the os.path.isfile you have there gets run at configure time, (buildbot startup) not run time.
I ended up solving this by letting the step run unconditionally but setting haltOnFailure=False. This way if the required file doesn't exist it fails but doesn't kill the rest of the build.
Just use simple python if statement before the step:
if(condition):
your buildbot step