Jenkins official Wiki page for Matrix projects isn't really helping me; so I have a few questions.
We're trying to build a couple of projects that are all essentially the same, just some are being branded differently for our customers. In other words, the software / tests / etc. are all identical, except for some tweaks to turn BrandA into BrandB (or BrandC, etc.)
I figure I should be using a Matrix project to create builds for BrandA, BrandB, etc. While I haven't figured out all my steps yet (including how to rename executables after they're built) I know that I will need to pass the Brand Name to many of my Jenkins Powershell scripts during the build process, and then use that brand n the script.
How do I get these variables into my scripts? Are they automatically passed in to every build step in Jenkins? What is the variable name to use?
Finally, is there a good resource on building these multi-configuration projects in Jenkins? I can't seem to find anything comprehensive online.
If you usually build the job for BrandA and only occasionally for BrandB and BrandC a matrix project may not be what you want. I recommend, instead, using a parameterized job where the brand is a parameter whose default value is BrandA. If the parameter is named BRAND the parameter is accessible in all of the builds and publish steps with ${BRAND} and as an environment variable as %BRAND%.
I refer you to the parameterized build wiki for more details.
Yes, ${BRAND} and %BRAND% should work fine.
If you're using Maven, ${env.BRAND} does this too.
There's a plugin that you can see all Environment Variables that are available to your job/build.
https://wiki.jenkins-ci.org/display/JENKINS/EnvInject+Plugin
I'm not aware of that kind of process but I suggest you tu use the Copy project functionnality.
New Job
Copy From existing job
You will have a copy of your Job and you'll be able to setup easily all specific fields.
Related
Should I bundle the source code, build script and deployment script together? In my previous company, they are always bundled together, but there is always a problem that when the company add a new server, they need to change the deployment script and create a new build version, however, there is no change to the source code. I would like to see what is your company practice on the source control, build and deployment.
The best practices for deployment are to have some standard system for that purpose. Usually that system will have a standard way to enumerate what hosts are available and what versions of software are on each host, so any scripts necessary for deployment become agnostic to the machines in use.
Similarly, in many environments, deployment uses a set of standard techniques. For example, it is common to use CI to run tests and then build one or more deployment artifacts, such as a tarball or container, and then all deploys using the same technique use the same deployment method (e.g. unpack the tarball into a directory named after the repository name), so in that case a deployment script may not even be necessary. If you use a standard method and one is necessary, then obviously you should include it in your artifact (which means it's included in the source code) or in the configuration for the deployment system (which should be maintained as repository as well).
As to whether one should include source code depends on whether it's needed. If you're deploying a project in a language like Python or Ruby, then obviously it will be needed. However, if you're deploying a project in a compiled language like Go or Rust, then it probably is not, and your build artifacts will be smaller and easier to work with if you don't include it and just build a binary artifact during CI.
In Azure DevOps I have a Pipeline variable "package version" and I set it using 0.1.3$(Rev:.r)-alpha .
I use that variable to replace the "Version" in the .net core project file.
In the Build tasks the dotnet build give me this error:
so I assume the Pipeline variable cannot use a nested variable.
There is a nother way or a different syntax to do it?
[Edit]
The nested variables should work.
I think the error was the dotnet pack using Automatic package versioning set to "Use the build number" that contain the wrong $(rev:.r) (lowercase!).
Using a custom string for Version in the VS project file gives noise on VS (error if the file is open, warning otherwise) and now the Pipeline Build give me an error on dotnet nuget restore because of the invalid Version.
I decided to use a simple clean version, <Version>0.1.2</Version>, that I can choose and document.
Then I want to find a way to read it in the build pipeline and create a custom variable attaching the Build Revision:
0.1.2$(Rev:.r) => 0.1.2.123
That is the result I want.
I found this: https://marketplace.visualstudio.com/items?itemName=tmarkovski.projectversionasvariable
I'm using it with the default settings.
I'm using the variables it creates to compose a new variable, "package version":
and I use that in the nuget pack task:
but it does not work.
The resulting variable still contain "$(Rev:.r)" not parsed.
I'll try to use again the "Automatic package versioning" and create the build number in the Options...
[Edit 2]
Set the build version number in the Pipeline Build Options worked.
It's not the optimal solution because the Build Number is not parsed and looks awful.
[Solution]
$(Rev:.r) is not available outside Build / Options.
I used $(Build.BuildNumber) and I'm able to create a composed variable:
$(Version.MajorMinor).$(Build.BuildNumber)-alpha
Build.BuildNumber is valorized in Build/Options: $(Build.DefinitionVersion)$(Rev:.r)
Version.MajorMinor is created by the Project Version As Build Variable add-on.
I don't know how to obtain the same result without using a third party component.
I'm glad my extension helped and thanks for the nice review. If you want more control and have a little time to spend to play with PowerShell, you can easily achieve what you want in two steps, without using third party extensions.
Read and parse the .csproj file as XML
Set build variables
Here are couple of links on reading XML files and working with build variables, it's actually pretty easy.
https://www.business.com/articles/powershell-read-xml-files/
VSTS: Pass build/release variables into Powershell script task
[xml]$XmlDocument = Get-Content -Path $env:project_file
echo $XmlDocument.Project.PropertyGroup.Version
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=vsts&tabs=yaml%2Cbatch
Write-Host '##vso[task.setvariable variable=package_version;issecret=true]0.1.2.'
Note: my answer wouldn't fit in a comment, I had to post an answer.
Following this Microsoft tutorial (Run unit tests with your builds), I was expecting to be able to run my unit-tests automatically when a build is triggered, such as a Pull request.
However, when I look in the Pipeline / Builds tab and try to edit my pipeline, there is nothing that allows me to add a new task (see screenshot below).
However, there doesn't appear to be any way of adding a task. I can't even switch to the YMAL without navigating to the source via the Repo. I was hoping to use the GUI though as my YMAL is non-existent.
I have created a test solution with the following structure, which is held in the repo:
Core Solution
|_ Class Library Project (.NET Core)
|_ MSTest Test Project (.NET Core)
I was hoping to have a build step followed by a Unit Test step using the tests in my MSTest Test Project (.NET Core) project once they were built.
There appears to be a Tasks option in the Releases tab by the way, but I was expecting to be able to add tasks for builds as well, especially Unit Tests.
Being new at this, perhaps I have missed or misunderstood something. I would be grateful for any help and to be pointed in the right direction.
It seems that there is an obscure link that takes you through to the correct process, found it quite by accident, see screenshot below:
It turns out that there is a little link titled Use the visual designer that I'd missed. Seems a little odd that most of the tutorials discuss this process and yet it's partially obscured. I guess that this shows that the platform is a work in process and still being added to and improved.
Another few caveats for those descending this little rabbit hole, after selecting the Use the visual designer link, be sure to select the Empty pipeline template, or whatever is appropriate to your project/solution type, and not the YAML option at the top of the list, otherwise you'll be back where you started.
Finally, it seems that automated Unit Tests don't work on Agents other than Hosted VS2017 Agents (that said, I haven't tried the Hosted option). This is set in the very next screen by selecting the Pipeline and selecting the Agent pool from the drop-down.
One last thing... The pipeline won't run automatically unless you check the Enable continuous integration checkbox on the Triggers tab.
Once this is all done, I simply chose the tasks needed for the build by clicking the plus symbol on the Agent job 1 item.
Good luck
Kaine
You have created a new YAML style of build configuration. Currently Azure DevOps does not support of having YAML & GUI editor at the same time (this is upcoming feature in Q1/2019).
To get a GUI editor, create new build pipeline.
In this selection click "Use the visual designer".
Then at template phase, don't select YAML. Any other template will do. You can easily delete all the build steps after selecting template, so don't be afraid to choose any.
I am deploying web application to azure using TFS CI automated build deployment.
In our config maintain build version like 2014.05.19.1 which is $(Date).$(rev) format.
All I want to update config each time build is deployed.For that I am passing value to 'BuildVersion' parameter in template to powershell script which actually performs publishing to azure.
I tried using $(Date:yyyyMMdd)$(Rev:.r) but it is considered string as it is.
I want to get current build version just like IBuildDetail.BuildNumber
within template.
My question is how to get the build version?
If you are using Invoke Process, instead of passing value for BuildVersion parameter you can directly use 'BuildDetail.BuildNumber' in parameters for process like
String.Format("-BuildNumber ""{0}""",BuildDetail.BuildNumber)
This would give the required build number.
If your PowerShell script is being executed from your TFS build, it should have access to the environment variables specific to the TFS context of the build. If that is the case, you actually don't need to pass the $(BuildVersion) parameter to the script, as it already is accessible to the PS script in the $env:TF_BUILD_BUILDNUMBER environment variable. Try testing something like $env:TF_BUILD_BUILDNUMBER | Out-File "D:\Dev\BuildNumber.txt" in your script. You should hopefully see the file containing your build number after running your build.
(I am assuming you are using a relatively new build process template...one that contains the "Post-Build script path" parameter, such as TfvcTemplate.12.xaml)
Hope this is helpful.
I would recommend that you use the right tool for the right job. The build system, is really only for building (compile & test). We have been using it for other things for years coz we did not have another integrated solution. However Microsoft recently bought InRelease and rebranded as Release Management for Visual Studio 2013. I have successfully integrated this with TFS 2012 as well.
Please note, although my specific example here involves Java/Grails, it really applies to any type of task available in Bamboo.
I have a task that is a part of a Bamboo build where I run a Java/Grails app like so:
grails run-app -Dgrails.env=<ENV>
Where "<ENV>" can be one of several values (dev, prod, staging, etc.). It would be nice to "parameterize" the plan so that, sometimes, it runs like so:
grails run-app -Dgrails.env=dev
And other times, it runs like so:
grails run-app -Dgrails.env=staging
etc. Is this possible, if so, how? And does the REST API allow me to specify parameter info so I can kick off different-parameterized builds using cURL or wget?
This seems to be a work around but I believe it can help resolve your issue. Atlassian has a free plugin call Bamboo Inject Variables Plugin. Basically, with this plugin, you can create an "Inject Bamboo Variables from file" task to read a variable from a file.
So the idea here is to have your script set the variable to a specific file then kick off the build; the build itself will read that variable from the file and use it in the grails task.
UPDATE
After a search, I found that you can use REST API to change plan variables (NOT global). This would make your task simpler: just define a plan variable (in Plan Configuration -> tab Variables) then change it every time you need to. The information on how to change is available at Bamboo Knowledge Base