The project that I am working on is a jQuery plugin. I have managed to get Travis CI to build a test project using Gulp/NodeJS successfully. Now I am trying to work out what workflow to use to bump the version number.
In TeamCity and MyGet there is a setting in the CI server to form a version number pattern that auto increments on each build, which can be used by the build script to update versions in the deployment files and to label the Git repo. However, in the free version of Travis CI, there doesn't seem to be an option for versioning at all.
I have read several articles on continuous deployment with Travis CI, here, here, and here, but none of them even broach the topic of versioning. Obviously, the version needs to be changed for the release. So what am I missing here?
Another problem I noted when going through the documentation is that it mentioned that Travis CI is not able to update the GitHub repository. Doesn't that basically mean it won't be able to create a Git tag?
If there is no way to version from Travis CI, then what is the typical workflow for the release process for such a plugin? Is the versioning always done manually? If so, how could there be "continuous deployment"?
Before it starts running the instructions in your .travis.yml file, Travis will set a bunch of environment variables (in the VM that is building your project) with various bits of information about your build, such as what branch is being built and so on.
You probably want one of these:
TRAVIS_BUILD_NUMBER: The number of the current build (for example, “4”).
TRAVIS_JOB_NUMBER: The number of the current job (for example, “4.1”).
But it's going to be very difficult to do anything sensible if you don't have control of the repository, because you'll need to upload a .travis.yml file into the root of your source code folder, otherwise Travis won't know what to do.
Use bumped for release versioning. When you're satisfied with the changes in master, run:
bumped release <major|minor|patch>
After you push the changes, either directly or through a release PR, you can check for the presence of new tags in Travis CI and publish the package to the registry automatically.
If you consider that every PR must end up to your enduser without thinking of the impact of such changes, then your version numbers have no meaning.
You don't give your user a way to know if it is a major change that break compatibility or a bug fix. You don't allow him to get update without worrying about backward compatibility.
Currently, the commit id is your version number.
If you want to give meaning to your version numbers then you have to think of the impact of your pull requests on the enduser (http://semver.org/). You have to choose a version number for a specific PR or a group of PR.
So basically, since you have to 'think' of a certain version number for a specific version that you want to deliver, you can't automate this process.
Release/tag creation is the way to go : )
You can accomplish this by setting up a script that would create a ~/.netrc file to access the repository. In this file you can specify something like:
machine https://github.com/xxx/yyy.git
login <blah>
And instead of putting in your credentials, you can pass an github access token. You can use the travis encrypt to register it in the .travis.yml file, and export the variable for your script's use. From there in your script, you can issue regular git commands such as:
git add <some file>
git commit -m "This is $TRAVIS_BUILD_NUMBER"
git push origin <branch>
Related
I have a continuously triggered Azure DevOps release definition that deploys a compiled Angular app to a web server and also runs Cypress e2e tests. The Cypress tests must run against the source code, so that means I need an artifact that is able to reference the same commit that was used to create the compiled app.
I created a GitHub artifact that gets the source code, but I can't figure out how to automatically change the branch/commit to whatever was used for the compiled app (it could be any branch and the names are not known ahead of time). Azure forces me to enter a hard-coded branch name and it does not accept wildcards or variables.
If I could simply use the variable ${Release.Artifacts.{alias}.SourceBranchName} for the default branch, I think I'd achieve my goal. Since Azure doesn't allow this, is there an alternative approach that accomplishes the same thing?
Note 1: The "Default version" dropdown has an option "Specify at the time of release creation", but that is intended for manual releases and can't be used for triggered ones, so no luck there.
Note 2: I looked into publishing the source code as an artifact, but it currently has almost 70,000 files and it adds more than an hour to the build step, so that also is not an option.
When you use the Release Pipeline artifacts, you are not able to set the pipeline variable in the Default branch field. This field only supports hard-coded.
is there an alternative approach that accomplishes the same thing?
The variable:$(Release.Artifacts.{alias}.SourceBranchName) can be used in the Release Pipeline agent job.
Workaround:
You can remove the Github artifacts and then add a Command Line task/PowerShell task/ Bash task to run the git command to clone the target repo.
For example:
git clone -b $(Release.Artifacts.{alias}.SourceBranchName) GithubRepoURL
PowerShell sample:
In this case, the script will use the same branch as Build Artifacts to checkout the source code.
I'm using Visual Studio Team Services to build my project which is stored in GitHub (here). The master branch contains multiple projects which make up the solution. Amongst those are a WebAPI project and a Cordova project. I need to build those using two separate build definitions in VSTS.
Previously I had set-up my build definition and used the branch filters to filter on what had been pushed to the repo. For instance:
master/src/API
This worked, but it doesn't any more. It seems as if the underlying code has changed. A filter of 'master' still works and I understand how this feature is probably meant to filter specifically on branches and maybe not on folders within the branch?
It's not a huge problem, but at this time all of my builds will trigger with every check-in, even if nothing changed in the meantime for that source code. So I'm not wondering what a good solution for this issue would be:
Put every project in it's own branch. Seems like a workaround
Some other filter option or maybe another syntax or something?
Leave it as it and don't worry about the extra builds (but that itches, you know...)
Anyone running a similar set-up?
Path filters is not supported for VSTS GitHub CI Build, it is available for Git CI Build on VSTS. You can vote this user voice: https://visualstudio.uservoice.com/forums/330519-team-services/suggestions/15140571-enable-continuous-integration-path-filters-for-git
The workaround is as you said that put every project in its own branch.
This is for a repository containing a library. The library version number is incremented (manually) each time a Merge Request to master is accepted.
However, if I want to access a file from version X.Y.Z, I have to look for the commit that incremented the version number to X.Y.Z, get its date, and then look in the history of the file for the version at that date.
I would like to create a tag per version, automatically when the Merge Request to master is created. Is this possible?
I hoped it would be possible with the new GitLab slash commands, but there currently is not support for tags.
Is there any other possibility than using web hooks?
While facing the same challenge, I stumbled upon this suggestion on GitLab's former issue tracker on GitHub1:
“You can write up a script to use GitLab API to accept a merge request, get the commit of the merge and then tag that commit.” --MadhavGitlab
(just to mention that — for me that's not sufficient)
1 EDIT:
Looks like all issues have been purged from the GitHub mirror, so this link does no longer work, but luckily the relevant quote persists right here.
I first tried to do it the gitlab way, by creating a .gitlab-ci.yml file in the project top-level directory. That file can contain the commands creating the version tag. The user executing the script has to have enough permission to push to the git project, and be configured with authoring information.
I finally did it on a Jenkins server, where I created a job that is invoked when commits are pushed into a specific branch. The tag can be created in the execute shell commands.
I have been researching and am trying to figure out the best branching and deployment strategy to accomplish the requirements below. Maybe I’m missing something but it is more complicated than it seems. Ideally, we’d just have one permanent branch, ‘master’, that could have specific commits tagged to mark releases to production.
Our current strategy is based on Git Flow and has permanent branches ‘master’ (only has releases to production) and ‘develop’. The primary thing that complicates using a multiple permanent-branches model is the concept of “promoting” the same build from the staging environment to production. Currently, this needs to be done in a separate source code branch (deployments to staging come from ‘develop’, deployments to prod come from ‘master’).
Tools: Git (VSTS), TeamCity, Octopus Deploy
Requirements (feature and hotfix lifecycles):
All code is reviewed via pull requests (enforced via branch policies)
All code gets deployed to a staging environment for testing
We can quickly go back to any snapshot of code that was deployed previously
If testing is successful, then the same build can be “promoted” from our staging environment to production (no need to build again)
Features accumulate over time before pushing out to production as a single release. Hotfixes have to be able to go through without getting caught up in the "all or nothing" next regular release.
I like the idea of having one permanent branch with tags (re: The master/develop split is redundant, http://endoflineblog.com/gitflow-considered-harmful), but having additional permanent branches may better facilitate deploying to different lifecycles/versions (feature and hotfix) to Octopus.
I have been wrestling with how best to pull this off and I may be over complicating things. Any feedback is appreciated.
It seems you have a number of questions and they are quite broad... I'll add some comments to each of your requirements as a conversation starter, but this whole thread might get blocked by moderators as it is definitely not the style of questions SO was made for.
All code is reviewed via pull requests (enforced via branch policies)
I haven't looked at VSTS for ages, but I'd expect they already support branch policies and pull-requests, so not sure if there's anything you need here other than configure settings in your repositories.
In case VSTS does not support that, you might consider moving to a tool that does e.g. BitBucket, GitHub, etc. Both of these have an on-premises version in case you can't (or don't want to) use the cloud hosted version.
All code gets deployed to a staging environment for testing
You achieve that with setting up lifecycles in Octopus Deploy, to make sure deployments/promotions follow the the sequence you want.
We can quickly go back to any snapshot of code that was deployed previously
You already have source control, so all you need now is traceability from the code that is deployed in an environment, to the deployment version in Octopus Deploy, the build job in TeamCity, the branch and exact commit in your source control.
There's a few things that you can do, to achieve that:
Define a versioning scheme that works for you. I like to use semantic versioning. "Major" and "Minor" versions are defined by the developers, and the "Patch" is the auto-incremented number from TeamCity (%build.number%). Every git push build the code and generates a unique build version (%major%.%minor%.%build.number%)
As part of the build steps in TeamCity, before you compile the code, make sure your source files are patched with the version number assigned by each build, the commit hash from your source control, and the branch name. e.g. if you are using .NET, make sure all the AssemblyInfo.cs files are updated with that version, so that the version is embedded in the binaries. This allows anyone to query the version looking at the properties of the binary files, and also allows you to display the app version on the app itself (e.g. status bar, footer, caption, about box, etc.)
Have TeamCity tag your source control with the version number of every build, so you can quickly see on your source control history. You probably only want to do that for the master branch, though which is what you care about.
Have Octopus tag your source control with the deployment version number and the environment name, so that you can quickly see (from your source control) what got deployed where.
Steps 1 and 2 are the most important ones, really. 3 and 4 are just nice-to-have. Most of the time you'll just open the app in the environment, check the commit hash in the "About", and do a git checkout to that commit hash...
If testing is successful, then the same build can be "promoted" from our staging environment to production (no need to build again)
Again, Octopus Deploy lifecycles, and make sure anything different in each environment is defined in the configuration file of the application, which is updated during the Octopus deployment, using environment-specific variables.
In terms of branch workflow, this last requirement makes it mandatory to merge changes into master (or whatever your "production" branch is) before the deployment lifecycle can begin.
I have a CI build that is setup in TeamCity that will trigger when a pull request is made in BitBucket (git). It currently builds against the source branch of the pull request but it would be more meaningful if it could build the merged pull request.
My research has left me with the following possible solutions:
Script run as part of build - rather not do it this way if possible
Server/agent plugin - not found enough documentation to figure out if this is possible
Has anyone done this before in TeamCity or have suggestions on how I can achieve it?
Update: (based on John Hoerr answer)
Alternate solution - forget about TeamCity doing the merge, use BitBucket web hooks to create a merged branch like github does and follow John Hoerr's answer.
Add a Branch Specification refs/pull-requests/*/merge to the project's VCS Root. This will cause TeamCity to monitor merged output of pull requests for the default branch.
It sounds to me like the functionality you're looking for is provided via the 'Remote Run' feature of TeamCity. This is basically a personal build with the merged sources and the target merge branch.
https://confluence.jetbrains.com/display/TCD8/Branch+Remote+Run+Trigger
"These branches are regular version control branches and TeamCity does not manage them (i.e. if you no longer need the branch you would need to delete the branch using regular version control means).
By default TeamCity triggers a personal build for the user detected in the last commit of the branch. You might also specify TeamCity user in the name of the branch. To do that use a placeholder TEAMCITY_USERNAME in the pattern and your TeamCity username in the name of the branch, for example pattern remote-run/TEAMCITY_USERNAME/* will match a branch remote-run/joe/my_feature and start a personal build for the TeamCity user joe (if such user exists)."
Then setup a custom "Pull Request Created" Webhook in Bitbucket.
https://confluence.atlassian.com/display/BITBUCKET/Tutorial%3A+Create+and+Trigger+a+Webhook
So for your particular use case with BitBucket integration, you could utilize the WebHook you create, and then have a shell / bash script (depending on your TeamCity Server OS) that runs the remote run git commands automatically, which will in turn automatically trigger the TeamCity Remote Run CI build on your server. You'll then be able to go to the TeamCity UI, +HEAD:remote-run/my_feature branch, and view the Remote Run results on a per-feature basis, and be confident in the build results of the code you merge to your main line of code.
Seems that BitBucket/Stash creates branches for pull requests under:
refs/pull-requests//from
You should be able to setup a remote run for that location, either by the Teamcity run-from-branch feature, or by a http post receive hook in BitBucket/Stash.
You can also use this plugin : https://github.com/ArcBees/teamcity-plugins/wiki/Configuring-Bitbucket-Pull-Requests-Plugin
(Full disclosure : I'm the main contributor :P, and I use it every day)