I am using Azure DevOps, and I want to add a policy to my dev branch which ensures that the number of StyleCop warnings will be reduced after merge.
Is that possible? If yes, how?
Note:
I cannot remove all warnings at once, so I cannot have check which tells if there are zero warnings or not.
I know one answer is to write a scrip which builds on server and passes or fails depending on the reduction, but I am more interested in a way which allows to add a policy right on Azure DevOps
Check if Stylecop warnings have been reduced
As we know, there is no such out of branch policy to detect whether the number of StyleCop warnings has reduced.
That because the branch policy could not get the number of StyleCop warnings directly. We have to manually develop a script to compare the number of StyleCop warnings on the two branches to be merged.
To achieve this, we could add a Build Validation in branch policy, then add a build pipeline to compare the number of StyleCop warnings on the two branches to be merged. If the number of StyleCop warnings does not decrease, then we set the result of the pipeline added in Build Validation to fail.
Related
My question is a continuation to this post:
Close work items automatically on Release to specific environment
This accepted answer will work perfectly, but only if I can make sure that a build shows just the delta since last build as associated WorkItems, instead of all work items from history. Sometimes I see all items in history as associated work items in a build.
Builds are happening for several environments (Dev, QA, UAT, Prod). How do I make sure that when I run a new build, it only has delta since the last build in that same environment so that I am only looking at new changes that are coming in with a new build?
Update:
I think I get what you mean. Please see if my understanding is accurate: your master branch has many PRs and links to many workitems. You create release1 from the master, and then when the branch is run based on release1 for the first time, the API lists the associated workitems of all the commits of the master. The second time, the incremental workitem compared to the first time can be displayed normally. Later, you created release2 from the master. When the branch was run based on release2 for the first time, the API listed the associated workitems of all commits of the master(This is not you want.). The second time, the incremental workitem compared to the first time can be displayed normally. . What you want is to display incremental workitems from the last run of the branch based on release1 the first time the branch is run based on release2?
If so, it's obviously not possible to use this API to achieve the requirements. As I said in the answer, this API fetch increment refers to the increment based on the same branch, it does not apply to different branches.
But you still have a way to get the "increment" you want, check out this API:
https://learn.microsoft.com/en-us/rest/api/azure/devops/build/builds/get-work-items-between-builds?view=azure-devops-rest-6.0
You just need to compare.
Original Answer:
but only if I can make sure that a build shows just the delta since
last build as associated WorkItems, instead of all work items from
history. Sometimes I see all items in history as associated work items
in a build.
Why do you say 'make sure'?
The API should only get the delta workitems(under the same branch).
I think changes should not contain all of the work items linked to all of the previous commits(since it is only 'changes').
Do you mean the behavior on your side is unstable, do you mean sometimes contains delta changes but sometimes contains all of the changes during the branch lifecycle?
If not, then the understood of yours maybe a little false.
If yes, I think you need to report this issue to the Developer Community. Please also provide your problematic build url there in this situation. StackOverflow is an open forum, so it's not suitable for handling stuff with private information.
Builds are happening for several environments (Dev, QA, UAT, Prod).
How do I make sure that when I run a new build, it only has delta
since the last build in that same environment so that I am only
looking at new changes that are coming in with a new build?
I suggest you put each environment in a different branch, then when you run pipeline based on the related branch, the pipeline should only get the delta changes of current branch and the API should only get the delta workitem commits since the last pipeline run of current branch.
Is there any way that let's me restrict merging two branches if one has less code coverage than what is expected? I use jest and added coverageThreshold in package.json file. But even though the coverage was less when I ran coverage, the branch was allowed to merge.
Couldn't find anything in Github settings for branch protection as well.
Branch protection rules are under Settings>Branches for the repo.
To add code coverage requirements, like anything else, you need to add pass/fail statuses to the github actions (or if you use a separate system like Jenkins, that needs to register statuses) and then say that the statuses are required in the rule. Not sure if you can do that on all github plans.
The project contains a test folder that is ignored by npm but is not ignored by GitHub. When a change occurs in a file under the test folder, should it be also published to npm in order to keep versions matching? Also, in that case, semantic versioning should be increased while there is no change for npm.
Assume that there is a repo in GitHub which has a test folder that is ignored by npm. It also has the package.json file which is tracking version number inside the repo.
Q1. When a change occurred in a file under the test folder, should the patch version number be increased?
Q2. Somehow (if the answer of Q1 is yes, as it is in the first question but there might be other similar cases), when a minor version increase happened in the package.json file, but nothing is changed in the files on npm side, what should be done?
Last edit first:
The short answer to your question is no. The repo will march on ahead of whatever is published in your feeds until you cut a new release.
So the problem here is that you apparently track version numbers either in a file in your repo, or in the commit labels/tags. Don't do that, it's pointless. I know it's a very common practice (I've made that mistake myself), but it is born out of lazy thinking. Repo's are not the right kind of database to track this sort of thing. Only published packages need to have any kind of version label on them. The data-flow should be from repo => build system => package repository. The arrows should never be reversed.
When you apply that rule, your question is moot. Your test content is a separate package feed from your release content (NPM). It already has repo hashes, that are unique, unambiguous and immutable. Those hashes should flow to the build system, then to the package feed/repository system.
There is always a difference in the release date/time stamps between test and product development. Product releases always lag test releases. The purpose of a particular version of the test suite, is to validate a product release. So you should always see test suite X.Y.Z+repoHashN, precede product version X.Y.Z+repoHashN. Note that the values of X, Y and Z for test and product, need not have the same values (a product patch, might be the result of fixing known bug surfaced by the test suite), but there should always be a 'repoHashN' that uniquely matches a test version triple and product version triple.
You produce two products, an app and a test suite, from the same repo, and most of the tooling out there tracks a version per branch that is applied to all packages produced from that branch. I can't directly address any specific NPM behavioral issues, I don't use it enough, but I am fairly sure you are not encountering a bug, rather you are most likely using it incorrectly.
Because your two products develop on separate cadences, you should consider maintaining a test branch for versioning test code. There's a seemingly endless variety of possibilities for your workflows here.
I recommend using a master or main branch for the tip of all development. Always cut prerelease (-<#>+master.<repoHash>) versions from this branch, and you might as well only ever bump either the patch number or the prerelease tag number (NPM supports either scenario). Then, when you are ready to cut a release, you fork master or main into a release branch (named after the target major.minor version) and cut only release packages out of that branch. Because your test code version is not relevant to the release package version, you don't need to track that specifically, it's always updated when you merge from main/master or test/dev branches to cut the next patch level release. The patch level of the release branch only gets bumped when you are ready to release it to the public.
Do test development in a test branch to hide the churn that doesn't need to show up in the master/main history. A dozen "BUG:### WIP" entries aren't needed there so you squash merge from test to master or main. Same thing holds for product code development. Do that in a Dev branch. Test and dev branches should only ever cut packages with something like a -<#>+<devNam>.<repoHash> tag on them and only published to private feeds.
The purpose of master/main branch is to provide frequent (at least daily) build and test cycles that include merged content from test and dev branches. This is where you maintain ground truth. At any given time you should be able to safely fork a new branch from here and count on it to build a product that meets your quality specs. This allows you the freedom to cut release branches at whatever cadence you need, independent of the current state of the test or dev branches. Test and dev, should attempt to merge their work into master or main, at least daily, and immediately fix any merge conflicts or build failures.
In the case where test == dev, you can combine them into a single branch. Generally, each developer will have many branches in progress, for independent work on various tasks. Having them merge directly to main or master is a valid workflow and even considered a best practice in many shops. Sometimes, too many cooks in the kitchen however, can cause problems, and when you find your dev's are spending too much time resolving merge conflicts with master or main. You'll want separate branches for different lines of effort, for them to stage their work into, before taking it to master. Then flow into master or main isn't quite so random, and will be easier to manage.
After reading and asking about how to deal with git, Github, branching, versioning and tagging, it looks like the best way to denote a new version of software, is by marking the last commit of a specific version with a specific tag.
A structure like this can be assumed:
Commit 1
Commit 2: Tag 1.0.0
Commit 3
Commit 4
Commit 5: Tag 1.1.0
Commit 6: Tag 1.1.0
Where once version 2.0.0 is released, most likely a 1.1.x branch is created to maintain the 1.1 release for hotfixes, etc.
Now I am wondering how it is possible to integrate this with a CI, like Travis for example.
What I do know is that:
A Continuous Integration system, like Travis CI, gets triggered on every commit to any branch, and it is possible to enable/disable this per branch.
I am struggling to figure out if it possible at all to start a build once you have tagged a certain commit, can anyone explain me how this can work?
The idea behind Travis and Continuous Integration systems in general is that first word: Continuous. You want it running on every commit because your tests should always be working (except when you know they're not). Running it on each tag isn't very CI-like, as it prevents the continuous part from happening, and defeats the entire point of it.
CI allows developers to catch bugs before they might've been noticed, even if your contribution docs say that you're supposed to have the build passing before pushing doesn't mean people listen. Travis, in particular, allows for lots of variation and multiple builds per commit with different setups, so you can test for a case that you might not think of when developing locally.
If you really just want to run it on each tag, you could write your build script to exit early if there wasn't a new tag defined. Travis defines a bunch of environment variables that could help you detect it.
a bit of background first...
I am setting up a versioning numbering system for our project which currently only has a development branch, but we are now moving towards our first deployment. We are using TFS and we use nightly builds on our dev branch.
The way we are probably going to go with this is that when we get ready for a release we take a branch off dev and call it 1.x. This shall be a test branch: we test it, fix it (then merge back to dev), test it some more then when it is all good we take another branch off the 1.x branch and call it 1.0. It is this branch that gets deployed to production. Any fixes in production will be made to the 1.x, tested and then a new branch 1.1 will be made.
My issue is with the testing of the 1.x branch. Before testing the branch will be locked for obvious reasons. My issue is that QA requires that a round of testing be conducted against a "version number" and if testing fails the next round of testing will be against a new "version number." Us developers want to tie the "version number" to the release and testing can iterate over that version...so there is a conflict here.
My first thought is to use the build number as the point in time that the code is tested against. When its time to submit a new version for testing, the 1.x branch is locked again and a build is kicked off and the VSTS number that is generated becomes a "release condidate 1 for v1.0." Mapping the RC to a build we can do manually in a spreadsheet...
...then someone mentions labels, and that the code should be locked labled and built prior to testing. I have never used labelling before and have just read that build itself creates a label in TFS.
I am now confused about what is the best way to go here. Is using a build number for a release candidate enough? Does manually labelling serve any purpose here (the only benefit I can see is that we can give it out own name and description)? Can I tell TFS NOT to create a label whenever it runs a build and just do all our own labelling at significant points in time (not every build is going to be a release candidate for example)? If so, is NOT creating a label after each build a bad idea, what does labelling give me?
I guess I am confused about where changsets, build numbers/names and labels all fit in with one another...
This is a broad question but its one of those ones where I am not 100% sure what to ask. Any help appreciated.
...then someone mentions labels, and that the code should be locked labled and built prior to testing. I have never used labelling before and have just read that build itself creates a label in TFS.
What you have read is correct. With TFS (unlike, say, SourceSafe), every server action constitutes a 'known point in time' which can later be referred to. You can see what I mean by doing a Get Specific Version... and looking in the Version dropdown: in TFS 2005 the relevant ones I see are Changeset, Date, Label. Now, as you correctly say, every build automatically creates a label. This means that at any future time it will be possible to retrieve the code exactly as it was after any given changeset; at any given date; and when any specific label was applied, thus including when any build was done.
The upshot is that you can use your own labels or not, entirely at your own discretion - the ability to retrieve a given snapshot of the code will be there anyway. I wouldn't suggest trying to inhibit TFS from producing a label for each build (I don't know if this is even possible) - labels cost you nothing.
Your branch 1.x is a consolidation branch which will contain many incremental small evolutions.
Locking the branch is not the answer.
Setting a label (specifically named "for QA test", and locked in order to not be able to move that label, is the usual way to signal the QA team they can build their own workspace and retrieve that exact label.
Then they can begin their tests against the code.
Creating a label after each build is not always practical, since not every build is meant to be tested by QA.