I have many Azure DevOps Extensions. Some recent, but also tasks dating back to 2015 when the extensibility model for build tasks first appeared. Over time much has changed in Azure Pipeline Land...
The biggest changes happened when we moved from UI based pipelines to YAML based pipelines. Some hidden internals of the tasks made their way into the light.
In the past a user would really only see the Task's Friendly Name and Description and Team Foundation Server would keep track of the real identity of the task through its TaskID, a GUID that was never supposed to change.
See a sample below:
"id": "3ca44a28-62de-4c60-8d77-a99065b95a8a",
"name": "VariableSetTask",
"friendlyName": "Set Variable",
"description": "Sets a variable.",
"author": "Jesse Houwing",
With the release of TFS 2017, tasks could be packaged into an extension which caused another identifying element to be created, the extension's publisher, extension id and the contribution ID:
{
"id": "jessehouwing-vsts-variable-tasks",
"name": "Variable Toolbox",
"publisher": "jessehouwing",
"contributions": [
{
"id": "vsts-variable-set",
"properties": {
"name": "vsts-variable-set"
}
}
]
}
These elements were also hidden in the age of UI based pipelines, but also made it to the surface when YAML came around.
steps:
- task: jessehouwing.jessehouwing-vsts-variable-tasks-dev.vsts-variable-set.VariableSetTask#2
As you can see the publisher, extensionid, contributionId and TaskName are now used to reference a Task, whether you insert them through the "View YAML" button:
When using the YAML assistant this will only happen when you have 2 tasks registered in your organization that have the same TaskName property.
Due to the fact that these values are now publicly visible, I've wanted to get rid of some of the bad naming I used in the past:
Get rid of the vsts- in the contribution-id
Remove Task from the TaskName property
Make everything lowercase to match Microsoft's own tasks
I'd love it even more if I could update the ExtensionId, which also references vsts- still.
And remove jessehouwing from the extensionId.
Changing names will of course have impact for existing YAML users, since all tasks are referenced by name, not by Id, but it would make my tasks easier to use for new users and for users moving from UI based pipelines to YAML.
Unfortunately, there are protections in place in the marketplace to prevent people from hijacking other people's extensions. This enforces a number of things with regards to the Task's Id: The Guid.
The Guid must be the same for all versions of a task published in an extension (for backwards compat one can publish a v1, v2, v3 etc in a single extension vsix so users get to chose when they move from A to B.
The Guid must be published in only a single public extension (no restriction applies to private extensions).
The Guid, for some reason, is tied to the ContributionId, whenever you change the ContributionId, you must change the TaskID.
You can't change the TaskID and the TaskName at the same time.
This essentially blocks people from changing the public identity of a Task once published in an extension it seems. When you try the marketplace will fail to validate your extension with an error message:
ERROR: Contribution Microsoft.VisualStudio.Services.TaskId.VARIABLE-TRANSFORM is re-using task ID of some other contribution from earlier version.
To publish the extension, change the task ID.
or
ERROR: All the task versions belonging to Contribution variable-transform should have the same task id.
The tasks at variable-transform/v1 (version 1.4.41) and variable-transform/v2 (version 2.0.41) do not have the same ids.
A possible workaround I see is to publish a new extension with a new Id, new Contribution Id's, new Task Ids and new TaskName, but that would require admin approval and installation of that new extension into any and all of the 6000+ users of the extension. That's an impossible ask.
I could accept that the ExtensionId is going to be impossible to fix and just add new tasks into the existing extensions, with new names, new Id's and all, but that will break the auto-update feature for all of the UI based users (I have no telemetry on how many of these there are), my guess is many, cause many of my tasks provide a UI to make working with script and command lines easier. The ability to seamlessly upgrade from one extension to the next is one of the things I've spent a lot of time on in the past. Adding aliases and sensible default values to input fields causes minimal fuss when upgrading from v1 to v2 for the Variable Tasks for example.
See the example below:
The input type changes from a input field to a dropdown, the YAML representation changes to a lower case for a number of fields and a number of new options appear based on the reflected value in the dropdown. All without the user having to do anything to get access to these new features.
If TFS/Azure DevOps doesn't recognize the Task as a new version of something in the pipeline, users of the UI based pipelines and old Release pipelines (and basically all TFVC users) will need to add a new copy of the task, copy all of the values from A to B and then remove the old task. I do not want to cause them that much pain.
I'm guessing the answer is going to be no, but have I missed any option to allow my users to seamlessly upgrade, yet allow me to keep improving the way new users have to type YAML to access these tasks? I'm not aware of any way to provide an alias identity for the Task itself (previous names, guids), nor for the contribution id... Similarly, I'm not aware of a way to get an extension containing build tasks renamed on the marketplace...
I think that you're already answered your own question:
Keep it this way.
Add new extension(s).
Yes, for a user it's a pain.
Especially when you rely on an admin approval for installing extensions.
You could offer a PowerShell script to migrate all tasks in the pipelines via the REST API of Azure DevOps. But that's still a lot of work and needs a certain technical knowledge (and permissions).
Microsoft tasks are the only one that are really short, but that's probably because it are native tasks and no extensions.
If you look at some extension names in the marketplace you're not the only one with this "issue". Maybe Microsoft will come up with an solution.
Related
We've been using environments for a while now for our dev and test instances. I like being able to go to the environment and seeing the commits and work items associated with the particular build/release but can't find a way to export them. Does anyone know of a way to export the workitems?
Using environments you also loose the ability to see which build/release a ticket went out in or have I missed something?
Your first and second question, kind of, have the same answer.
In short, you don't want to be using Environments to view which work items are associated with a release.
Azure DevOps has a built-in mechanism to view work items directly within the Release or Pipelines page (Depending on whether you're using classic or YAML).
Here is a great guide published by Microsoft regarding how work pipeline-based work item linking works, as well as how to retrieve work items associated with a release:
https://devblogs.microsoft.com/premier-developer/how-to-retrieve-all-work-items-associated-with-a-release-pipeline-using-azure-devops-api/
Update:
Specifically regarding how to view linked work items in Pipelines. You'll want to look under "Related":
Clicking on "x work items" will show you the full list of work items:
Is it possible to add additional mapping to Get Sources at runtime?
Like in a prejobexecution task?
We are currently using a Powershell script that determines which additional mappings to setup based on iteration, area and different business requirements, maps them to the current workspace and then runs tf get.
This works, however, the changesets and work items from the additional mappings are not linked to the run.
We have also tried a different approach, where a “starter”-pipeline runs the scripts and modifies another pipeline (updates the tfvcMapping) and then invokes it using a build completion trigger.
All changesets and work items are linked, however, the approach does not seem right.
Add additional mappings to Get Sources at runtime (Azure DevOps pipeline - TFVC)
I have encountered a issue very similar to yours before (I use git). Personally, I prefer your second solution, which saves all the linked information (changesets and work items) at the cost of an additional pipeline.
For the first way, just as you test, we will lose some relevant information, which is not what we expected. Although we can use the checkout command to get the latest changesets, we cannot simply complete it for workitems, because it is done by Azure devops. It is difficult for us to obtain the associated workitems through changesets and associate them with our build.
The solution for me, we create a pipeline(as you said starter-pipeline) to invoke the REST API Definitions - Update to update the get source for another pipeline, then hen add build completion trigger:
PUT https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=5.1
Check the similar request body here.
Hope this helps.
I come from using Rally and Pivotal Tracker. In both I could assign work items to Releases as a planning tool, and to have a historical record of those work items that were deployed.
Even with all the highly-specific guidance from Microsoft about Azure DevOps, it is silent how to organize work against future Releases. I can't see a place to even associate a work item to a Release. Is there something I am missing in all the documentation, or is there some workaround strategy more robust than just using tags to pro-actively plan against releases? Or is Microsoft's expectation that I use some separate product management tool to manage work items against Releases?
There are multiple techniques employed to accomplish this, not "one way":
use a parent iteration path that groups the iterations you plan against a certain release. This works best when you completely finish one release before starting the next. Otherwise, it usually becomes a mess with multiple active iterations.
Backlog Iteration
+ Release 1.0
+ Sprint 1
+ Sprint 2
+ Sprint 3
+ Release 2.0
+ etc
-
use tags for releases. Add a tag [Release 1.0] top all work items that are included in that release, this is one of the most flexible options.
use Azure Pipelines to keep track of which Work Items were associated to which Git commits and thus was included in which Build artifact. Track the build artifacts across environments to see which work items were deployed to an environment by looking at the intermediate builds.
add a custom work item field to the work item types you want to track. You can change the work item process being used and you can add a field to the work items in which you can specify the release name/number. There are custom controls available that can scope the version numbers to a specific list or can fetch the allowed values from any REST API.
Azure DevOps is more flexible in the usage patterns as you can see, but it also means that sometimes you need to "figure out what works best" for your team.
Another extension you may be interested in is the Bravo Notes extension. Or one of the other extensions that can generate Release Notes based on you work item data, commits and/or pipeline artefacts.
In Jira, you can create Releases for a project. As part of a Release, you can specify which issues are part of it as well as add Release notes.
These are extremely useful when you have automated build scripts, as the API for JIRA can be queried by the scripts as part of a CD pipeline.
You can therefore do things like (but not limited to):
Fill in a changelog dynamically
Stop deployment if there are an issues part of the Release that are not Done
Retrieve version numbers
Question: Is there a VSTS equivalent?
I do not think there is currently anything directly comparable to a Jira 'release' built into Azure Devops, which would allow you to package up completed Work Items on a board into a 'Release' work item.
You could implement a "poor man's" version of this by creating a custom process for your project that included a new 'Release' work item type. Each 'Release work item' could then be manually linked to work items you want to include in that release and could contain custom fields for the 'version' number or any other meta data you wanted to store with that release. This could then later be queried from a CD pipeline which, taking one of your examples, would allow you to do something like iterate over the linked work items for the release and make sure they are in a 'done' status.
Edit: As an example of integration techniques, the REST API for Azure DevOps supports a simple REST GET request to query all work items in a project for a custom work item type:
GET https://dev.azure.com/{organization}/{project}/_apis/wit/reporting/workitemrevisions?types={YourCustomWorkItemType}&includeLatestOnly=true&api-version=4.1
The API will also return any custom fields you have associated with this WIT, listing them under the key 'Custom.{YourFieldName}' within the "fields" object of the WIT response.
If your team was working with sprints, the other potential method I could think of to do this would be for each 'sprint' to become a named version which would become your 'release' once the sprint was complete. Work items that were not implemented as part of that sprint/version/release) would then be moved into the next sprint or closed. I'm not sure this approach would be very sustainable for complex projects.
There are features of interest listed on the Azure Devops Features Timeline that may improve this workflow in the near future (for example, "Release traceability – Work Item integration", planned for implementation in 2018 Q4), although it's difficult to find out any implementation details.
I'm trying to execute a bit different build steps in VSTS based on type how build was started: automatically or manually.
I'm especially interested in accessing that information from powershell script. But so far was not able to find suitable solution or workaround.
Did someone faced similar requirement before? How did you solved it? I would appreciate your help!
Seems you want to know whether the project build is happening through TFS triggered build or manually triggered build.
There is no such feature for vnext build for now. About this , you could submit your uservoice to this link, TFS Product Team is listening to your voice there.
As a workaround either to use two build definitions through different version patterns or manually add a specifical tag after a manually build finished. Through using tags to set some labels on the build to distinguish manual and automatic builds. But this is a manual action, it would be better if we can do this automatically.
It seems to be I've managed to find an option that allows to determine wherever build was triggered automatically or manually.
All builds started manually have actual user in $Env:BUILD_QUEUEDBY variable, while automatic builds have system account there. My value was [********]\Project Collection Service Accounts.
I don't know how reliable it is, but for me so far following code did the job:
# Identifying who triggered the build
$OwnerId = $Env:BUILD_QUEUEDBY;
$OwnerId = $OwnerId.ToUpper();
if ($OwnerId.EndsWith("PROJECT COLLECTION SERVICE ACCOUNTS"))
{
Write-Host "Build was triggered automatically. Resulting files considered 'BETA'"
}
else
{
Write-Host "Build was triggered manually. Resulting files considered 'STABLE'"
}